Dexter J,
I need to e-mail you the Saabvideo logs. I just received them from the system administrator in what seems to be a .gz compressed archive.
SG
Dexter J,
I need to e-mail you the Saabvideo logs. I just received them from the system administrator in what seems to be a .gz compressed archive.
SG
Actually - all you need is to pick up compression tool for the file. GZ is a type of zip file that should have your logs inside. Have a poke around here for an app to decompress:
Once they are decompressed, use a text editor (word wrap off) or spread sheet to review transactions with the server. Here is a fairly good start page regarding how read raw Apache Logs:
Most folks wimp out and go for a 'web log analyzer' - I don't recommend them until you at least have enough basic literacy to understand the raw log. The features on most log analyzers make *much* more sense if you understand what they are doing for you.
So, very basically, each visit should generate several log lines as each page item is passed to the visitor. You are looking for anomalies in the pattern where the visiting IP *doesn't* read the top page and all it's images first - and/or - repeated visits from the same IP on any kind of schedule - but most importantly - video downloads without any corresponding top page visit.
Towards the end of each log entry the 'user agent' is listed. You should see visits from google, yahoo et al (relatively useful bots/slurpers). They usually only take the page text body - but not always. Pay attention to the detail here, because I'm still betting you are seeing a pile of visits from things like Wget, DA, psbot and a bunch of others. Here is a good page for reviewing bots and their place in your universe:
As I have said previously, I don't think it's worth your money to hire me for what is simply a hobby site on a shared server. I dearly wish you had a fat sponsor and it was worth the cash because I like the idea of your site, but I guess the best you are going to be able to do is enough reading to stop the obvious packet leaks like most everyone else.
I will again repeat my advise as regards your basic architecture though.
You should at least look at a teaser index page with several video specific sub-pages as your top page now runs better than 1.2 megs every time it's viewed without downloading any videos. While you are at it - you might want to look at changing links to specific video's you have already posted out here - it will help you narrow down who is swiping your vids (they will now appear in the error logs as dead link attempts) and it should slow down the transfer totals for a while.
Dexter J wrote: While you are at it - you
I don't know much html, and no Apache, so this is a dumb question, but would there be an easy way for Ryan to eliminate the href on the thumbnail and replace it with a "download" button to keep people from just clicking and viewing live or casually viewing that way? I'm thinking he could force users to save it to view it, and to hide the videos from web bots.
Yes - he could drop the vids into a couple of passworded sub-directories on the site - then provide a password hint in the message box that pops up when a user clicks through to download. Then make an adjustment to the mime types to attempt to force an application.vnd exchange to a known null client.
Not fool proof by any means and not everyone 'gets it' when they come across the password hint in the logon box - but it does force a somewhat more targeted automated slurp and mostly forces a download (this latter part being driven much more by the particular browers visiting). Do not include reference to the password and ID in the page link other than to say there is one and how to figure it out.
It's only a stop-gap measure though, pro-slurpers are clever beans. The correct way to go right now is to use mod-rewrite and a cgi so that the system demands several parameters be met in addition to hitting the link and filling out the password.
Someone is going to suggest an FTP server somewhere this process. I
*highly* recommend *against* it at this point in time and have dropped the standalone FTP server as part of the Mark-III build here at Radio Free Dexterdyne.There are several very serious things going on regarding the FTP cracking and the firewall worlds right now that is rapidly making FTP server technology both unsafe and unreliable in production. I built a backend cgi that performs the all same functions as FTP that, while not as easily load-balanced as pure FTP, is *substantially* less open to exploit and more reliable through most firewalls.
If you do go with the hinted passworded/mime type .htaccess kit - you want to cycle the passwords every month or two. This is sort of labour intensive on a bigger site. Not that much of a hassle for a small hobbie site though.
Still - at the end of the say - it's all in the log work though. Know what's supposed to happen there in a normal exchange and it makes it
*much* easier to deal with exceptions.Dexter,
I don't have the time now to read the logs, I thought you were offering your time to help given that I have spent approximately 280+ hours to create
SG
time to help given that I have spent approximately 280+ hours to create
You know what? You're a real pain in the arse. Someone tries to help you and all you can do is come back with is more requests. You should really try to understand that
time to help given that I have spent approximately 280+ hours to create
Grunff,
I just thought I was being nice. :)
Sheesh Grunff. :)
SG
Salutations:
While I don't think I would be quiet as forthright as brother Grunff, I feel that I should point out that at 500g a month, you don't really have time *not* to look at your logs.
I suppose it is of very little passing interest to most normal people like yourself with real lives more than 25 feet away from a stinking ethernet jack - but I have better than four years into Radio Free Dexterdyne (my hobby solution) and all the hoopy things it turns out it can do.
As you don't know me from Adam, I feel I should also tell you that I've actually been at this i-net thing since the late 80's here in Canada when it was still a 300 baud TTL University network us geeks used to pass French Postcards around on between systems dumps for the chem labs. I'm not trying to brag or anything - but I want to set some context for the comments below.
I fear sir that you are making a mistake that *so many* good basic ideas for websites make early on. While I may be dead wrong - I'm pretty sure that less than 30% of your total transfer represents real human beings in front of a computer at your actual site.
Don't feel bad though - that was what the whole 'internet bubble' was really all about anyway.
So what advise I have provided, which I'm afraid you pretty much have all of until you can make some time to analyze your own logs and come back with some deeper questions not normally available through google, is as posted here so far.
Now, all that said, please don't feel I'm trying to be rude or that I am being dismissive of your work. More than most people, I know exactly how tough (and expensive) it is to get actual honest new content online in the any context (hobby or professional).
Which is exactly why I think your content is being piked by the pirates, most of whom have much better advertising sales staff than you and why I have been trying to give you some real leads before you throw up your hands in disgust and walk away from it all.
All this said - my grandfather never allowed a radio or television in the house on the deeply held premise that furniture shouldn't speak or sing. He lived to be 75 on a pack of filterless Craven-A and a pint of Gin a day all his adult life.
This internet thing isn't for everyone despite, as I previously mentioned, the common wisdom that any 6 year can slap up a full on Multi-Media portal and become the next iTunes without ever really trying.
Sadly, like most things in life, that is just no longer true - because several of the 6 year olds have figured out how to hijack other people's content before the site owners can ever make a profit on it themselves.
Dexter,
That is all good news. I will pass this along to the sys admin for the site :)
Thank you! :) I do appreciate your time and help.
SG
You know me Dex, especially after a hard day at the office ;-)
time to help given that I have spent approximately 280+ hours to create
Ayup.
Me too brother Grunff - me too.
Again, my grandfather lived to be 75 on a pack of Craven A filterless and a pint of Gin daily and did not allow a Radio or a television in the house. He was a QC here in Nova Scotia and so never spent any real time at exercise.
So - in the face of the obvious life shortening aspects associated with the former - I sometimes wonder if he may have known a fair bit more about technology than you and I given the latter.
Cheers brother Grunff.
Bro Dex,
You know, I have considered this life philosophy stuff at great lengths, even before your mentioning it. I am convinced, though not solely through your evidence, that human kind is by far its own worst enemy. It surely seems that a simple but pleasurable existence is the shortest route to fulfillment. But we insist on cluttering our lives up with all sorts of unnecessary detritus.
As I approach the mid century mark in my life I am leaning further toward the attitude of living for the moment. I just picked up a fine motorbike, a naked Ducati 750. Hell yeah, I figure if I get creamed out on the road by some "brain-dead booger-head", I already have good (enough) insurance and my kids are already college educated and will prosper. In the meanwhile, I think I'll do whatever damn well pleases me and get on with what remains of my mortal existence. Especially since I remain somewhat uncommitted as to anything beyond it...
Does anyone else find even the least bit of irony in the fact that most old folks (yeah, I'm talking older than me) are far more health conscious than the 20 somethings?
How any of these previous rambling relates to the tedious whining of Mr SaabVideo in this thread is a stretch at best. Sorry... Now back to the Saab Video Channel...
your time to help given that I have spent approximately 280+ hours to create
Dave,
No, I was just being nice to you folks, but barely anyone thanked me for it.
That's ridiculous. Those who have downloaded the movies and never thanked me, should feel a bit guilty.
SG
your time to help given that I have spent approximately 280+ hours to create
should feel a bit guilty.
Saab Guy,
No, you should be feeling guilty for commandeering and dominating this group. Your previous behavior shows you are highly unlikely to do this, but perhaps your traffic is sufficient to justify creating a couple of new groups:
alt.videos.saab For your periodic posting notices, requests for binaries, and IT consulting inquiries
alt.fan.saab-guy For your frequent, pouty demands for ego massages and other miscellaneous approval-seeking
There are many useful, fun, and/or interesting Web sites relating to Saabs. Few of the webmasters of such sites post here demanding cash contributions, free technical services, and bronze statues built in their honor. So I'm afraid all I can say is, "Saab Guy, you're doing a heck of a job."
_________________________________________________________________ Messages sent to the address in the header will be ignored. Personal replies should be directed to user k.l.bailey at the Institute of Electrical and Electronics Engineers.
your time to help given that I have spent approximately 280+ hours to create
should feel a bit guilty.
Why should anyone feel any *GUILT* ! ?
It was *your* project. No-one asked you to do it. It's *your* baby.
Some posters here have sensibly suggested you reduce the file sizes too and you've ignored them. No surprise then that the site is a bandwidth hog. Although I personally have no intention of downloading 2.5G of video just to see a promo for example.
Now get over it.
Graham
Ok, I was going to leave this but no, you seem on a mission to be praised for something that perhaps wasn't even asked for. If you choose to embark on a mission spending your time and money and someone doesn't thank you for it then you have to ask if they wanted it?
Just because someone has downloaded a video doesn't actually mean they
*wanted* you to do it, just that it was there, novelty factor, lets have a look, bit of fun.Everytime someone hits my webpage, they don't send me an email, thanking me for the information, every time they download something, they don't send me an email thanking me for making it available. That's just not the way the web works.
should feel a bit guilty.
Why? Sorry dude, you spent your time and "offered" it to the world, don't expect everyone to fall over themselves thanking you!
FWIW, the start page takes way too long to load because you've got thumbnail after thumbnail and you've already been told about this.
The size of the videos is such that I can't be bothered to download more than the one that I did. That one took several hours to arrive and I am just not that excited about it. If they were much smaller then fine but as I have said before, for me, I'll just watch it on the PC, no way am I going to spend more time authoring a DVD in order to watch an advert, I have enough of those already beamed at me all day.
You have been offered various solutions about how to improve things, Dexter has already commented on the home page, you've had suggestions about a thumbmail and maybe a sample clip which then links to a bittorrent source...
...BUT YOU'RE STILL NOT HAPPY!!!!
You now want people to thank you every time they download a file?
I give up.
David.
should feel a bit guilty.
Yes, YES, we understand. You put up a website with questionably legal downloads of material you clearly don't own. I'm sure it's a very _nice_ website of questionably legal download files.
Then you whined about hosting cost, and several people suggested that the numbers you were mentioning were way high. You ignored us.
Then you whined about bandwidth cost, and several people suggested bittorrent. You ignored them.
Then you whined about weblogs, and at least one person spent a non-trivial amount of time giving you free advice on how to use them to identify your bandwidth problem. Instead of taking that advice and using it yourself, apparently you feel that "Hey, this guy is good for free consulting; he knows how to do it, let _him_ do all the work". I recognized the pattern early on, and 20 years ago I had already decided that that wasn't a game I'd play any more.
(snip)
He'd be paying less for hosting if he had taken the free advice offered. He'd be paying less for bandwidth if he had done any of _several_ pieces of advice freely offered.
He'd know how many people are enjoying his site if he'd look at his freaking logs. News flash: website visitors rarely send fan mail. If it sucks and they hate it, they just don't come back. Hits is all there is, bucko.
Dave Hinz
Salutations:
OK brothers - I think I'm more than a little uncomfortable with the flaming SG is taking in a thread with my nick on it. Deserved or not.
I feel, though I guess his site is sort of touchy point for many here, that this thread is still not entirely in the best tradition of this particular NewsGroup - one of the best on the i-net in my books, as I have said on many occasions.
So, the greater points having been well made about netiquette, I sort of feel that I should comment in defence of SG that everyone is a newbie at something sometime. That is why I offered the advise in the first place.
Brothers - I really do find the idea of a site dedicated to archival advertising footage from SAAB sort of interesting. Sort of like the Prelinger archives if anyone has time to visit there. As it remains sponsor free - all I was trying to offer were some leads on how better to operate it so it doesn't disappear from the firmament.
I happen to have one last, hopefully thread ending, link to offer with my apols that it's on my own hobby site.
I try to not offer links to my own site other than Top Tune o'be-do-da-day in my sig for a lot of good reasons - not the least of which being that it always feels like I'm pitching business with my electro-pals here. Which is always a black-balling offence in my books too and why you are taking such a beating here regarding your plea for help as regards saabvideos.
So - SG - grap yourself a big coffee and have a careful read over this:
The Village Bicycle - Basic HTML
A lot of folks swipe the html source (right click and save as) on that page - use it as a template for liquid tabular layout if you like. I ask that folks leave a link to Radio Free Dexterdyne somewhere if they do - but most that have nicked the source are in a spot where it would be difficult to explain to their pay masters why they had to swipe a source in the first place.
That's alright by me too I guess, anyone (including me) who ever starts to brag that they never poked around a third party html source for their own sites is lying.
So - please brothers - could I again ask that this unfortunate thread be put to bed? I will refrain from offering WWW advise here in future so as not to stir the pot again. Thank you.
MotorsForum website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.