Full disclosure (of useless Web statistics), Dec 2004

« Another one switches to WordPress | Main | Seen elsewhere: Life without Firefox… »

Full disclosure (of useless Web statistics), Dec 2004

January 2, 2005

How fun, starting a new year with a bland post on Web statistics.

December 2004 was a record month for this blog, with record highs in visits, pages and bandwidth. I exceeded my monthly transfer limit of 20GB by just a little bit as well.

Getting an average of 4.7K visits per day, or 10K pages per day.

Thanks to the “Homer Simpson uses tabbed browsing” post, transfer spiked to almost 3GB on Dec 15.

Top referrers included SpreadFirefox.com and Bloglines. There was a referrer spammer (brushed out in the screenshot) which made it to the top 10 before I caught it and blocked it.

Here’s how referrers are blocked (with .htaccess), for those who are still looking for a solution. What these 2 rewrite rules do is to, respectively, rewrite the listed IP addresses, and rewrite the referrers with the following strings in them, back to the client’s IP. This works extremely well, and I know because I’ve had my server (a VPS) brought to its knees from constant referrer spamming. The hit rates from these spammers were so high that it practically became a DOS attack. Since I’ve implemented these rewrite rules, all has been peaceful and I can finally see uptimes of more than 3 days (believe it or not, the VPS did go down that often).

RewriteCond %{REMOTE_ADDR} ^195.175.37.26 [OR]
RewriteCond %{REMOTE_ADDR} ^195.175.37.24
RewriteRule ^.* http://%{REMOTE_ADDR}/ [L]

RewriteCond %{HTTP_REFERER} texasholdem [OR]
RewriteCond %{HTTP_REFERER} example\.com [OR]
RewriteCond %{HTTP_REFERER} example2\.com
RewriteRule ^.* http://%{REMOTE_ADDR}/ [L]

The referrer spammers are still going at it (though they don’t appear in AWStats) - I can see them hitting me when I check my server-status page (see the mod_status module).

I don’t think I’ll need to upgrade my hosting plan yet, since the bandwidth spike was a one time thing caused by the crazy Homer Simpson entry.

And here’s the top 10 browsers as requested. Unsurprising that Firefox tops the list.

Well, here’s to a better year ahead for everyone. For myself, I look forward to having a good time at my new job, (finally) getting published, finding religion, and finally getting a move on in life. I’m not one for “Happy New Year”s and new year resolutions though. So blah, have fun.

Look me up in the Cenarius server if any of you are playing World of Warcraft - my main character’s Lucita (gnome rogue).

Posted by Chu Yeow at January 2, 2005 SGT |

Permalink |

| Category: Blogging, Personal

14 Comments & TrackBacks ()

Wow, you sure as hell do get lots of traffic! Thanks for the .htaccess code, too.

I was wondering which Stats Program you’re using?

Posted by: Mathias Bynens on January 2, 2005 5pm

Mathias Bynens, it is undoubtedly AWStats…

Posted by: minghong on January 2, 2005 6pm

Thanks for the tips on blocking referrers… I’m not getting hit as hard as you seem to be, but it’s still annoying, so I’ll add a few rules to my .htaccess file.

Mathias: Looks like AWStats, which is what I’m using too.

Posted by: Roger Johansson on January 2, 2005 6pm

No problem… Anything to defeat those damned referrer spammers. A complete waste of bandwidth as well. I’m just glad that I’ve found a solution to the problem without an impact on my bandwidth.

They are still going at it (they don’t appear in AWStats) - I can see them hitting me when I check my server-status page (see mod_status).

Yup Mathias, it is AWStats.

Posted by: Cheah Chu Yeow on January 2, 2005 8pm

It seems everybody’s using AWStats. If it’s really that good, I should jump on the bandwagon, or at least try it out. Thanks for the tip.

Posted by: Mathias Bynens on January 2, 2005 9pm

It’s rather good, but don’t expect stuff like click trails.

You can check out the demo.

Posted by: Cheah Chu Yeow on January 2, 2005 10pm

Wow, that’s quite a lot of uniques, hits and traffic.

Thanks for those rewrite rules, I will add that to my blog now :-)

Posted by: Ingoal on January 2, 2005 10pm

Congrats.

Posted by: Patrick on January 2, 2005 11pm

If you want to see users flow I recommend you check out the the webdruid

(not affiliated with them, it’s FOSS anyways)

Posted by: dood on January 3, 2005 12am

Do you have web browser agent stats too?

Would be interesting to see how Firefox ranks on your blog :)

Posted by: David Crick on January 4, 2005 8pm

David: You got it :)

Posted by: Cheah Chu Yeow on January 4, 2005 8pm

Wow, it’s amazing to see how much your traffic has rocketed when comparing with your stats for December 2003.

Good work Chu Yeow!

Posted by: Ben Milleare on January 4, 2005 8pm

I’ve recently implimented a traffic monitoring feature on my website, viewable at http://traffic.dogtoe.com

Instead of parsing through the apache log file which awstats and many others so, you include some code on each of your pages, and it creates its own log files. For me, it provides “real” usage statistics because I don’t really care about the hits to images or the wide-range (and very random) files that I have uploaded. It too shows referers and other information (bots, browsers, “unique” hits, etc.)

Worth checking out.

Posted by: Brian on January 8, 2005 12pm