Why RSS isn’t always a good thing
Via Doc Searls: Gary Lawrence Murphy’s The End of RSS complains about the excessive bandwidth used by his RSS feed. Dan Sugalski also complains. They don’t know the half of it.
The Quotations Page offers RSS feeds to syndicate daily quotes. My logs show 74,257 requests for these files on a single day last week. Most downloaded the entire file despite the fact that it changes only once every 24 hours. Based on this, the RSS feeds use 157 MB of bandwidth per day. This is negligible to me (the rest of this busy site uses almost 5 GB per day) but I’ve had to do quite a bit of tweaking over the years to keep the sheer number of RSS requests from overwhelming the server.
In my case, a large part of the problem is Ximian Evolution, an information manager for the GNOME linux desktop. My feeds are included by default in every installation, which resulted in an effective distributed DOS attack against my site until I took measures against it. Thousands of sites using this software poll my site every 5 minutes.
Nearly 65% of my RSS requests are from Evolution. I have configured Apache to return a 403 error code to these requests. I hate to make the feed useless for these clients, but I had no other choice since my bug reports to the Evolution coders have been consistently ignored, and it will cut my RSS bandwidth in half.
[Note to administrators who run across this: Evolution uses the gnome-vfs library to make its HTTP requests in earlier versions and the Soup library in more recent versions. Look for these user agents in your logs to see if you’re a fellow victim.]