Optimizing Flickr RSS reads

Monday, October 1, 2007 by Dave Winer.

There's a lot you can do with the Flickr API, I've pretty much mastered it, and it may look a bit intimidating at first, but it's well designed, and once you learn how to do one set of calls you pretty much know how to do the others. I would have relied more heavily on XML-RPC encoding for uploading pictures, but now that I've been uploading pictures for a couple of months, the pain of developing that glue is fading.  

BTW, I've got glue for the OPML Editor, it should work well in Radio or Frontier. I've exported a snapshot of the system.verbs.apps.Flickr, for all to use. It may be included in any distribution. (To be clear, OPML Editor users don't need this, it's already in the update stream, just choose Update opml.root from the File menu.) 

Anyway, I want to expand the Flickr/Twitter experiment, but before I do that I wanted to check with the Flickr folk because this service does a lot of polling of their RSS feeds. I suggested it might make sense to create a service that would allow a caller to find out which of the feeds its interested in have changed since the last time it checked.  

They responded with a simpler feature, they implemented a Last-Modified header on the RSS feeds, so that you could skip processing if a feed you're interested in hadn't changed. I've now updated my code so it does that, and everything seems to work. You'll find the script at Flickr.readFeed. Here's a text listing of the script.