Home > Weblog Archive >  2007 >  April >  13


Sitemaps, day 3

Friday, April 13, 2007 by Dave Winer.

Since I've been playing with sitemaps, of course I created one for the RSS 2.0 site. Permalink to this paragraph

And I've checked to see that the maps I deployed for scripting.com are properly updating, and they are. Permalink to this paragraph

But when I checked, I realized that I would have done it differently, so that the sitemaps, in adition to helping search engine crawlers, might be interesting things for human beings to read as well. Permalink to this paragraph

I refer back to sitechanges.txt, a simple project I was doing in 1997 that was like sitemaps. It was also before I did XML. ;-> Permalink to this paragraph

A picture named goodbye.gifThe idea was that the content server was responsible for providing a daily reverse-chronologic list of pages that had changed. Then a crawler would keep track of when it had last visited my site, and only suck down the files that had changed since then. This would enable search engines to be more efficient, and provide more current content. It was nice because you could read it yourself and see what had changed. Contrast this with sitemaps, where you have to go hunting for the changes, it's no better a user interface for finding the new and newly updated stuff than the file sytstem is. I was kind of disappointed. Permalink to this paragraph

Another thing I would have done differently is allowed sitemaps to include other sitemaps. There really is no need for two file types, just let me link to an index from an index, much like inclusion in OPML 2.0. This added an extra layer of complexity for everyone implmenting sitemaps on moderately large sites, or old ones where some content changes frequently and other content not so frequently (like scripting.com). Permalink to this paragraph

However, on balance, it's a great thing that all these companies got together and did something to make the web work better. We need more of that!  Permalink to this paragraph

If anyone is working on more stuff like this, I am available to review it before it's cast in stone.  Permalink to this paragraph

© Copyright 1994-2007 Dave Winer Mailto icon.

Last update: 4/26/2007; 12:44:59 PM Pacific. "It's even worse than it appears."

Click here to view blogs commenting on  RSS 2.0 feed.