Google upgrades the Web

Saturday, August 18, 2001 by Dave Winer.

Google and weblogs 

Quietly, without much fanfare, Google is upgrading the Web.

They're indexing lots of weblogs, including Scripting News, every day.

I've been asking for this feature since 1996!

http://davenet.userland.com/1996/09/07/FloatingIdeas

I call it a JIT-SE or Just-In-Time-Search-Engine. The JIT-SE feature is particularly suited to weblogs, which are time-oriented websites. The Google crawler notices that I update my site every day, so it knows it should come back and re-index my site every day. It finds lots of links to new stuff. Presumably it indexes those pages too.

They incorporate knowledge of past behavior in the algorithm that determines how often they index sites. We use the same technique in several UserLand Web services, and it's a good heuristic, it offers a reasonable tradeoff between scaling and currency. Beautiful.

What this means to users 

We get a much more current view of the Web through Google.

Here are a couple of examples.

1. Evan Williams, the author of Blogger, started working publicly on its XML-RPC interface on August 6. A Google search reveals that a lot of work has been done by a new community in the last twelve days. In the past there would have been no way to watch the community build through a search engine.

http://www.google.com/search?q=blogger+xml-rpc

2. Here's a Google search for Gassee and Palm. It reveals dozens of stories about Palm's acquisition of Be, which was announced on Wednesday of this week.

http://www.google.com/search?q=gassee+palm

A more responsive Web 

This is a very noteworthy milestone.

Thank you Google.

Dave Winer