Click here to show or hide the menubar.

Home >  Archive >  2010 >  October >  13

Previous / Next

Eric Kidd's feedhose client in Javascript
By Dave Winer on Wednesday, October 13, 2010 at 7:41 AM.

I love doing projects with Eric Kidd. We've never actually met face to face, but he helped enormously with XML-RPC and RSS when they were in their formative stages. Eric did the C implementation of XML-RPC. Really did it right and it stood the test of time. permalink

A picture named tennis.gifWell, we're playing tennis again. I hit the ball over the net, he hits it back, etc etc. In this game you try to give the other guy a clear shot, and he hits a line drive back to you. Nice hard hit, no spin, nothing up the sleeve.  permalink

I've been working on a protocol called FeedHose. As input it takes RSS and Atom feeds, and on the other end, it produces a stream of items. The key new thing here is that you connect to it via a long-poll request. The server returns as soon as a new item is available. None of the usual polling, and none of the latency that comes with it. Truly realtime! And it freaking works. (Many thanks to all the people who pioneered long-polling before I started doing it. I'm standing on the shoulders of giants.) permalink

It caught Eric's eye, and imagination, when JSON support went in.  permalink

So along comes Eric and he says: "I can make that scale." permalink

He builds a node.js app, works at making it scale.  permalink

He thinks his app, which fits on as a sort of "nozzle" to my hose, can spray items at tens if not hundreds of thousands of clients. That's really fcuking cool. :-) permalink

A picture named diagramSmall.gif permalink

The picture above illustrates the architecture. But the proof is in the use: permalink  permalink

This is a demo system, but it's useful because it gives you the news from the NY Times, as it becomes available. permalink

The goal is to get many of these hoses running, for all kinds of publications, and for new kinds (microblogs). The key point is that the updates flow in real time, yet there is no single central server. Want to build a decentralized network that works like Twitter? You this to make it work. permalink

This is on the road to the decentralized news system of the future.  permalink

If you run a realtime news organization, let's look at making your feed technology reflect that timeliness. :-) permalink

RSS feed for Scripting News
This site contributes to the community river.

© Copyright 1997-2012 Dave Winer. Last update: Wednesday, October 13, 2010 at 8:23 AM Eastern. Last build: 8/26/2012; 5:58:29 PM. "It's even worse than it appears."

RSS feed for Scripting News

Previous / Next