A strange confluence of events. I was writing a series about rebooting RSS. Then I had a bike accident that left me dazed and in pain. One evening I woke up and found I could sit for a few hours in front of the computer without pain. That was pretty cool, because even lying in bed was excruciating. My poor body was in need of some rest. Programming could provide it.

So there I was in front of the machine. Now what? I could write some code. But what?

I had been playing with long-polling for instant outlining, with excellent results. I wonder if the same could be done for feeds. I had found an OPML file on the NY Times site with all their feeds, so I hooked it up to River2. Now I had a relatively realtime flow of every story from the Times. So I wrote a minimal server that would wait until something new showed up and then return it. (Later I wrote a filter that eliminated duplicate titles, so even if a story appeared in two feeds it would only be shot out through the hose once.)

A small embellishment, let the caller give me a "seed" that indicated where he left off. If any new items came in while he was processing the last ones I returned, he wouldn't miss any. Every packet includes a seed which you can send back to me.

Then I wrote a client. Then I added another stream, and broadened my client so it could handle multiple streams.

The result is a nice sweet little protocol that behaves like two familiar things:

1. A long-poll server.

2. RSS.

It's pretty cool. Let me know if you're writing some scripts to test it out. The format is described below.

Here's an example of a query:

http://hose.scripting.com/?name=nytimes&timeout=3.

Click the link. It'll just sit there till it times out after 3 seconds or, if a new item comes in before the timeout, it will return with that item.

Here's an example of a response with some items. There are two forks to the XML, items and metadata.

items are just a series of RSS items (or Atom entries, for feeds that are in Atom format). However we add three elements to each item, feedUrl, feedTitle and feedLink. My app, on the other end of the hose, needed this info. Seemed reasonable to provide it since I had it there.

In metadata, there are two elements, now -- might add others in the future. seed is the one you need to include as a parameter in your next call. If the seed returned is xxx, then your next call would be http://hose.scripting.com/?name=nytimes&seed=xxx.

The seed is supposed to be opaque, that is -- you shouldn't expect to understand its structure, but you can see mine is pretty straightforward. It's a walk of the hierarchy in my object database. Year-month-day-serialnum. So 2010-09-30-00651 is talking about the 651st news item on Sept 30 (today).

One more thing, I wrote a handler that returns the current seed in case you want to start off with a seed. You don't have to, but your code might be a little simpler if you have it to start out with. http://hose.scripting.com/seed

Update: I've released the source code for the Frontier tool that implements both sides of the protocol.

 9/30/2010; 7:33:50 PM.   .

A couple of possibly final thoughts about Angelgate and as a bonus some thoughts about the value of money.

1. I started my first Silicon Valley company in the early 80s with angel money. Then, and now, it was a collegial thing. A bunch of guys (they're almost all men) play at entrepreneurship the way other men play ponies. They're not going to live the life of the 20-something. They couldn't, it's too much stress, and they have lives, and more important they have cash. It's hard to push that hard when you aren't betting it all. The young guy (he's almost always a guy, too) is like a racehorse. The angels all get to hang around the stables, and watch the game closeup. The price? As little as $10K or as much as $500K, depending how much of a thrill you like.

When I was coming up, they had their meetings at the Stagecoach Restaurant on Woodside Road. It's long-gone now. It was a dark, smoky greasepit of place. While I was raising money I'd have breakfast there every morning and if one of the angels showed up I'd pitch him. I'd call my chief angel from the office later and he'd go over and see how I did. Invariably he'd come back with a check. We raised a couple of million this way.

I don't think there was any collusion back then. The angels sure talked to each other, all the time, but you know what, that's allowed. There's no law against talking. My angels did try a couple of rebellious things, but it was mostly because they didn't understand how software worked. Software was a relatively new business then, and retail software, which is what we made, was completely new. I chalk it up to confusion, which was my fault -- I was their leader. In any case it all had a happy ending. We sold the company in a big stock deal and went public and got rich.

2. What we didn't know while Angelgate was playing out, that Mike did know -- is that he was getting close to completing the sale of his company to AOL. Note that I didn't say he knew he was selling to AOL, because until the deal is signed you never know. But in that light, it could take on greater significance. It could be Mike declaring his independence from the financial community of Silicon Valley. It's easy to see that before AOL owned it, TechCrunch would be subject to some amount of pushing around by the financial people. If he was on the outs with any of them, he would lose an important flow of information, that would possibly go to one of his competitors. Mike's ace were his sources in the VC and angel communities, and in the upper echelons of management at the big tech companies and startups.

But now that his publication is in league with Engadget, they're going to get their stories no matter what, it seems. And despite what everyone says about Mike staying on, the longevity of founders in acquisitions usually is limited. And Mike is a typical founder (that's neither good or bad, it just is). I stayed at the company that acquired mine for six months. And a lot of that time was spent at home, plotting my next moves. If you're the kind of guy who starts things, the endless corporate activities of the company that acquired you seem terribly off-topic, almost to the point of irrelevance. Give me something to chew on! I kept thinking. I'm sure Mike will too.

3. It occurs to me that this may be the first big pile of money that Mike has gotten. If so, it must be a wonderful feeling. I remember when that happened for me. I sold a bit of my stock to the VCs who (for all practical purposes) owned the company that bought mine. It was, in the end a small part of the money I received, but none of it mattered as much as that very first check. It was for $425,000. I had to sit down. My legs were weak. Literally. It was at that moment that I realized that my days of being poor were over. That my struggle was over. I had attained a huge safety net. There was a level below which I wouldn't fall. Until that point, I had no such feeling.

After that were years of confusion, until I learned something very important.

Money is a good thing to have, up to a point. But it can only buy you one thing that's worth more than all the rest -- your time. After that, the returns diminish rapidly. Sure it's nice to drive a luxurious car, and live close to where it's all happening. Those are also good things. But if the choice is to give up yourself to have those things, there would be no choice.

There were other things I learned from not wanting for money. Live a simpler life. Every time you have a choice to complicate things, choose the other way. I've made the more-complicated choice many times. In hindsight, the good decisions were ones that kept things simple.

One final thought -- money does not buy you security. It can't. We are not secure. If you get rich and wonder why you don't feel secure, that's why. If you're poor thinking the rich folk feel secure, they don't. For more on that, I wrote a piece 10 years ago called Transcendental Money.

 9/30/2010; 9:20:53 AM.   .

LEV stands for The Local East Village.

It's a news site run by the NY Times and NYU.

I am an adivser to this project in my capacity as a Visiting Scholar at NYU.

One of the first things I did was put together an aggregator of all the East Village news sources I could find, to try to quickly get an idea of what was already there. This has proved useful to the team, so they've added it to the navigation bar at the top of every page on the site.

I think it will really start being effective when:

1. We really start to feel the pulse of the community through the aggregator.

2. When the bloggers of the East Village start to get traffic from both LEV and the aggregator.

The second is more important, because flow is the currency of the web, it's the ultimate respect. Pointers are nice, but traffic is even nicer.

But to get to #2 you have to first, imho, get to #1. Along with their course credit and a grade, our students should also come away with a feel for the community they covered, and probably more important if they're going to have a career in journalism, what it means to have the feel for a community.

Right now it's still the early days.

PS: I used the River2 aggregator, which I wrote myself. It's pretty easy to set up and it's free. I recommend it to any news organization that wants to keep in touch with the community it's serving.

PPS: This is how NYU is viewed by some in the East Village.

 9/30/2010; 5:11:10 AM.   .