Scripting News

Why Flickr should do Realtime RSS

Fri, 02 Oct 2009 02:57:47 GMT

1. Flickr comes from Yahoo.

2. Yahoo used to be a source of innovation. These days --> not so much.

3. Twitter defines realtime.

4. Twitter doesn't do pictures.

5. Flickr is all about pictures.

6. Flickr already supports RSS, nicely.

7. RSS lets anyone play realtime.

8. Flickr gets to do it more openly than Twitter.

9. With pictures.

10. It's innovative and new and interesting and pretty easy. (But not trivial.)

11. A way for Yahoo to do something useful, interesting and innovative.

12. An example of a realtime photo feed. Note the use of Yahoo's Media-RSS extension.

Reboot the News at ONA09

Thu, 01 Oct 2009 15:10:36 GMT

Jay and I decided we wanted to do a live podcast at ONA09 so much we rented our own room. And it's open to the public, even if you're not going to ONA09.

Here's the deal. Come to the SF Hilton at 4PM on Saturday and look for "Rebooting The News" on the TV screen in the lobby. Or you can follow me or Jay on Twitter, we'll certainly advertise our location. Be sure to be there before 4:10PM. We'll all sit around a conference room table and reboot the news.

No punditry, or savvy church-goers, just talk about how the news will work after the new system is fully rebooted.

Flying cross-country

Thu, 01 Oct 2009 14:02:52 GMT

I'm flying today from JFK to SFO.

Trying something new, taking pictures as we go, as cloud cover permits, and uploading them in realtime.

I have enough people in my loop so that almost anywhere I post a picture of, someone is in the the picture, and a few people can tell me exactly where it is. I learned this on the west-east trip last week, when I took a picture in the middle of Colorado that turned out to be in Ouray County near Telluride. I've been there! Beaufiful country.

Here's the Flickr set of pics taken today.

Realtime interactive social media. Whatever you call it, it's really coool.

Two proposed rssCloud additions

Thu, 01 Oct 2009 13:50:27 GMT

On Tuesday I outlined the next steps on two proposed changes to the rssCloud walkthrough document.

I plan to implement these changes shortly. Joseph Scott of Wordpress, who proposed the second change, has said he will also implement them in his software.

Also see the short-term roadmap I posted last Friday.

An open reboot on Saturday

Tue, 29 Sep 2009 15:22:05 GMT

We're going to combine this week's Rebooting the News with next week's and do it Saturday afternoon live in San Francisco. Jay will be in town for the Online News Association meeting, and of course I live a BART ride away in Berkeley, so we'll meet at 4:30PM and do a 75-minute Rebooting the News special, and you're invited!

We're looking for a place that's very near to the Hilton. If you have a conference room nearby, within say a 5 minute walk, would you like to host this small meetup? We'd find an appropriate way to thank you in the podcast.

Also, please post a comment below if you'd like to come. Can't guarantee how much room we'll have, we may end up doing it at a Starbuck's!

Why no Twitter clients with an API?

Tue, 29 Sep 2009 12:10:28 GMT

Scoble says the iPhone version of Tweetie is so excellent that it might be the one everyone switches to. I don't know about that, I'm seriously considering a divorce from my iPhone, so I don't care so much about iPhone apps.

He specifically calls out Seesmic who doesn't yet have an iPhone app.

While I use my iPhone for tethering (and won't install the 3.1 upgrade because it would kill that) -- Scoble is tethered to his iPhone for everything. He does all his tweeting and friendfeeding from the iPhone. I find this both amazing and ridiculous. So many compromises. I guess Scoble values connectivity above all else, and wants to travel light.

He really wants a curating application, which I have, it's the app that manages my 40 Twits page. I've released it as open source, and set up an account for Scbole, but it doesn't work on the iPhone because it depends on a bookmarklet and I guess they don't work on the iPhone? I don't know enough about it to say for sure.

Bookmarklets make a lot of things possible. If they don't work on the iPhone then Apple should get to it and make them work. (Apparently they do.)

So Scoble is taking another tack to get what he wants. He's trying to scare Seesmic into providing the "curation" feature for him in their non-existent iPhone app. It would be simple, just a slight variant on retweeting. Maybe they don't want to do it because retweeting will soon get an overhaul. (And imho will finally work as it always should have.)

Which brings me to the point of the piece. Maybe Twitter clients are now mature and competitive enough of a market that they should support plug-ins of their own. It's the way things go. They are pretty much plug-ins themselves, but then so is Twitter a plug-in for the Internet, which is the end of the chain (it's not a plug-in for anything).

If Seesmic supported plug-ins then they wouldn't have to wait for their competitors to beat them in the market before they implemented something. They wouldn't have to worry if it pissed off Ev or Biz, they could just shrug it off as something a developer did (whatever it is that pissed them off).

It may be impossible for them to support plug-ins, if so, many apologies. But in case it's not, it could be the way to answer all of Scoble's feature requests. "There's a plug-in for that."

A cloud-enabled podcast feed

Mon, 28 Sep 2009 20:33:53 GMT

In the early days of podcasting, in January 2001, we got the bootstrap started with a feed of Grateful Dead songs. Every day or so I added a song to the feed, in case anyone was interested in writing an application that grabbed audio enclosures from feeds. I needed something to test with because Radio UserLand, which we were working on at the time, had the capability.

It took a few years for podcasting to catch on, but having that example feed made a difference. You have to start somewhere.

Now in 2009 we're trying to bootstrap a network of realtime feeds, and it's going pretty well so far. Podcasts are implemented with RSS too, and while we have excellent examples of realtime photo feeds, we don't yet have a realtime feed with audio.

So a week or so ago I started exploring options, and thought I'd use the Grateful Dead again, until JY suggested using a fast-updating audio feed from the Internet Archive. I took one look and realized this was it. It took a bit of a coding to check it periodically to see if it has updated, add a cloud element and notify one of my cloud servers. Now it's done.

http://static.newsriver.org/archiveOrg/podcastRss.xml

So if you're working on podcatching software give it a try.

Since River2 is a podcatcher, it automatically works with this feed.

TechCrunch is cloud-enabled!

Sat, 26 Sep 2009 13:55:23 GMT

It's a cause for celebration every time a feed is cloud-enabled, but it's a special victory when a large tech news site takes the leap. The first to do so was GigaOm, which became a source of realtime updates on the same day that WordPress got the feature. Now the second big brick falls into place -- TechCrunch.

I've been emailing with their new CTO Nik Cubrilovic, for the last month. This morning I got an email from him saying their service was up.

I added the TechCrunch feed to my River2 aggregator, and it worked exactly as it should!

Here's a screen capture of the Log page from River2.

1. I subscribe to the feed.

2. River2 requests notification from TechCrunch.

3. They check out its notification handler, and apparently like what they see.

4. They accept my request.

5. My River2 logs the registration.

Now when TechCrunch posts a new article my aggregator will be directly notified, will read the feed, and will immediately post the new item to my home page. All this should happen, if everything is working, in a few seconds.

Thanks TechCrunch for supporting rssCloud!

PS: Last night I posted an rssCloud status report.

Where we're at with rssCloud

Sat, 26 Sep 2009 02:46:00 GMT

A brief report on where we're at with rssCloudLand as September winds down.

It's been a great summer. In July, we got busy rebooting the RSS cloud, based on the experience in 2001 and 2002, with lessons learned, and a lot more success than we had seven years ago.

Based on the walkthrough document, the community has shipped multiple implementations of all three sides of the interface.

1. There have been many implementations of the cloud software, including my own rssCloud.root and wordpress.com and the plug-in that turns every standalong WordPress installation into an rssCloud server. Total number of installations: millions.

2. Lots of feeds and authoring systems numbering in the millions. Example real-time podcast feeds and photo feeds.

3. Not so many aggregators. So far only two -- first my own River2 and LazyFeed. There many more on the way. I hope one or two Twitter clients will ship rssCloud support in the month of October.

What next?

I have four immediate priorities:

1. Features that will make it possible for Google Reader and other large scale aggregators to wire into rssCloud. These are technical features, to most users they are details, but nonetheless are important. It must be possible for a server to register a notification request using a domain name, on behalf of another server. And along with this feature, most developers believe a stronger verification process is needed. I am one of them. Yet it must remain simple to implement, so the barrier to entry is as low as possible. There have been proposals in both these areas.

2. Long-polling proxies for clients running on systems behind NATs or firewalls. This technology is well-understood, and a number of developers are working on solutions. I hope when code emerges, there will be an effort to make their interfaces compatible, so maximum interop can be achieved.

3. Conventions for naming, so that URLs can be mapped to names that are managed by DNS, the naming system of the Internet itself. I've done a simple experiment in this area to get myself educated on the issues and learn how users and developers see this.

4. A high-level user-oriented vision statement for rssCloud with a roadmap for developers and marketers. My goal is to create an open loosely-coupled message network that achieves more or less what Twitter does, but in a decentralized manner. It must use the architecture of the Internet to achieve what Twitter achieves in a centralized fashion. No doubt there will be tradeoffs, some things the open network will do better, and other things that the centralized one will. This is similar to previous layers of the Internet, where systems like AOL and CompuServe provided more sophisticated features, where the World Wide Web was more low-tech, more basic. Both ways of doing things have advantages and survive to this day.

Comments, questions and suggestions are welcome.

Arrington in the airport

Fri, 25 Sep 2009 01:23:08 GMT

A question for DNS gurus out there in InternetLand

Thu, 24 Sep 2009 04:55:45 GMT

I bought a cool domain for a project I'm working on, r2.ly.

It takes Libya forever to approve these things, but they finally have, and I've changed the DNS to ns1.slicehost.com and ns2.slicehost.com (where I now have an account, with DNS control) but I can't tell what's going wrong cause I don't really know how to debug DNS.

I just know that it's not resolving on any of my machines.

Can you tell what the problem is with it?

And if you solve the problem you get to call me stupid in the comments and I won't moderate it out.

I want to divorce my iPhone

Wed, 23 Sep 2009 18:02:02 GMT

The iPhone is so totally not my spiritual soulmate.

I refuse to become dependent on apps grown in their environment. To me it's like contributing to the enslavement of my brother and sister programmers. I don't care how sexy the environment is as a user or a developer, the fact that Apple holds up apps and rejects them often because they compete with their own software is to me like buying a coat made of the skins of endangered species. I won't use iPhone apps for ecological reasons.

I use my iPhone as a: 1. Phone. 2: Camera that can communicate (very valuable feature to me). 3. A Bluetooth tethering device for places my Sprint MiFi doesn't work (and that's a lot of places).

For that I pay about $100 per month. I think I'm being ripped off. (Sure of it.)

Okay Scripting News readers -- tell me I'm crazy but I want a divorce. Enough of this bullshit.

But I need a phone that does 1, 2 and 3.

What will I fall in love with?

PS: I have my contacts in GMail. Must be able to synch with them. One of my favorite iPhone features.

PPS: I never use it as an iPod. I prefer my Walkman.

Sharing links in the River2 community

Wed, 23 Sep 2009 12:54:56 GMT

A new feature makes it easy to share links to stories from the River2 news page to followers on Twitter.

http://newsriver.org/r2ly.html

It also maintains a Top 40 list for River2 users.

This feature is itself a bootstrap since one of the key ingredients are the people using it. Just the right number and the right kind (thoughtful, passionate, forgiving, visionary) are needed. The result can be an editorial product in its own right. And the first experiences will ~~probably~~ certainly suggest the second and third level of features.

Bootstrapping is for users too. In fact at some point, without users participating, the bootstrap stops. You have to wait until they show up, or keep trying to figure out what it will take to entice them to participate.

I realized something yesterday, that bootstrapping is hard because you have to use things that don't exist yet.

I often use bridges as metaphors to describe bootstraps in software. Here goes..

People hardly notice driving across the Golden Gate Bridge, they might notice the scenery, or the walkers or bike riders, or think about the oncoming traffic. But one day, a long time ago, someone stood on one side or the other -- in San Francisco or Marin -- and looked at the Golden Gate (which was the waterway before it was the bridge) and thought "I bet we could put a bridge here." Nice thought, but then what? What's the first step? And when that doesn't work, what's the next first step? Long before there's a bridge, someone has to make the trip in a bucket hanging from a cable. Today, River2 users are those kinds of people.

I've been having an on-and-off discussion with Marshall about this. He's been frustrated by the software. I understand. But I need people like him to struggle through it, tell me what went wrong, so I can try to fix it, and then tell me if it worked. The rewards mostly come from within, to know that you played a role in making something new happen. In this case it will be a distributed loosely-coupled 140-character message network that is free from YWFFTMMR. And there might be business opportunities to provide services to users. No one's saying you can't get rich. But it has to be fair, and people should always have lots of choice.

One more note -- it's taking a long time for the domain I'll use, r2.ly, to come through the registration process. So to begin with we'll use a teamrss.com domain to share links. If you see one of these on Twitter it's coming from this part of River2.

What is the real-time web?

Tue, 22 Sep 2009 22:49:44 GMT

Four words: It Happens Without Waiting.

Narrative: Today I wrote a piece about the Berkeley Public Library on InBerkeley.Com. I wanted to find a pointer to the library website, so I switched over to Google. Looked up Berkeley Public Library. My piece, publshed less than a minute earlier. was the first item. Real-time web. (True story.)

The SUL as a tool to control news?

Tue, 22 Sep 2009 12:29:43 GMT

SUL is Twitter's Suggested Users List.

It's a group of approximately 500 Twitter users who are "suggested" to new users when they create an account. The stated purpose is to provide people to watch when you're starting out. But are there other purposes? Could it be used to reward positive coverage and punish negative coverage? I think we now have some data on that.

There's no doubt that Twitter has received a lot of help from the press, and much of it is genuine enthusiasm for a communication tool that at least hints at the future of news.

Many of the suggested users are news organizations, reporters, columnists, marketers, and as a result, most have over a million followers. Almost all of the top tech news organizations are on the list. And TechCrunch was one of them until something happened in July as is evident in this TwitterCounter graph.

Compare this to the graph for Mashable, over the same period.

And plotted on the same graph.

It's pretty clear something happened in July.

We know this much -- TechCrunch was dropped from the Suggested User List, right around the time their follower count started heading down. As to why, we can only speculate that it was because they ran a piece that Twitter didn't like.

7/16/09: Twitter's Internal Strategy Laid Bare: To Be "The Pulse Of The Planet."

People have always questioned whether there was a connection between being on the list and not being too critical of Twitter. At this point, there isn't much doubt that the connection is there.

What became of Radio's POST button?

Tue, 22 Sep 2009 02:34:58 GMT

An interesting story of evolving software.

In 2002, my company shipped a product called Radio UserLand. It was a very popular blogging tool, but it was also the most popular RSS reading tool of the day. And because it was both things, we could do integration that no other product had ever done before, or since.

Adjacent to every item in the aggregator was a button that said POST. When you click it, you flip to the blog post entry screen with the text of the item in the big box. You could add your own words, shorten it, whatever you like. When you were done, hit Submit and you'd have a post that pointed to the original article with your comments.

As an aside, this is where the RSS <source> element came from. We'd embed that, invisibly, in your post so tools could find their way back to the original. This was in response to an outcry from bloggers that we were helping people steal content. Seems like a foreign idea today, doesn't it?

Anyway...

Fast-forward to 2009, and I'm back at work in AggregatorLand, and like it or not, Twitter is where we push links to these days. So now instead of a POST button look what's there in its place.

Now, it is very much more clever than the POST button was back in 2002. Just how much clever -- you'll have to wait to find out, because I'm still working. But when you see my links to test.teamrss.com on Twitter you'll know that I'm testing the new stuff. Murphy-willing it should be released to River2 users tomorrow.

TopTwits tracks your Twitter linkage

Mon, 21 Sep 2009 14:41:11 GMT

When tr.im announced that it was going open source, I said I would also release the code of the app that does my Top 40 page and that of Jay Rosen, Kevin Tofel and Zach Seward.

TopTwits is that app. It runs as an OPML Editor tool. This means you must have the OPML Editor installed on your machine, and then install topTwits.root in the OPML Editor.

YWFFTMMR

Mon, 21 Sep 2009 07:00:00 GMT

When you get something going all of a sudden the ~~parasites~~ opportunists swoop in and want to take control. Funny how they leave you alone when you're not so hot.

I suppose I should see it as a good sign, but they all resort to the same kind of character attacks when I decline their offers. Some nastier than others. The offers amount to me working my ass off to make them rich, for which, in turn -- I get nothing. $0. Bupkis.

Only in the tech industry do people have the audacity to look you in the eye and say You Work For Free To Make Me Rich. YWFFTMMR.

The stupid thing about it is that's at least part of the reason I'm trying to get out of Twitter. I don't like their economic proposition but, I do like microblogging. I figure if I'm not going to make any money off my work, then I'll work in an environment where no one does. It's weird that the people behind Twitter are supposedly capitalists yet seem to not understand that very simple idea. People don't work for free. They don't pour out their passion in the cause of making you wealthy. They might be motivated to do it if they saw some upside.

I'll let you know when someone approaches this space with respect and an offer that isn't usurous.

Today's a big shipping day

Sun, 20 Sep 2009 16:31:28 GMT

I didn't plan it this way, but a lot of stuff is ready all at once.

Over the day I'll link them in here. They will also appear in various other places such as the rssCloud mail list, on Twitter, and I'm not sure where else.

1. A rssCloud walkthrough for publishers. This doc is a shortcut for publishers so they don't have to wade through the michegas that cloud and aggregator developers do.

2. A self-documenting OPML reading list of real-time photo feeds. Lots of new stuff working here. I also added a photo feed for television, in anticipation of the Emmies tonight.

3. DNS for RSS feeds. I have a proof of concept app up, for you to try out and comment on.

N. And last but not least, and I had nothing to do with this one -- the Scobles shipped another Scobleizer last night. Baby and mommy are happy and healthy, and according to daddy, cute. Happy birthday Ryan! It's nice to have you on board for this crazy thing we call life.

Hooking the lizard brain up to the cerebral cortex

Mon, 21 Sep 2009 03:21:15 GMT

This is worth a special post as a followup to the earlier piece, it's so interesting.

I was talking with a friend today, he's an expert in DNS, and I said it's too bad people can't open their supercloud.org sub-domain in a browser. I understood why this wasn't possible:

1. The browser is looking for an A record or a CNAME, and we're setting the TXT record.

My friend who doesn't want me to say his name (and I respect that) said it might not have to be that way. We talked about connecting the DNS with HTTP and it was really intriguing. If you think about it, DNS is like the lizard brain of the Internet, and HTTP, while it is in some ways lower tech than DNS, is the cerebral cortex. To have them integrate is like bringing the Wright Brothers plane they flew at Kitty Hawk on an Apollo moon mission. But that's the way we do things in techland.

Then I thought -- wait a minute, what's to stop me from also registering an A record that points back to me, and then keeping a database locally that associates a name with an RSS URL. Then when you open your supercloud.org subdomain in a browser:

1. It comes to my machine.

2. It looks you up in the database I'm keeping that's a mirror of what's in the TXT records in DNS.

3. From there I get your RSS address.

4. To which I simply do a 302 redirect.

Hah! I had to try it. And guess what! The fcuker works!!

Try it out: http://davewiner.supercloud.org/.

And then go through the registration process, and try it out yourself.

This is actually important because everyone expected taht you'd be able to open the supercloud.org sub-domain in the browser. Now you can.