Scripting News

My buddy Rex

Sat, 24 Oct 2009 22:20:45 GMT

These days my blog posts are always essays, but it wasn't always so. In the beginning it was all links, with pointers to articles both on this site and off-site. Example.

Then it became a hybrid, at the top of the page were the links and at the bottom were articles. Example.

Then in early 2007 (Jan 6, to be exact) I went all essays, and then a few weeks after that, started using Twitter. It's funny how one event followed another.

Anyway, this article by Rex Hammock is so lovely and so vindicating, I'd do a special post just to link to it.

Rex Hammock: Facebook goes River of News.

And one little thing, I'm going to have a linkblog up in the not-too-distant future. Again. Everything is new again, every few years, it seems.

Also it's sad that my friends, people like Rex, have to hedge so much because of a handful of stinkers who follow me around on the web. I'd like to encourage my buddies to just go through it, and say what you want to say and let the stinkers stink up some other place. Life is too short. With much love, Dave.

Testing BuddyPress

Sat, 24 Oct 2009 16:22:52 GMT

When Matt told me that WordPress was going to support rssCloud that got me started using WordPress with new purpose. I've been learning to use the product through wordpress.com. I haven't yet started my own installation. My attention is focused elsewhere.

Anyway, the point of this post is to get help learning how to use BuddyPress. I don't want a huge hosting obligation. Ideally I want a freemium deal like the one at wordpress.com. However, it doesn't seem to exist anywhere, yet.

I just came across a site that says it lets you test BuddyPress.

http://testbp.org/

I was expecting to have to create an account, but it (apparently) found me on Facebook, and I'm already leaving a trail there. Totally not happy about that, but I suppose my gripe is with Facebook, who somehow has decided that they own the web and can give access to my account to anyone who asks for it? I was never asked to opt into this. Unless I'm missing something this seems just plain bad.

Anyway, I thought BuddyPress was supposed to be like Twitter. It doesn't look anything like Twitter. There's no box at the top of the page that asks What Are You Doing? Without that it's not Twitter-like.

Confused.

Update: I found my "wire" page -- and on that page, there's evidence that I had been here on April 30. So that lets Facebook off the hook. I must have created the connection then.

Update: Some free advice for the BP designers. The home page of my site has to look more or less exactly like the home page of the Twitter site. Any difference is going to equal pain for users, and pain for users means slower adoption. Later, when and if you achieve dominant market share, you can slowly evolve the UI, if you really feel you must. Users are less interested in innovation in the UI than you would think they are.

Our all-you-can-eat lifestlye

Sat, 24 Oct 2009 12:44:38 GMT

First thoughts on our San Quentin field trip.

I went for a late lunch in Sausalito with Scoble after spending most of the day inside the walls at San Quentin state prison. We were sitting on a quiet beautiful street with healthy, well-fed people walking by, driving in to eat at the Indian restaurant, riding bicycles and stopping to ask for directions. Scoble entered our location into Foursquare and a few minutes later a clean, friendly young man showed up with an infant wrapped in a blanket. He greeted us with a smile, Scoble instantly knew he was. We didn't in any way at any time feel we were in any danger. I was pretty sure most of the people with us there had never killed anyone.

That may strike you as an odd way to describe a lunch in the center of high-tech land, because that's our normal reality. We expect so much, and we get it. We live the all-you-can-eat lifestyle. But just a few miles away reality is very different.

We met a man who had never used the Internet, had never seen a cell phone, had no clue what Twitter is, and probably a million other things we talk about all the time. He's been in jail since 1987. He talked to us for a while in the courtyard just inside the entrance gate. He's in a "program" and my guess from the way it sounded, will be paroled in January. He murdered his little sister when he was 18. Blew her head off with a shotgun. He did it because she and her brother and mother hid his money and drugs. He told his brother that he'd kill his sister if he didn't tell him where the stash was. The brother said he'd never kill her. He did.

He didn't tell us this, Rudy Luna, the assistant warden who was taking us on a tour, did.

The warden said that, ironically, that prison is a revolving door for people who commit minor crimes, but for murderers like the guy we were talking to, sometimes they get out and stay out. He says there's a point, usually at 11 years, where they realize that they could change. The guys who get sentenced for smaller crimes don't get there.

The guy we were talking to might not commit another murder, but I don't see how he can live with himself.

Everwhere we went we were being watched.

By everyone.

That may have been the oddest thing. I am accustomed to leading what I think is a fairly anonymous life. Sometimes on BART a stranger is staring at me, I imagine they recognize my face from my blog. But most of the time I move around without anyone paying much attention. Not inside the prison.

And it's not just us they're watching, they're all watching each other, all the time. Because prison is a dangerous place. Everything they do seems to be about keeping from getting slashed or beat up or killed.

We saw thousands of people in tiny cages.

We saw the outside of a building where people are locked up all the time, their crimes so heinous or infamous, or they attract so much attention, or they are people who will try to kill anything that they possibly can.

It's the contrast that is so striking. And what it tells me about who I am.

Having just lost my father, I'm thinking about what death means in a much more real and present way these days. Our guide tells us at one point that most of the people we're looking at, and there are hundreds of them, killed someone. And they're walking around like you and me in a park. Except it's nothing like the way we walk around in a park. Everyone is watching everyone. All the time.

I'm sure there will be other insights. Coming out of it, I think none of us knew what to write. That's the sign that we were doing something very different, something very very outside our normal experience.

Is Google/Microsoft/Twitter in the news business?

Thu, 22 Oct 2009 20:05:49 GMT

Yesterday the earth shook, and at first glance you might think it just shook in Silicon Valley, but I think a few years from now we'll look back and realize that the earth was shaking just as hard in the media industry.

I've had this really strong feeling ever since I got enamored of Twitter in 2006 that it was something the news and entertainment world would jump on if it had leadership that was as bright and ambitious as Steve Jobs and Bill Gates at their prime. Alas, those days must be long gone, because they busily tried to litigate peace in the old war, which they lost long ago, looking for the reparations that are due the loser who manages to make the victor feel guilty. (In other words the crumbs left on the table after the meal is over and after the cleaning people have made their first pass.)

"Look over here!" I've said over and over. "You should be competing here."

"We don't see why," they respond.

Here's why.

Twitter got Google and Microsoft to pay for the content that the media industry should have been hosting instead of Twitter. There was money here. And as we all know, the media industry can't find enough money to keep going. They're looking for handouts from the government. Meanwhile there was money everywhere. They just had to evolve.

I'm not saying this payday means Twitter has it made, they don't. Google and Microsoft are sharks and Twitter may be a goldfish. It could be that Evan Williams and his team have the competitive instincts of a Gates or Jobs, if so, they certainly have a few tricks up their sleeves, or they wouldn't have done these deals. They better, because Google and Microsoft are almost surely executing an Embrace & Extend. What that means for Twitter is that they have clones of Twitter in development. The race is certainly on. Have they cross-licensed their streams? In other words, does Twitter have reciprocal rights to any realtime content generated by users of Google's TwitterLand? Microsoft's? Even if they do, could they handle the load? My guess is that both Google and Microsoft will quickly take the search function away from Twitter. Now everyone has the Twitter stream. What streams can Microsoft and Google add to differentiate theirs?

And what business are they in now? I believe they're in the news business. This isn't tech anymore. This is what the Times and CNN should have become, what CBS, NBC and ABC should be. What Jay calls a pro-am system where everyone collaborates to create the realtime stream of news on all levels, national, international, local, broad coverage and specialized stuff. Everything from newsletters to nightly news. Everything flows through the same pipes, and curators pick off the good stuff and route it to people who are interested. This is the way news is done from here-on. We're not talking about the future, we're talking about now. And the moguls of the media industry, without a single leader thinking in advance of the wave, are sitting on the sidelines, hoping someone will give them some money because they're such great writers or whatever it is they're so great at. Soon they'll be looking for reparations.

They should own the platform.

And it's bad for the rest of us that they don't because the moguls of Silicon Valley have a very crude understanding of what news is. Witness the longevity of the 140 character limit and the inability of Twitter to carry any type of content other than text. The horror of the Suggested Users List. I don't expect Google or Microsoft to do much better, but they'll probably have the sense to hire a few news pros to advise them on how to build a system that works for news. The Twitter guys are fumbling around, and in doing so, holding all of us back.

And FriendFeed. Oh man what a wasted opportunity that was. If they had an ounce of competitive spirit they would have noticed that the news industry wasn't seeing their way into this space, and they would have gotten on a plane and camped out in NY and found someone, anyone, with a good flow of news to partner with, to guide them toward creating the fantastic news system that Twitter wasn't building. They had the technical ability to do it, but they were too much of homebodies, they enjoyed the comfort of other engineers too much.

This is what we still have to do -- create the connections between people with technical knowhow and people who can make the news flow to create a safe harbor for the millions who want to participate in news to do so, without being owned and controlled by the titans of tech.

Flickr in hindsight

Wed, 21 Oct 2009 23:11:39 GMT

You know what they say about hindsight being 20-20. It's that way with Flickr, which was way ahead of everyone else in the social network thing. With a few tweaks three years ago, with active entrepreneurial development and the resources of Yahoo, it could have been everything Twitter or Facebook is today.

I don't know enough about what goes on behind the scenes to know if it really was possible. It could be that the code is a mess and that the last engineer who understands it left five years ago. If so, the previous paragraph is probably nonsense. But if there are still some good people working on it, it may not even be too late for Flickr to act as a backbone for at least part of the future realtime web.

The thing that Flickr does that Twitter doesn't is, as I said a few days ago, payloads. In Flickr a payload can be a picture or a video. Another thing Flickr has going for it is a great API with lots of developer support. And while Flickr does go down sometimes, it's a lot more reliable than Twitter. And there's no Suggested User List.

What it's missing is a Twitter-like timeline. But I honestly don't think that's so hard to do. I think they already do the hard stuff. Maybe when Carol Bartz gets over the flu we could meet to talk about giving Flickr a lot of independence and a bunch of cash and letting it be free to compete in wild, free of the constraints of corporate Yahoo. Then, once it's flying, the various Yahoo properties can latch on to its growth as any other developer could.

It could be a terribly bad idea. But I still love Flickr and use it all the time. I even pay them money every year for privilege. I bet a lot of other people do too.

Bruce Sterling at Reboot

Wed, 21 Oct 2009 21:11:41 GMT

Bruce Sterling gave a wonderful talk at the Reboot Conference this summer in Copenhagen. At the beginning of the talk I wanted to strangle him, but as it progressed, it made more and more sense. By the end I thought it was one of the best speeches I'd ever heard, a story that I think everyone should hear. I've made an MP3 of his talk because I want to make it available to people in my family as a podcast. I hope Bruce and the people at Reboot don't mind.

He talks about clearing your life of posessions, how you should divide everything into four categories: 1. Beautiful things. 2. Things with emotional value. 3. Functional things. 4. Everything else.

Divide each category into the things you keep and the things you get rid of. In category 1, you can keep it if it's on display in your house, if you show it to your friends, if you share it. If not, then you don't need it, it's taking up space and time, which you're paying for with your money, time and health. Take a picture, put it on a thumb drive, take it everywhere with you and get rid of the original. In category 2, if it has a compelling story, one that you actually tell people, you can keep it. In category 3, unless it's very good at what it does and it does something you do a lot of, it goes. And of course everything in category 4 goes.

He says you shouldn't try to do this in normal times. Wait until a spouse dies, a divorce, a child is born or a child leaves home. Wait till you move. It pays to figure out now what you want to do when that time comes.

I know Sterling is right because I've had things like that happen and I've done it both ways. Most of the time I don't clean house, and miss the opportunity to improve my life. But sometimes I do make the changes and it's always, in the end, been a good thing. Most people advise you not to make changes in times of great life turmoil. That's exactly the wrong advice. Those are the only times you can make change.

This is a hot topic in my family because of Father's Day. It just happened, and the shock is just now beginning to set in. It's strange that along with the pain and sorrow, there's also a new sense of freedom, of possibilities. It's palpable. And it doesn't take a second to locate the source -- it's the changes Sterling talked about so eloquently.

Anyway, most of the time most of us are not in position to do anything about the mess in our lives. But listen to Sterling's talk. It's only 43 minutes. It might be the best 43 minutes you've ever spent.

How to learn to love the Fail Whale

Wed, 21 Oct 2009 17:23:44 GMT

What if your Twitter client stayed up when Twitter is down?

Believe it or not -- it's possible. I'm sure it's crossed the minds of the people who run Tweetdeck, Tweetie, Brizzly, Seesmic, et al. What if users could keep communicating with people who used the same tool you use and have all your tweets flow out through Twitter when they're back on the air?

Even better, what if Tweetie and Brizzly got together and worked out a way for their users to stay up and communicate with each other, even if they used the other guy's tool? (See where this is going?)

An rssCloud case study: Brizzly & Seesmic.

Most of the time people don't think this way, except when the Fail Whale is showing up, as it has been for the last couple of days. Then creativity kicks in and you start wondering if it's possible, and if it were, what's in the way?

Maybe Twitter wouldn't like it if the client companies got too independent? Maybe they have some way to punish those who stray and reward those who don't? Some people think they do, with the little ads in the right margin of the Twitter web page. They say those ads really work, and if you don't play ball the way Twitter wants you to -- no soup for you!

I know how the Twitter clients can become free from Twitter, yet still work with it. You might have to give up your tasty Twitter soup, but you might be able to find new users you wouldn't otherwise, if word of mouth started carrying the message that your client doesn't go down when Twitter does.

Innovating outside the 140

Tue, 20 Oct 2009 21:45:26 GMT

I pitched a simple idea at yesterday's GigaOm meetup, one that is easy to explain verbally but I've not yet attempted to explain it in writing. So here goes.

When the Mac came out in 1984 it had a file system that worked much like a Unix or PC file system, it was hierarchic, had volumes, folders and files. But files had two "forks:" 1. Data and 2. Resource. The data fork was like a regular file, but the resource fork was really cool and different. It was like a file system within a file, but not quite.

Resources had a type and an id. The type was a four-character string, and the ID was a number. There were standard system types like WIND and MENU, and in them you put designs for windows and menus. There was a resource editor that shipped with the OS that had tools for the standard system types, the menu editor let you add a command to a menu or delete a command. The window editor let you set the default size of the window, its initial title, and what WDEF routine was responsible for drawing it. A big part of learning how to program the Mac was learning what all the resources were and how to set them up. Then you'd write C or Pascal code to open the windows or draw the menus.

But that wasn't all you could do with resources -- because -- and this is the key point -- you could define your own resources. You didn't have to get anyone's permission (okay theoretically you did, but we never bothered). So if I wanted to write a DAVE resource to my file I could. And then I could tell you what a DAVE resource contains and your app could read and write them, and all of a sudden we've just enhanced the platform. Pretty cool! And we did this kind of stuff all the time.

So why shouldn't tweets also have resource forks? Then if I wanted to attach a picture to a tweet I'd just pack it up in a blob and shoot it up to Twitter as part of a PICT resource along with the 140 characters which would then be a description for the picture. Or why not have a menu go with a tweet? Or a bit of HTML? Or whatever the fuck.

This would get Ev and Biz out of the loop, they could just kick back and run a storage system and stop worrying about what features to add to the platform. You see the users get to innovate inside the 140 characters, but there would be so much more action if the developers could innovate outside the 140.

Random Tuesday notes

Tue, 20 Oct 2009 18:39:35 GMT

Good morning. Getting back to work after a half-day at a conference followed by a five-hour baseball game. The conference was at Om Malik's and was the first tech conference I'd been to since the disaster at Gnomedex in 2007. This one was much better. I got to talk with a number of people I'm working with on RSS-related projects, and met a few developers with interesting projects. Om gathers an interesting group, that's for sure. And the mood at Om's place is respectful and collegial. We got some work done. Nice.

I told Om I'd like to try out the flash conference idea at his new 2nd St office. Great location and a good size. So next time there's a rush to get people's ideas on some new tech development maybe we can get together to talk about it at Om's place.

The baseball game was the third in the ALCS between the Angels and the Yankees. I'm really liking the way the Angels are playing, and of course I'm always up for rooting against the other New York team.

There will be more baseball and I'm speaking at Jeff Pulver's 140 character conference next Tuesday, a week from today. It's in Los Angeles. I can invite the regulars at Scripting News as my guest, so if you'd like to come, send me an email or post a comment, and I'll send you a link that gets you free admission. Jeff was very kind to let us party on his dime in LA.

Meanwhile, two really interesting articles you all should read:

1. Wired: How Users Took Over Twitter.

2. Mediate: We Keep Finding Historically Significant Photos. Will Our Grandchildren?

The latter question was the punchline of yesterday's Rebooting The News podcast, which has yet to appear in the feed. The irony is that we could be doing a better job at archiving our thoughtstream, but we're actually doing a worse job. Our pictures, movies, recordings, thoughts have never been more ephemeral.

Yesterday at Om's we wasted (imho) a time talking about seredipity. The time could have been better used working on more mundane topics like maintaining a memory that lasts more than a year at a time. We love the latest and greatest stuff, but don't recognize the patterns, we've seen this before. It's like the recurring theme in BSG, it's happened before. But we threw out the archive! Oy gevilt.

I refuse to solve the problem only for Scripting News, because I don't want future generations to think I was the only one writing in the early part of the 21st century. One of our bloggers is the next Ernest Hemingway or William Faulkner, Willa Cather or Emily Dickinson. Let's make sure we have their earliest emails, Flickr-ings and tweets.

BTW, this guy at Yahoo is surprisingly funny. I didn't know they were allowed to hire people with a sense of humor. Hate to say it, but I've never seen anyone at a big tech company, Microsoft, Google, Apple, Facebook or whatever that had a sense of humor. I have an idea, they should make being funny one of the requirements to work at the Bigs. Who knows the software might be more er uhhh you know -- fun.

The Internet abhors a funnel

Sat, 17 Oct 2009 18:19:30 GMT

This might be one of those fundamental Internet laws, or it may just turn out to be a way to express another one.

The Internet is a bunch of pipes and tunnels with processors located at the junctions and at the ends of the pipes. Some of the processors are little things like iPhones, routers, blogs or webcams. Some are huge like Google's cloud.

All is good until something comes along that tries to get the whole Internet to flow through it. Compelling things that get more compelling the more stuff you flow through them. Instead of being shaped like pipes, they are shaped like funnels (as illustrated to the right).

Platforms that are owned by companies start out as funnels. They may trick you for a while into believing that they made of pipes, after all it would be suicide for the platform owner to turn the lovely ecosystem into a funnel, but companies can't help themselves. They compete internally to control resources and to the people inside the company, the platform looks like just another resource to fight over. Eventually it gets funnelized, if it wasn't from the start.

This is why my current motto about Twitter is "How I learned to stop worrying and love the Fail Whale." I can love the Fail Whale, even though as a user I hate it, because it says to me that I will soon control my own destiny in this space. That cute little whale says the idea of Twitter as a funnel (it started out as one, always has been one) is not working. As the network grows, it becomes less viable.

The funnel is also why I don't like a lot of the things Google does. Things like SideWiki that are only useful when everyone uses them. It shows funnel-oriented thinking over there. Long-term they can't make the funnel work. But as Microsoft showed us in the 90s, they can slow the rest of us down while we wait for the shakeout. Better if they never tried, imho. Better for them, for us, and esp for the Internet.

Rebooting the reboot of rssCloud

Fri, 16 Oct 2009 17:01:12 GMT

It's been a really rough month for me, personally, and that has stalled some of the forward motion in rssCloudLand. I was able to find a little time between crises to implement both of the proposed changes, and to outline an open discussion page.

Originally I had planned to have four or five days between the implementation and updating the rssCloud walkthrough, but then Father's Day happened, and well, all my cards went up in the air. I'm still not firing on all cylinders, so please check my work carefully, but it is time to clear the space for more deployment of what amounts to a crucial feature for rssCloud implementers.

This morning I updated the walkthrough document to allow for:

1. An optional domain parameter on the REST request for notification.

2. If the notification request included the optional domain parameter, the verification process works differently.

A note about evangelism

Fri, 16 Oct 2009 17:06:11 GMT

If anyone in the rssCloud community is marketing against PubSubHubBub, we will ask them to stop. My position, and I hope that of the community is that these are non-commercial efforts, no one is going to profit (or lose) based on the success of one or the other protocols.

I see adoption of PubSubHubBub as a win for the Internet, and believe strongly their advocates should see adoption of rssCloud the same way. If they feel pressure from rssCloud, it should result in them more fully embracing RSS, which I felt they weren't doing when I first reviewed their efforts. Once that happens the differences will probably melt away and everyone will be happy.

Personal attacks in furthering advocacy are totally unacceptable and should not be tolerated. I hope this goes without saying, but unfortunately it appears it still needs to be said. Please.

Seeing past Twitter's limits

Fri, 16 Oct 2009 15:49:06 GMT

When thinking about the future, something I've spent a lifetime training myself to do, I try flipping the classic question around. Instead of asking "Do you think X will happen?" try this -- "Can you imagine X not happening?" It doesn't always yield a breakthrough, but it often does. It helps you see past the limits of today.

Anyway, I'm giving a 10-minute talk to open Jeff Pulver's Twitter conference in LA on the 27th. The title of my talk, suggested by Carla Casilli, is: "How I learned to stop worrying and love the Fail Whale." It's a ripoff of the sub-title of one of the greatest movies of all time, Stanley Kubrick's Dr. Strangelove. In his version, they replace the Fail Whale with The Bomb. In the 50s, 60s and 70s people were as obsessied with nuclear weapons as they are today with Twitter (that's only half a joke).

Anyway, today a tweet is 140 characters with an ever-evolving cadre of metadata marching alongside. And it's about the metadata that I wish to ask the inverted question. But first, I'd ask you to click on the small picture below, and go to Flickr and look at all the metadata that's assembled around a picture of my aunt and uncle taken by my mother sometime in the late 70s or early 80s.

What a rich collection of information there is. Yet you could imagine more, yes? If you read the paragraph that introduces the picture you'll see that there's some data that isn't reflected in the Flickr database. It was taken by my mother, and the two people in the picture are related to both of us. One by blood (the man, my uncle Ken) and one because she is married to the other (my aunt Dot). That data is missing probably because Flickr stopped actively evolving before social networking had fully gained a foothold in online culture. Same with the web itself. Wouldn't it be cool if a pointer could imply a familial relationship? I know that's what TBL has been talking about. Maybe we're getting closer to actually having it.

Anyway, if you go crazy and try to imagine what Twitter might become, you can see that a lot of what it is is in the Flickr metadata without the thing in the middle -- the picture. I've been urging Twitter to support payloads for years now. I can't imagine why they're not doing it. One piece of metadata is all that's needed, minimally, the URL pointing to the picture. Today we have to cram it into the 140 characters. Meanwhile they're advancing, adding geographic data and lists and retweets, all of which add little bits of data to a tweet, but for some reason they won't add the url. Which gets back to the question. Can you imagine that Twitter will never get this feature? No, of course not. It will someday get it. Why not get it over with?

Techmeme curiosity

Fri, 16 Oct 2009 14:12:03 GMT

First a prediction -- some people are going to say I'm writing this because I want to be on Techmeme. They are entitled to say that, but they are wrong. I am neutral about it. If a story of mine belongs on Techmeme, it should be there, if not, it shouldn't. And it isn't up to me to decide, it's up to the people who run Techmeme.

Also, please read this whole post before commenting, not just one paragraph or phrase, because it's complicated.

I've always believed that Techmeme was a combination of a bot and human judgement. This was confirmed a few months back when Gabe, the guy who runs the site, hired a human editor.

I've disabled their bot by including a line in my robots.txt file that tells them not to crawl the site. But there is no such thing for the human beings, you can't make it so that a person can't read your site. So I always thought that if one of the humans at Techmeme thought something I wrote was interesting, they would publish a link to it. As far as I know this has never happened. (And I watch pretty carefully.)

Sometimes I take the block out of the robots.txt file to see what will happen. In those cases my pieces often turn up on Techmeme, almost never as a major item, rather as part of the "chorus" -- commenting on one of the major articles. I'm often in the chorus with the story everyone is reacting to, including the guy who got top billing. I don't know how this happens, but it's a large part of why I block their bot. I really dislike the chorus. It's what makes the blogosphere like a mail list. You end up with a lot of people chiming in with nothing to add, who just want the flow from being there.

Anyway, what made me think of it is that today Charles Arthur at the Guardian has a nice piece which is centered on my review of Twitter's lists. It's getting a good run on Techmeme. If Techmeme were doing their job well, they'd flip it around and present it as him commenting on the original piece. I'm saying this in case Gabe and Company think the bit in the robots.txt is a prohibition on their human editors. It is not. Read up on robots.txt if you don't believe me. It's all about robots.

Twitter's lists

Fri, 16 Oct 2009 00:07:53 GMT

Like a bunch of other people I had the new Twitter lists feature turned on today. Immediately I needed a list to explore and had a bunch of ready-made ideas. I had already done aggregated feeds of the top 100 most followed people on Twitter, the employees of the Twitter company and the NY Times, the Twin Cities and most recently Berkeley Twitterers.

The last one I replicated as a Twitter list.

If you don't yet have the feature turned on, you'll get a blank page on the last link. Here's a screen shot of what it looks like.

Basically you get a flow of all the people I've chosen to put on the Berkeley list.

Depending on what the API looks like we'll probably see all kinds of tools for combining and cloning lists.

It's a new authority system. The number of lists you appear on is a kind of page-rank. So let's hope Twitter does two things: 1. Provides an open API to crawl this data set. 2. Doesn't pollute it by artificially inflating the rank of friendly press and their industry friends. Stay out of the editorial space and let a healthy ecosystem develop. It's another chance to not screw it up.

Some people have said it's somehow related to the Suggested User List but I don't see it at all. This feature is for advanced users, the SUL is for total newbies. Unless Twitter somehow data mines our lists, something they could have done right at the start, it won't have any impact on the newbies' user experience.

Update: As you would expect, Scoble is going crrrazy with this feature!

Uncle Arno's books

Thu, 15 Oct 2009 16:24:28 GMT

My mother's uncle, Arno Schmidt, was a German author who lived between 1914 and 1979. He left a collection of his work to his sister Lucy, my grandmother, and when she died in 1977, the collection was passed to my mother. Legend has it that he gave the collection to his sister because he wanted a backup of his work in the United States.

Time goes by, and now my mother is thinking about passing on her stuff, and we want to do the right thing with Arno's legacy. There's a museum in Germany that wants the books, but they already have copies, and Arno wanted a copy in the US, so we're going to try to get him his wish.

I've listed our collection of books in a separate post.

We're looking for a university in the United States, probably one with a German Studies Department and a library, to take the whole collection. We're not looking to make money from this donation, it's strictly a gift, but we would like to have the materials accessible to the public. Ideally we'd like to work with a university in NY or the Bay Area because that's where our family is.

I wanted to make this offer publicly, through my blog, because it seems consistent with my efforts to get our electronic work archived. Here's a chance to learn how it's done with writing created in the mid-20th century. If you are at a university that is interested, either post a comment here, or send me an email at dave dot winer at gmail dot com. Thanks!

Friends see things you don't

Wed, 14 Oct 2009 16:28:38 GMT

This is a very personal time for me, lots of observations, things I'm learning about myself, things you can only see when a big tree falls and light shines on spaces that were previously hidden.

Here's a reason why it's good to have friends (as if one needs a reason) and why you should share your childhood pictures with them as soon as you can. I waited way too long for this.

When they see pictures of you as a baby and toddler and then as a small child they see something you never see in those pictures --> You!

Doc Searls' comment, as usual, hits the nail square on the head. "That's Dave! Great to see the kid, the son and the big brother, and how they match with the man and the friend."

Francine Hardway sees it too.

What a revelation. I never liked looking at those pictures, and now I know why. I didn't believe it ever happened. I have no memory of it, but friends can see what you can't -- that you were there. Now all of a sudden things your parents say start making sense. They were there too. Of course they have the advantage of remembering.

Some of Dad's pictures are so good they deserve to be called out specially.

That's my grandmother, Lucy Kiesler, on the left, and my mother, Eve Winer, on the right. My brother Peter Winer is sitting with his back to us, and the guy in the loud 70s style coat is me, at age 21. The picture was taken in New Orleans, not sure exactly where (possibly Commander's Palace) just before or after my college graduation. Click on the picture to see the full image. And the full set of pictures, uploaded yesterday, is on my Dad's memorial site.

Another observation. My Dad did a great job of organizing all the pictures. All I had to do was write some scripts to merge the captions, and link the thumbs with the originals. I'm storing all the stuff in Amazon S3 because I think it's the most reliable storage we have right now. Should I die or become incapacitated they will just keep billing my credit card, and hopefully my successors will just let the charges go through. Amazon really ought to allow people (as opposed to companies) purchase perpetual hosting. What a contribution to our culture they would be making if they did so.

What's obvious about netbooks

Wed, 14 Oct 2009 14:46:57 GMT

Some people in the tech industry believe netbooks are a mistake that will be corrected any day now. Problem is they've been saying that for hundreds of days while the netbook market keeps growing, as the market for more expensive portables stagnates. The trend won't reverse because netbooks represent a far more fundamental shift than they recognize. Far more significant than I've realized until just the other day.

12/17/08: What is a Netbook?

At one level, a netbook is just a new product, something between a notebook and an iPhone. It fits easily into a briefcase or knapsack, has a keyboard and screen that while not exactly spacious, are functional.

The iPhone is a compromise, a good one because of its supreme portability (it fits in a pocket). But netbooks are a good compromise too.

An aside, I know some people use iPhones for their networking. Scoble is my prime example. I can't get him to use my software these days because it runs in a desktop style web browser. Of course what he's doing is a reasonable choice. But it's never worked for me because communicating on an iPhone is too slow. Not just the typing, not just the cramped keyboard, but the actual networking. I had an interesting experience the other day on my mother's new FIOS home network, all of a sudden I could use the iPhone the way Scoble does. For some reason it doesn't work very well on my network at home, and the net connection from AT&T is infinitely too slow to be usable. So it could be that my iPhone experience is much worse than most people's.

As a product a netbook is less expensive, more portable, and has longer battery life than a notebook.

It has a bigger keyboard, bigger screen and is a better reading and writing tool than an iPhone.

It's in the middle -- and there is a middle -- a product between an iPhone and notebooks

But the netbook is not just a new product. It exposes something pretty ugly about the computer industry. They've been controlling prices. There must have been price collusion before netbooks came out. It's clearly possible to build notebooks for much less than they manufacturers are charging.

That's a matter for the Federal Trade Commission to look into, but even that is not what's going on with netbooks. The truly significant thing is this -- the users took over.

Let me say that again: The users took over.

I always say this is the lesson of the tech industry, but the people in the tech industry never believe it, but this is the loop. In the late 70s and early 80s the minicomputer and mainframe guys said the same kinds of things about Apple IIs and IBM PCs that Michael Dell is saying about netbooks. It happens over and over again, I've recited the loops so many times that every reader of this column can recite them from memory. All that has to be said is that it happened again.

Once out, the genie never goes back in the bottle.

This should serve as a lesson to the architects at Twitter and Facebook. The day will come when your users figure out that they can do what you do without the costs you impose. Better to prepare for that day, factor it into your economics, than be surprised by it as Michael Dell appears to have been. He's complaining about netbooks probably because his expense structure can't sustain the business. He's a leading vendor of netbooks, yet is selling against them. This means only one thing, and it's kind of obvious -- he's losing money on each sale. He can't afford to be in the business. Which means he can't afford to be in business at all.

Windows 7 may be nice, I don't know and I don't care. I like my XP-based netbook just fine with its 10 inch screen, 160GB hard drive and 8-hour battery life. My computer cost $350. I'm not a likely customer for an upsell. Those are the new economics, like it or not.

An open letter to Om

Wed, 14 Oct 2009 15:26:23 GMT

In a thread about netbooks on his blog, Om asks what he's missing in his analysis of the relationship between the computer industry and users re netbooks. I spent some time thinking of an analogy, and came up with one about the industry we all love to hate -- the airline industry.

So here's the story...

Once upon a time there was an MIT professor who said: "I have an idea, let's design an airline that can get you to a great resort for $100! Then every schoolkid can afford to have a great vacation in Mexico, Italy or Thailand!!"

No matter how hard he tried the professor couldn't come up with a solution. The moguls of the airline industry sighed in relief. Their margins were safe.

Not so fast! A Taiwanese airline named Susa discovered a way to do it for $600, which was still a lot cheaper than the airlines could get you to Italy or Spain for. Huge numbers of people signed up and the airline bustled with new business. People came home from their holidays and told everyone that they had a great trip. And with the new economies of scale Susa was able to get the price down to $350. Not the $100 that the MIT professor promised, but still cheap enough that lots more people could have a vacation, and while some people still went on the more expensive and luxurious airlines, more and more people were flying Susa.

In fact this happened in the airline industry, and it forced a lot of the big airlines to either merge or go out of business. Maybe analysts in the airline industry say that any day the users will tire of the cheap flights, but they'd be just as wrong as the analysts in tech who predict the demise of netbooks.

What I've learned about Hyperlocal

Tue, 13 Oct 2009 15:01:01 GMT

First, the news -- InBerkeley.com is coming to an end.

Other projects are consuming more of my time. And in the last couple of months, family stuff has taken me away from Berkeley, and I'm not sure where my attention will be in the future. So a big part of my decision to move on is personal.

But I've also learned why sites that we're calling "hyperlocal" are difficult, and why I failed to get the site to grow the way I hoped it would.

I thought we could apply the same approach that worked in bootstrapping weblogs, RSS and podcasting for a local site. One or two people start writing about their personal experiences. A small audience develops. Debates, discussions follow. More perspectives. At every step you invite people to participate. You always ask for the people who used to be called the audience to become full participants. That's how the idea scales. As I said, it worked for blogging and related technologies.

Instead, what happened at InBerkeley.com is that the people thought we were running a news organization, and they did stories the way reporters do them. That can't possibly work, imho -- for the same reason the news industry is in crisis.

An example. Suppose you're writing a story about a parade on Shattuck Ave at noon and you don't happen to be there, you just heard about it from a friend, but feel you have enough information to write a story. Now you want a picture for the story -- in fact, you think that without a picture you can't run the story. The reporter will hold it, but the blogger runs it anyway, and at the end asks if anyone has a picture. It's that little difference that makes the hyperlocal idea scale, and it's what my colleagues at InBerkeley were unwilling to do.

Instead, we got in trouble, twice, for taking copyrighted material from other sites. The first time it happened, I apologized, and nothing terrible happened. The second time, I decided we had to shut the site down, because we were doing it totally wrong. I want to stress, this is my opinion, but as one of the founders, it's my reputation that's out there. I didn't think what we were doing was noble, or even very good. Not something I was prepared to stand behind.

Further, I was sure that at some point I would be giving a talk and there would be a reporter in the crowd who would ask how news can reboot if it's dependent on scarfing copyrighted work from pros. Now if I get asked the question I can say I think it's possible, but we failed to prove it at InBerkeley.com. And I'll be telling the truth.

The other people at the site, including my former partner Lance Knobel, are going to start a new site, and I wish them the best. It's possible at some point I may even contribute, if they want my contribution.