Every time I write about Sources Go Direct, like clockwork, someone asks how will we get Woodward & Bernstein reporting without great reporters sniffing around for stories that bring down Presidents.
Never mind that it only happened once and hasn't happened since. Only Richard Nixon has been forced from office by the press (or anyone) and it turns out that had W&B not been sleuthing, the NYT had the story too, but they blew it. So it's possible that without W&B, Nixon would have remained in office. However it's hard to imagine that he could have been worse than Bush II, who the press of today did not bring down. (A fact the defenders of projournos never address.)
In the rebooted world, Deep Throat, the source, might go direct -- anonymously, using a proxy server, possibly -- and a Twitter account created for just such a purpose. BTW, I don't assume there won't be reporters in the rebooted world, but I do assume the sources will be able to go direct if they choose to. And in this case I don't see why they wouldn't.
If we're lucky enough to have a Woodstein out there digging, let's hope we have the sense to listen to them. And if we don't get blessed or if the economics of projourno crumbles, and Deep Throat wants us all to know how screwed the President is, let's make sure that anonymous pathways exist so they can say what they know and stay in place to keep reporting.
I write this from Berlin, a place that has seen huge change, many times, in the last 100 years. The kind of change that would make Americans weep with fear.
It's 9:11AM as I write this, and back home in California, it's just after midnight. I've done this so many times, but it still seems something of a miracle. How a day can be starting and another day be ending, all at the same time.
Glad to see Scoble blogging again. I predict a return to blogging as people discover the power of being able to finish a thought, and to link to another site without going through an intermediary. Once again people will discover the power of Small Pieces, Loosely Joined.
Chris Anderson is right, of course -- and uses a device to bait the press to object. Good way to sell books, perhaps, but I'd rather he'd not taken the shortcut. Malcolm Gladwell objects, and my friend Howard Weaver says that Gladwell got it right. I don't agree. When you think of news as a business, except in very unusual circumstances, the sources never got paid. So the news was always free, it was the reporting of it that cost.
What Anderson calls "Free" -- we in Rebooting The News call Sources Go Direct. Absolutely nothing strange about it. The Internet always disintermediates. Did you see the "media" in the middle of that word? It's the middle that's hurt in the new world. Sorry. The new world pays the source, indirectly, and obviates the middleman.
Following up on yesterday's piece about Wikipedia, I wrote it, unintentionally, so that people reacted to the bit that's uncontroversial and missed the controversy. Of course it's good that the reporter escaped and his life was spared. Who could think otherwise.
Here's the controversy:
"What about when information on Wikipedia, true or not, hurts people in other ways? Why shouldn't anyone be able to get whatever they want redacted?"
Clearly some people can get stuff redacted sometimes.
Who, and under what circumstances?
Further, no one has that power over the web. Would it be better if Wikipedia worked like the web?
I always get a cold at about this point in a Europe trip. The jetlag must weaken me, and all the walking and sweating and temperature changes, and so forth -- I get sick. And I always do the same thing, go right through it. I'll have time to get over it when I get home.
Right now I'm spending my first full day in Berlin. So far I like it.
Two news items caught my eye this morning. One a controversy in the press over whether the President had the right to call on a Huffington reporter second in his last press conference, when it seemed as if the President had an idea what kind of question he might ask. Did anyone breach some ethical requirement.
A much bigger issue, I haven't heard any reporter ask about, yet it is no less incestuous than the previous issue. Did the press do anything wrong in covering up the kidnap of a NY Times reporter by the Taliban? Assuming it wasn't wrong (since no one seems concerned) what is over the line? What kind of person wouldn't receive this kind of favor? And then this morning we learn, from the NY Times, that Wikipedia did the same thing, and it was a much more complex affair. And this raises all kinds of other questions. What about when information on Wikipedia, true or not, hurts people in other ways? Why shouldn't anyone be able to get whatever they want redacted? I don't think there's a bit of difference between news organziations and Wikipedia. And let's hope they're ready with an answer, or are they just making it up as they go along?
My feeling, Wikipedia suffers from the same centralization of authority as Twitter. No one should have the power that Jimmy Wales has, to shut down Wikipedia as a conduit for truth. It appears from the NYT story that, at times, they had verification that the reporter had been kidnapped. At that point, they had no choice but to include it, whether it jeopardized the reporter's life or not. It's a line you can't define, there must be many situations where the presence of information on Wikipedia puts people at risk, just as information on the web puts people at risk. The difference is there is no central authority on what belongs on the web, and there is one on Wikipedia.
Same problem on Twitter. Reading Fred Wilson's interview with John Battelle, I'm horrified at how involved the management of Twitter feels in the flow of information and ideas on Twitter. If it were just the web, it wouldn't matter -- his opinion is just one person's opinion, and it can be balanced by other people's point of view. But in Twitter, some people's opinions are much more important than others' and Fred is one of those most important people. This is unacceptable. There's no other word for it. I have a feeling that when the reporters on Twitter are aware of this they won't like it either, but so far I don't think they are.
Last night, unable to sleep due to jetlag, I watched Lives of Others, a movie set in East Berlin. It was a beautiful movie about the contradictions that come up when thoughtful people are tasked with controlling the lives of other thoughtful people. This is where we're going in Wikipedia and Twitter.
Amyloo pointed out that arriving in a new city would have been a blog post three years ago. She's right. I think it deserves a blog post this year as well.
Today, after breakfast with Thomas Madsen-Mygdal, I left for Berlin, which I can already tell is a spectacularly beautiful city, even though I've only been here an hour. I'm staying at the Grand, a beautifully restored historic hotel where one of my favorite movies of the 1930s was set. Getting off the elevator, I laughed because I immediately recognized it as a set where Joan Crawford, Greta Garbo and John and Lionel Barrymore performed their magic. It's a gorgeous hotel. I can't imagine it looked any nicer in 1932.
I have a very pragmatic reason for wanting to post something on the blog. I am here in a strange city on my own. Reaching out to any readers of this blog, let's have a dinner. I'm here for three nights. I love beer. I hear there's some good beer here!
Myth: Twitter is perfect. If you change even one thing its magic disappears.
Apple products are often thought to be perfect. Steve Jobs had a lot of people convinced that video had no place on the iPod, until he revealed the iPod with video. All of a sudden it was the Number One most sought-after feature.
The myth of perfection is mostly skillful marketing.
The reality: We Make Shitty Software.
Software is a process. It's never done. "Our software is shitty," the honest hard-working developer says. "But watch, we'll make it less shitty." You're buying a process not perfection.
This has practical applications. In the last Rebooting The News podcast, toward the end, the subject of changing Twitter came up. When I was able to convince Jay that Twitter could be changed, he said his #1 most desired feature was to be able to edit a tweet.
Twitter will get more features. Bugs will be fixed. You can be sure of that.
One of the most obvious features it will get, and I'd bet really soon, are groups, which are similar to features already implemented by some of the clients.
What's your number one most desired feature?
PS: This discussion is a continuation to the the piece I wrote earlier this week about the 140-character limit.
I think I must have been Danish in a former life because when I arrive in Copenhagen there's something familiar about it, comfortable, nice. You get a sense that people live well here, don't know what it is, but Copenhagen puts me in a good mood.
Tell you another thing, if you can time your arrival for 7 or 8PM you can do the nicest thing re jetlag -- immediately go to sleep for a full 8 hours and if it's midsummer, almost never have it go dark on you! That way if you get up at 3AM it feels like 6AM back home in California, but that's pretty disorienting because it's actually 6PM.
The right way to think about the trip from SF to Copenhagen: The flight takes 10 hours, real time. Add another 9 hours lost to timezones, and it's at least a 19 hour trip, including the fact that you have to go through somewhere else to get here. Neither city is large enough to have direct flights between them.
Tim O'Reilly says: "At the end of the day, folks like Scoble and Winer are unhappy because they aren't on the list. It doesn't feel fair to them, so they do the next best thing, seeking publicity by complaining about it."
That's not true.It's far more complex than that. O'Reilly should stick to speaking for himself.
I'm writing this on a plane that's boarding now, heading for Frankfurt from San Francisco, so obviously this is not a debate I will be able to take part in, but I did want to clear this up.
PS: Sprint MiFi is wonderful. I left it on in my knapsack which is in the overhead compartment. Scoble called on my iPhone to alert me to this. I whipped out my netbook and launched my editor and quickly wrote a blog post. We live in wonderful times.
I'm leaving tonight for Copenhagen to participate in the Reboot conference. This will be my third Reboot. It's a very nice group of people, very far away from Silicon Valley, and I always have fun. Looking forward to partying with Thomas and his posse and Paolo, Stowe, and everyone else. I'll be leading a talk on Thursday evening on Rebooting the News.
After Copenhagen, I'll spend three days in Berlin, then head back to the US via Chicago on July 1.
See you on the other side of the world, tomorrow night!
PS: I recorded a podcast with Phil Windley of IT Conversations last Monday. A little bit of time has gone by but I think it's pretty good. We talked about the technical side of Rebooting the News.
I wrote my first RSS aggregator in 1999.
Believe it or not the core of that aggregator is what's behind the aggregator I've been shipping in the OPML Editor. Since then I've written all kinds of specialized aggregators, and it turns out it's not that much work these days.
Rather than live with all the decisions I've made over the last 10 years, I started over. The result is River2.
I just completed the first version, which I'm calling the "designer's release."
Every design element can be changed through CSS.
You just save your change, refresh the page, see the result.
The web server runs on your desktop, inside the OPML Editor.
To get an idea of what you're working with, my copy of River2 saves its home page to a public server every ten minutes. Yours will look like this too, until you change the design!
So if you're interested in designing the look of a River of News aggregator, it's ready for you to try it out.
If you have questions or comments, leave them here, or in the comment section on the page above.
That's like Be Kind Rewind.
And of course everyone retweeted this and everyone clicked.
Nothing here. Move along.
I love telling stories, especially ones with happy endings.
Once upon a time, way back in the early 80s, a young man (me) had written a program called ThinkTank. It ran on the Apple II, which only had 48K of memory -- not very much when you consider that an average PC today has 1 gigabyte -- or 21,845 times the memory if you can believe that!
That's like comparing a single 140-char tweet to the Library of Congress.
The Apple II had an infintesmally small memory, but its disk was a little larger. So the operating system I used, the UCSD P-System, did "overlays," which allowed big chunks of code to stay on disk until they were needed. When code in an overlay was called, the OS would throw out another chunk of code and replace it with the one you called. So, in the worst case, if a command needed code in two overlays to solve a problem that involved looping, the disk light would stay on for a long time while the computer "thrashed" out the answer.
This isn't unlike the way an Amazon Kindle keeps part of your library on its computer and part of it on the Kindle itself. When you want to read one of the books on their computer it just downloads it again, replacing something you haven't read in a while.
This business of writing code in overlays was very taxing to the developer, because thrashing wasn't very good for the usability of the code, so you're always moving code between overlays, or making a copy of an often-used routine, all to prevent the disk light from coming on and thrashing the app (and its user) to a standstill.
This clever code-writing is a lot like writing 140-character tweets today. You delete and abbreviate, throw out important ideas, all to fit into that tight little space. And then your readers, like the disk light, thrash with confusion, and think you're a fool, because you have to be a genius and a mind-reader to figure out the gibberish you wrote to fit in 140. Oy!!
So, with the app in the Apple II days, it was often too much trouble to add the feature. With Twitter, it's often easier just to say nothing. And that's not the goal of blogging, macro or micro. The goal is to provide a platform for saying what you have to say, not for not saying what you have to say!
Anyway, the Apple II story had a happy ending. It was called the IBM PC. Instead of 48K it had 640K. So when I recompiled my app for that machine I just threw out the overlays and let all the code reside in memory and the thing ran like a bat out of hell! I was finally able to finish the features I wanted, and instead of thinking the program just had potential, people loved it, and it sold, and we raised money, and everyone was happy.
Jonathan Edman tweets: "I deeply understand how crippling 140c is, but what is the right number? Don't you run into the same problem at almost any num?"
Since my answer is too long to fit in 140 chars, I answered here.
Jonathan, I don't know what the "right" number is, but I have some ideas.
First, almost anything above 140 would be seen by power Twitter users as an improvement, and a cause for celebration. It would be a sign that someone is listening. And it would immediately give us relief. It's as if, in 1981, Apple found a way to give us 72K instead of 48K. There would be a burst of creativity like the Summer of Love.
Now, here's what I would do first, to try to come up with the right number.
Read the feeds of the NY Times, BBC, and a few other professional news sources for a few weeks, and count the characters in the <description> elements of each <item>. Average the number. Double it. That's what I would go with.
The theory being, if professional writers can summarize a whole news article in, X characters, then the average person should be able to express an idea in 2X characters.
In my new River of News, I cut the intros off at 280 chars, arbitrarily, and it seems to work pretty well. Previous versions included full posts, and that was a problem, because some sites, like OpenLeft, write whole books in their posts. I also strip out markup. I'm tired of all the huge pictures people are throwing into the river. I see it as a gimmick to try to get more attention. I say let their ideas compete with everyone else's on a level playing field.
Show notes here.
I was walking on Solano Ave approaching Alameda when I saw a few people looking at an accident scene down the block.
Several fire trucks, a police car blocking traffic. I could see a bicycle and an ambulance. The other people didn't know what had happened beyond that there had been an accident.
I took a picture of course.
So now it's an hour later, I'm at the computer, and poking around the City of Berkeley website, the fire department, police, Alta Bates Hospital, and come up with nothing. There's no record of what happened, or is there?
if there is a log -- where is it?
If not, who do I talk with about getting one going?
It seems this is the most basic beginning to having an effective local website.
Update: There is a public information officer in Berkeley.
Update: Berkeley Crimelog. (!)
It's great that the MSM has adopted blogging tools to cover the Tehran protests, which seem to be ending, perhaps tragically.
However, for next time -- can I suggest that they create an RSS feed for each flow where each mini-post is its own <item>. That way we could easily follow multiple flows without having to refresh all those pages.
Scripting News started as a link blog, so you'll find plenty of prior art looking at its archive. Here's a folder that contains the RSS archive for 2003. (View source, today's browsers totally mangle the display of XML, in the name of progress. Oy.)
Great piece yesterday in CNET about how Twitter is no longer young. Paradoxically true and a must-read.
Jesse Stay has an interesting piece on how Twitter is going after people who game Twitter to get more followers. It's a good piece, well worth reading carefully and understanding. And I support what Twitter is doing. But...
The problem is that Twitter is the worst offender here with the Suggested User List.
I'm watching a NY Times columnist, who was added to the list last week, leapfrog his competition. It changed the way he posts. (He openly says that, he may have been joking, but you should watch those jokes, they usually reveal some truth, that's why they're funny.)
NY Times: "A writer with an interest in comic books can become the expert on comic books."
How long before the professional gamers privately start paying people who are on the SUL to point to them? (My guess is that it has already happened.)
What are the editorial guidelines for people on the SUL?
And why would Twitter want to enter this space? And are they ready to take an editorial interest in the people who use their system. This is why lines exist in journalism, to keep the publishing interests from having to worry about the editorial interests. Inevitably, the lines get crossed, you can't avoid it, but you try to avoid it. Twitter made a huge mistake by crossing the line with such gusto. Now you can see them approaching the contradiction. They want to stop users from doing what they themselves do so much better. Can't make that work very much longer.
Net-net: They will eventually have to publish guidelines for SUL members. Watch for a rebellion from those now very powerful people, who will neither want to give up their power nor submit to guidelines from Twitter.
This subject came up earlier this week when @anamariecox admitted that the White House treats her with new deference because she has 650K followers. A couple of months ago she had 3K. So the change is significiant and clearly due to the gift from Twitter.
Update: Getting real, we know they already have implicit editorial guidelines for the SUL. It's why people like me, who are unpredictable, will never get on the list. They don't know what I'm going to say, and they might not want to stand behind me. That's the problem, because they don't know what anyone else will say either. Sooner or later someone who they propelled to the top will do something bad. It has to happen. And that's why they needed a really strong separation between the platform and the content, and the problem, for them and the platform, is they have no separation at all. A major oil spill is inevitable.
8/26/00: "What's a Mind Bomb? An idea that's so strange or powerful that it explodes in your mind. And that's a good thing!"
Question to professional reporters: If your publication is on the SUL, or were on the SUL, would you submit to editorial guidelines from Twitter, Inc?
I've done some more work on the CSS-in-Rivers project.
I'm sticking with the plan. I'm going to have a new tool that makes it really easy to configure the CSS in realtime, without having to change any code, so people can play with a real aggregator and hack up its appearance.
You can see the result in the public page, which is updated every 10 minutes.
I expect to release the tool before the end of the weekend, Murphy-willing.
PS: Yes I know it's ugly! By design. To make you want to change it.
Everyone who's tried it says it works, so I'm giving it a go.
Here's how to do it.
0. I have an iPhone 3G, not a 3GS.
1. Visit this site in the browser on the iPhone. Follow the instructions to install the configuration file it needs for the country you're in. (I'm in the US, of course.) Took me about a minute. Most of that was reading the various instructions, warnings and disclaimers.
2. Then I followed the instructions from Apple to turn it on in the iPhone user interface. Easily done.
3. Now I'm going to see if I can pair the iPhone with my netbook using Bluetooth. Back in a few minutes.
4. As with everything on Windows it takes a bit of fussing, doing things a few times, but it works.
5. Now I have a $400 toy that I no longer have any use for? :-(
The first episode of the new podcast ready to go!
And it wouldn't be Bad Hair Day if there wasn't a major glitch in the show, right at the beginning.
Might as well get off to a Bad start!
Yes, as they used to say It's even worse than it appears.
But it was a good show, some might even think it had moments of greatness.
Here's the RSS podcast feed.
If you're going to subscribe in iTunes, choose Subscribe to Podcast in the Advanced menu and enter the RSS link above. That's it!
Read the show notes here.
Wishing you bad hair, today and in the future!
I upgraded my iPhone last night to version 3.0. Everything seems to be working but there's no camera icon on the desktop. I'm lost without my camera. Help!
Update: The ultimate fix was to go to the Settings app, General/ Reset/ Reset Home Screen Layout. That brought the camera back.
I'm starting a second series of podcasts about tech with Marshall Kirkpatrick of ReadWriteWeb. We're recording the first show tonight. You'll be able to listen live, but there will be no call-in. There will be a feed, of course.
Every Thursday at 7PM, Murphy-willing.
We'll follow the model of RTN, the weekly podcast I do with Jay Rosen, but we plan to expand the cast beyond Marshall and myself. But the first show will be a duo.
The name of the show is BadHairDay. As I say in the teaser, that's every day for me. I'm pretty sure Marshall has good hair. So that balances things out.
Here's a list of things I'm interested in talking about in the first show (no way we'll get to it all): iPhone 3.0, tethering, netbooks, Twitter clones, backing up Twitter, Hackintosh, Google Wave, Any hope for Yahoo?, Opera Unite, desktop web servers.
Marshall has his own list.
We'll be doing the show on BlogTalkRadio.
The website for the podcast is http://badhair.us/.
The feed will be here (no shows yet): http://badhair.us/rss.xml.
This morning I posted a query about CSS that would make my River of News aggregator look beautiful. It was hard to communicate what I was looking for. So I've decided to take a new approach.
1. I'm going to use tables. This really is an application for tables. That was made clear in the discussion. If, when we're done, someone can show me how to do the same thing without tables, I'll change to do it that way.
2. I'm going to provide a style sheet in the app, but I'll make it very easy to have it use your own. That way people can tinker with the real live working app while it's running and share the results for others to see.
3. If anyone comes up with a really fantastic way of displaying the River of News with CSS, I will use their CSS, with full attribution and accolades, and release the result under the GPL, including the aggregator. Then we'll have a beautiful River of News aggregator that's available in open source.
I've started to work on this approach, and will post when I have something you can install.
Yes, the first and third columns are necessary. I haven't filled them in yet.
I'm re-doing the way the NewsRiver aggregator displays the most recent news. Up until now it has used tables. Now I want to use CSS.
I've uploaded the table-based version of the page so you can see what I'm starting with.
I'm looking for examples that do something similar, in CSS. All pointers are appreciated. Help me get this right and I'll publish the results, as an OPML Editor tool, open source.
Update: Thanks for all the ideas! Based on the discussion, I've got a new plan.
Thirteen is a lucky number when it comes to revolutions!
Go get it! (And it's in the scripting.com feed, too, as always.)
Update: Dan Conover transcribed one of the funnier moments from the podcast.
Never has it been more clear -- we are building a dangerously precarious centralized system that will, given everything we know about computer networks, at some point, fail. It's so important now that the US State Department got in the loop in the last couple of days.
Meanwhile there's an incredibly vibrant competition in the Twitter client space. At least three leading apps: Twitterdeck, Seesmic and Tweetie, are slugging it out. Each with strengths, waves of new versions, users comparing products, always something new to look forward to. The kind of rapid evolution we desperately need in the back-end.
There's a little bit of Facebook in the mix (it has a lot of users, but not many of them use these clients, I think) and yes there is Identi.ca, but it has a very small user base compared to Twitter and Facebook.
In a thread that was spawned from a Twitter post earlier today, we talk about the possibility of a competitor to Twitter coming from Google or Facebook. Not sure who else could launch a back-end that would find enough support among users to gain critical mass. And I agree, totally, with Don Park, that if Facebook wants to play, they must start from scratch, with a totally simple system that matches Twitter, and adds stability, performance, beauty, or a few sought-after features.
Google would compete by building a system out of components of the open web, the small-pieces-loosely-joined approach. I outlined how this would work in an earlier blog post.
I love Woody Allen movies, more so over time, as I grow older, they seem to get better. A couple of years ago I went through them all, Annie Hall, Manhattan -- classics, but there were also some surprises, some great movies that I didn't remember as being great. I pretty much liked them all.
This weekend, I finally saw Vicky Cristina Barcelona, which got mixed reviews, but I loved it. On Twitter someone said it's just a beautiful postcard of Barcelona. Agreed, and what's wrong with that! People who love art somehow can't forgive a movie for not being heavy on story, but rather leaving an impression. Those are some of my favorite movies, they're more like paintings or postcards. Here, look at this scene and now look at this one. If it's beautifully done, if the acting is superb and the story convincing, as it is in VCB, what's not to like?
So, when I saw that Woody Allen was the guest on Fresh Air, I savored it. He doesn't do many interviews, and this one was disappointing.
Terry Gross went for the scoop. She wanted him to slip up and confess something about his personal life, so she repeatedly asked probing questions, which he skillfully and for me, painfully, dodged. This is the interviewer interfering, getting between the subject and the listener -- preventing the subject from talking about what the listener is most interested in. With Woody Allen, that would be movies! Who would be a better person to just let ramble about the art of movies. To remember his favorites, or what it was like to work with the writers and actors he's worked with.
There are little bits of this -- the script of his new movie was originally written for Zero Mostel, but he died before they could make the movie. You get a little peek behind the scenes, how he works, his craft, and how it relates to Mostel's.
Gross often nails it, where other interviewers are selfish, she lets the subject be the story. But not this time, unfortunately.
Oh the NYT. They do such a great job with the news, but they do such a terrible job of running the business.
In the last few days while CNN et al completely dropped the ball on the Iran story, they were right there, on top of it. Great stuff.
Everyone else in the news business missed the Twitter SUL story, but the Times nailed it. I was so happy I can't tell you.
But in the meantime they're cutting the pay of Boston Globe reporters, and have no idea how or if their business will operate next year or the year after.
All this at a time when their product is in high demand. People love news, and we love the way the NYT does the news. So why is there a problem?
Oddly enough, I know, and I can tell you.
Get your coffee, have a seat, let me tell you a story...
Three years ago I got a Blackberry and fell in love. I was riding all over the place on the BART system and I could take the news with me. It didn't take me five minutes to realize it was the perfect River of News device, so I adapted my NYT and BBC rivers to work in their browser.
Unlike most developers I have the phone number of the CEO of the NY Times Digital, so I rang him up and told him how wonderful the Times was on my Blackberry and please please let's tell the world about it. After all he had an incredible communication system for doing exactly that. I wanted to fly to NY to show it off, but he said we should have a phone conference first. I thought this was a bad idea, but I did it. I shouldn't have.
I have no idea who was at the meeting, but the first thing they did was tell me about their upcoming mobile version of the Times that they had spent millions developing. Right off the bat I knew it had to be terrible. The only way to spend that much money on a mobile news site is to put all kinds of hurdles between the reader and the news. I said I had a totally simple way to do it that I had developed in a couple of days, by myself. (I lied, it actually took about an hour.) Then they asked what I wanted. I knew we were headed off a cliff. I said that isn't important, they pressed, I said yes -- I probably did want to be paid for my work. That was the end of the meeting. They were off the phone in less than a minute. I'm sure their version of the story will be different. But the net result was indisputable. They waited over three years before they had a reasonable way to deliver news to mobile users.
Yes I know they have millions of people reading their mobile site. But I'm talking about something else. I'm talking about the backbone of news delivery, and today that's indisputably Twitter. The stupid thing about our meeting, the lose-lose about it, is that right then and there we were on the edge of inventing it. And because I didn't get on a plane (my mistake) and because they had so much invested in doing it the wrong way (their mistake) we didn't do it.
So the first-level problem for the Times is they now are authors for Twitter, doing great work, and not being paid for it. Once again, they're going to be complaining, soon, that the tech industry is pocketing the profits while they do the work. (They'll be wrong, a lot of other people are working for free too.) The higher-level problem is they aren't competing. They're just sitting there. Spending money in obvious and wasteful Dilbert-like ways, and letting the small nimble competiton run circles around them.
I would, if I were them, ask their Twitter users (and they have quite a few) what was so wonderful about Twitter as it covered the Iran story. Ask them to explain the role the NYT played in it, and if it was generally appreciated (they were great, and in general it wasn't appreciated). And then, and this is the key question, ask them how it could have been better.
There's still an opportunity to create the news system of the future. But only if you're very smart about it.
In this morning's NYT, there's an article on Twitter's SUL. It's excellent. I recommend everyone read it carefully.
Here are some excerpts.
"Twitter has become a kingmaker of sorts, conferring online stardom to a mix of writers, gadget geeks, political commentators and entrepreneurs."
"...an actor like LaVar Burton, decades away from his glory days as a star of the TV drama 'Roots,' has a personal audience of 635,000."
"A writer with an interest in comic books can become the expert on comic books..."
"Did he realize he was helping to create an arbiter of popularity? 'We didn't think that far ahead,' [Biz Stone] said."
"The list is cobbled together by a team of employees whose identities were withheld"
"Ms. Sampson said 'there's sort of a criteria' for the list 'but not really.'"
Finally here are some comments and background that led to this.
A lot of people told me to stop writing about it, but Twitter's Suggested User List was just plain wrong, and I was sure that it would become more evident over time, and it has. Here's a brief recap.
1. Until early this year, follower-count was evolving as a user-developed way for Twitter users to give authority to each other. Like all things in Twitter, it was crude, a better version could have been designed, but that's the way things work in Twitter.
2. People like Scoble, Guy Kawasaki, Jason Calacanis and Leo Laporte, and to a lesser extent myself, brought authority with us from other places. In this way we were investing in Twitter, every bit as much as Union Square or Spark Capital.
3. The major tech pubs (Mashable, Om, TechCrunch) mostly ignored Twitter. They had much lower follower counts than the people above. Same with the celebs, Oprah, Ashton Kutcher, etc who weren't present at all until early this year.
4. Follower-count worked very much the way eBay users rate each other, same with Amazon.
5. The reason follower-count was so big was its huge visibility in the user interface. It was the biggest number you'd see when you clicked on someone's profile. In FriendFeed, where follower-count is visible but fairly buried, it isn't a big part of the culture. (I don't have any idea what my own follower count is in FF, in Twitter it's about 23K.)
6. Then Twitter adds the Suggested User List to the mix. The way I discovered it was noticing that @anamariecox's follower count, which had been around 3000, had jumped to 40,000 then 50,000 then 60,000, all in a matter of days. No one could figure out why until @ev posted a comment on a blog explaining. Then we could see the effect all over the place. All kinds of random people were jumping in follower counts only because they were on the SUL.
7. I wrote a piece on March 12 asking if a reporter could accept so much extra juice, for free, without disclosing.
8. Then the celebs come. Kutcher's campaign. Oprah. Cover of Time. Etc. Twitter explodes. Good for them. In the meantime, our investment is swamped, more or less lost. I don't begrudge celebs for their followers, as long as they earned them, as long as they brought users in as we did in step #2 above. What I object to, what anyone would object to, is Twitter gaming the system to favor people who did nothing to earn their follower count.
9. That's where it stayed until this morning when the NY Times ran a piece that more or less lays it out the way I've told the story. They add things I wasn't willing to add -- the utter incompetence and lazyness, lack of thought, lack of caring in any way about the users of Twitter. It's time for some sobriety. People don't like being pushed around like this. I've looked into the thought behind the SUL and found the same thing that the Times did. There is none. It's insiders doing favors for other insiders. It's newly rich and powerful entrepreneurs throwing their weight around, rewarding people and publications that do their bidding, and punishing those who are independent.
10. What is so frustrating about this is that Twitter has this incredible promise as a platform for journalism. How ironic that the NYT piece comes out as the people of Tehran are using Twitter, putting their lives on the line, to route around a corrupt government.
11. At some point there must be a way for users to convey authority for other users that isn't spoiled and polluted like Twitter's follower-count is. PageRank for people. We were bootstrapping a way to do that until the company blew it up. This opens the door for Twitter's competitors to take advantage, and come up with a way to convey authority that isn't subject to gaming. No vendor should put their finger on the scale to favor one person or organization over another. I want a level playing field, I want as much of a chance as anyone else. When I find out someone else is cutting in line, I lose all interest. I have lost that kind of interest in follower-count. Give me another field to play on, this one is spoiled. I don't see any way for them to fix it. We're going to have to start over from scratch to do that. Shame. We had a really good thing going.
Amazing photos coming out of Iran on Flickr.
Still getting a few analog stations.
An interesting discussion over on FriendFeed, spawned from a series of comments I made yesterday on Twitter about the cyclic relationship between the tech press, the tech industry and the users. I think this time around the loop things may change for good, the cycle may just break.
John Blossom: "Journalists stay in business by cultivating relationships with sources - that's a pretty universal fact, not just with the tech press. It's always a dance to avoid getting too close and cushy for the sake of something less than pure motivations. In today's environment, though, the multiplicity of online channels in any market segment in conjunction with purely social media buffers us against this kind of corruption. As soon as someone spoons too lovingly for something they get outed."
My response: "John, that's why blogging took off -- because the tech press was so rotten with the vendors, they'd never say anything negative about them. So when you wanted to find out if a product really worked, you'd do what we do now -- listen to other users. Amazon built an empire on that idea. Of all the Web 2.0 companies they may be the only ones who get that the press doesn't control what users know anymore, that the users are getting it for themselves."
The cycle of users taking control of the tech industry and press goes back a long, long way. My first experience was in the late 70s, as a grad student in comp sci. I'd go to the student library, a quiet reading room where they left copies of the computer industry publications. I remember leafing through them thinking that this stuff seemed overly complex and irrelevant. When my generation went out into the world, we started over. That's the cycle.
It has always seemed possible to me that a company could make the transition from one generation to the next without getting caught in the gears, but I've yet to see it happen. Maybe the closest is Apple, but they went through hell between the advent of the web which overturned a lot of their assumptions and the rebirth of Apple under Steve Jobs 2.0.
These days the press can reform itself very quickly because the printing presses are very cheap. It cost News.com millions of dollars to start up in the 90s. TechCrunch started four years ago with nothing but a Wordpress installation and an entrepreneur with a little extra time. So when the press gets too cozy with industry, the next layer of the press forms in an instant. When users want to know if the products really work they just inform each other.
BTW, much of the press now calls themselves bloggers. They can do that, no one owns the trademark. But that doesn't mean they are immune to being routed around by bloggers. It's as if you changed the name of "rain" to "sunshine." You'd still need an umbrella.
Latest news -- we were about to get a Charles Hotel, run by the same people who run the Charles in Cambridge -- a truly classy hotel (a great place to wait out a snow storm). At the same time I found out about it, I found it's been cancelled. Oy! It's almost too much to bear.
Lots of new posts on InBerkeley and now you can follow us on Twitter, and never miss an update. I'm having so much fun with this project.
I just posted some pictures from my evening walk. Lately we've been showing newly vacant storefronts. In this walk I show two recently empty stores that have new businesses. One a new restaurant and the other new Internet cafe. Berkeley has plenty of both, but imho there's always room for more. Both are at the intersection of Cedar and Shattuck.
Also walked through the North Berkeley farmers market. Every Thursday, "all year round, rain or shine."
Everyone should start a hyperlocal site. It'll give you fresh eyes for: 1. Blogging and 2. The place you live.
From time to time I get requests from developers who want to modify RSS so it can do something that wasn't forseen when the spec was frozen in 2002. Here's what I wrote to a developer who asked about that today (the specifics are xxx'd out).
Hi xxx -- I think it's a great idea to integrate xxx and RSS.
However.. I don't know enough about xxx to understand the substance of what you want to do.
You can add all the information you want to a feed by defining a namespace or by creating a new format called something other than RSS that shares its properties and adds anything you want, or for that matter, changes anything you want.
RSS 2.0 has been frozen since 2002, and that's absolute. It was necessary to do that to keep it from becoming a moving target. Lots of people had ideas for adapting RSS to their own projects, and that's supported through the two mechanisms I outlined above.
I admit to being confused by the event that Facebook has planned for
According to the Sydney Morning Herald: "At 2PM on Saturday, the social networking site will allow members to register their own user names to make it easier for others to find their pages."
What does it mean? Well, I'm sure I won't get dave or davew and there's a fair chance another davewiner will beat me to it. That means I'll have to go for one of my nicknames.
Why don't I have a chance at dave? Well, they're doing the usual Silicon Valley user generated content thing -- playing favorites. According to TechCrunch they're favoring journalists they "work with." Oy. Should we read that as "Journalists who write stories we like?" As if journalists need another reason for readers not to trust them.
But the thing that strikes me as weirdest of all is that after years of insisting that people only use their real names on Facebook, they've now set up a system where it will be virtually impossible for most people to do that, even if they want to.
If I cared more about Facebook, I'd have more to say about this.
I wish this period of the Internet would end, it's so exactly like AOL. I've seen this show before, I know how it ends. Only this time there won't be a Time-Warner to cash them out.
PS: Read Anil Dash's hilarious takedown of this mini-debacle.
Over the weekend I started a new site with my longtime friend and fellow Berkeleyite, Lance Knobel. The site arose out of a dozen conversations with friends and neighbors. "Does Berkeley have a site that's just about Berkeley?" The answer, always: "I wish there was one." Lately the conversations have been more urgent. Why don't we get off our butts and start one already.
So we did.
We all agree, we hope, that Berkeley is a great place, but it means different things to everyone who lives here. To some it's a great university town. To others it's a place to live, or a place to work. To others it's a cultural center. There's a huge freeway that passes through town, and a train line that goes to Chicago, New Orleans, New York, Seattle, Canada, probably everywhere else in the country. We have poverty and wealth. A new shopping district and an old one. Manufacturing, a winery, car dealers, biotech and computer firms. Some of the best public transit in the world. We have artists, scientists, great thinkers, journalists, many of the smartest people on the planet are our neighbors. We voted for Obama but we saw some McCain signs on front lawns. We have strong opinions, but we also value tolerance.
For me, Berkeley is a refuge -- it's a place to live because you have to live somewhere. I tried a lot of places. I spent 20-plus years in Silicon Valley, it was the right place for me when I was an ambitious young man determined to prove his worth. I liked living in Cambridge, the people were great, the intellectual life fantastic, but it was cold. I liked Seattle, but people worked so hard. I loved the beach in Florida, and the people were nice, but their politics were too different from mine.
I tried living on the road, but I needed a permanent place to sleep, write, and a consistent set of friends to hang with day in and day out. I could have had that in a variety of places but I chose Berkeley because it's beautiful, the politics are a good match (not in the cliche sense that rightwingers think) and the people who live here are intelligent, friendly, not pretentious and they don't work too hard, as a rule, so they have time to play.
Tom Hunt, a longtime Berkeley resident said it well. If you take out the university, Berkeley is a small place. The day he said it I ran into three people on the street who I knew on my daily walk. But over time I've given it thought and realize that it's not a small place, but it feels that way, it's approachable.
Now there are things not to like about Berkeley. And I suppose each of us has our own list. For me, it's the black hole that downtown is. I don't like going there. I don't understand why a great city like Berkeley doesn't have a thriving and bustling downtown. With the great public transit and the world-class university, located in the middle of one of the most dynamic metropolitan areas of the world, why isn't the downtown a place more people want to come to, not just from within Berkeley but from all around the Bay Area, the state, the country, the world?
To me, having lived here only three years, most of what I know about Berkeley is how much I don't know about Berkeley. But having a blank page to fill in is one of my favorite things. With my good buddy Lance, and hopefully with a lot of help from friends in and around this great place, I hope InBerkeley.com will become a place to learn and share and grow a greater Berkeley.
When we started bit.ly, about a year ago, I had a very strong idea of how we'd make money with it. Unfortunately bit.ly never got there. Now a couple of new shorteners are doing it, so I want to tell the story.
Like a lot of other developers I'm hooked on Amazon. It started with their storage system, S3. Now I use EC2, and hope to find an application for SimpleBase. I'm using many of the other smaller features of their cloud. It's great stuff.
However, there are a couple of components I'd like to see added to Amazon's cloud.
1. Internet-level notification. I'd really like them to offer the basic notification facilities of Twitter. See this piece that says that like it or not Twitter is becoming an essential part of the infrastructure. It's true. We need them to have competition, and it should come from Amazon, and many other places.
2. URL-shorteners. They're a fact of life. But I should have my own shortener at my own domain, so I control the future of the URLs. That way if the service I use should go down, I could switch to another. I also want to generate stats from the URLS.
Bit.ly was supposed to do #2. My plan for the developers went like this. When you have a question how to do something, do it the way Amazon does it. I want the API to be like theirs, the docs, and most important -- the pricing.
If Amazon did a URL-shortener, they would charge by the URL, and they would charge for each access. The prices would be very low, but they would add up. The same way they do it for S3 and EC2. Bit.ly was to be the URL-shortener that Amazon would have made.
Now today I learned of two URL-shorteners that are offering to host domains for you. One charges a flat rate of $49 per year and is available today. Another promises to do it for free, but it's available soon.
Watch this area closely, it's important.
There's something I'm not getting here.
A few weeks ago I tried to get a very simple script working with the Tumblr API. I kept getting a return value that said the API was down. I waited and waited for it to come back up before finally throwing my hands up in frustration. I tried everything I could think of. Turns out this was an error in the proxy server, the API was still up. Tried some more still couldn't get it to work.
Today I started fresh. Got the same result. Here's the URL that I'm POSTing to. (The password is xxx'd out.)
Here's the response I get.
I really would like to get past this, so if anyone has working code could you check to see what I'm doing wrong. TIA.
Update: Finally got it working. The params had to be in the body of the HTTP request, and I had the wrong content type on the request. Their messages could use improvement, they sent me down the wrong path, and a page with some examples would also have saved time and embarassment (I don't mind the embarassment, but I do mind losing all that time.)
TechCrunch used to do an annual list of Web 2.0 services they couldn't live without. The list wasn't about which apps are cool, but which ones are so useful that you build your online life around them. Products that becomes mainstays, apps you use all the time, tools that would cause panic if they went away.
Examples of Web 2.0 services I can't live without: Flickr, GMail, Twitter, Kayak, (though it pains me to say this) TechMeme, FriendFeed, Mininova, Amazon. I'm sure I'll think of others. But that's about Web 2.0, today I'm asking a different question.
Are there any Twitter apps that you couldn't live without? If so, what are they?
I'm fairly sure most of the apps will be clients, tools that read and post to Twitter from the desktop that in some way work around a limit of the Twitter web app. Tweetie is very popular. Seesmic, TweetDeck, the curiously named Destroy Twitter.
What are the others?
The latest Jay/Dave podcast, recorded last night at 7PM Pacific.
A little glimpse inside the news industry's mind: the recent recommendations of the American Press Institute. Charge for news, go after the aggregators, police fair use, look to consumers because the advertiser doesn't pay the bills anymore. A suicide pact, Dan Conover says.
The New York Times has a neighborhood blogging experiment, The Local. This week it extended an invitation to users: be the journalist. "Here is your first assignment: We're looking for someone to go to the 88th Precinct Community Council meeting next Wednesday, the 10th."
Three interlocking elements of a new system. The start of our kit for re-booting the news.
1. The pro-am invitation: help The Local cover Ft. Greene. Help us investigate.
2. Posted guidelines: how to cover a meeting for The Local; how to contribute to Chicago Now.
3. Assignment desk: an organized online list of everything we would cover if we had complete coverage of...
The launch and logic of inberkeley.com, a new local news blog that Dave and Lance Knobel have started. "It may not end up being the Berkeley blog. It may be the other thing that starts because people hate what we're doing."
The coral reef method of getting things done online. The Wikipedia stub. Their equivalent in news.
"Why wouldn't you want to be the newspaper of record...?" (Dave) vs. The Era of Omniscience is Over (Jay).
Sources of inspiration (Jay's turn this week.) Andrew Leonard's 1999 article in Salon, Open-source journalism. "This vision is alive."
Just tweeted: "Microsoft's problem, they employ billions of dollars worth of engineers who produce stuff no one wants."
I pointed to this article.
Short version of this post: Microsoft -- Let the netbook guys put whatever they want to in the box, and sell them XP Home for a reasonable price and stop trying to tell us we have to use Vista because people don't want to.
Netbooks are great Windows machines. I remember seeing a $600 pricetag on an Asus last year and thinking "Geez that's cheap!" so I bought one. Now it seems expensive. Same computer now is $280. That's even cheaper. So cool. And it runs Windows XP Home so I can run my software on it. Now I'm totally uninterested in buying an iPhone-like laptop, which Apple almost surely will want to sell me.
You'd think that would be great news for Microsoft! You'd think they'd be running ads on TV saying "Holy Shit People Like Our Stuff Now Man That's So Fucking Cool."
But you'd be wrong.
Because. Because. Well. You tell me why they're not super excited about this. Steve? Ray?
As a user, I'm happy as can be. I love this new stuff. And I'll tell you what. It's found money for them, whatever they get, because I wasn't ever going to buy a Microsoft product. I'm amazed that I like XP. But only because it runs on these coool new netbook computers.
And the netbook market is incredibly competitive. They keep dropping the prices and they want to add features, but Microsoft won't let them. If they add more features, they say, they have to put Vista on the computer. People don't want Vista. And Microsoft must be worried they don't want Windows 7 either.
That's their problem, not mine. Their job is to create software people want.
I recorded a brief podcast about this, but if you've read this post you don't need to listen to it. You've already heard what I have to say.
XP is cool. Sell it and be proud. Create products people want, and all is good. Create products people don't want, go back to the drawing board or find another line of work.
Just tweeted: "mifi is a battery operated wifi router that fits in your pocket and connects to the net via cellular."
Since I already had a pretty good service plan from Sprint, and switching would be quite expensive, I just got the Sprint version and so far it works really well.
Here's a picture of the Mifi router next to a DVD to give you an idea of how small it is. It really fits in your pocket and runs for hours on battery. Not sure exactly how many hours.
And here it's shown with the Cradlepoint router and EVDO modem it replaced. A fragile bit of tech that worked well, but the new version is much cooler.
Ultimately the Mifi router will be replaced by software running on my iPhone, when Apple and AT&T decide to let us do that. It's probably a question of how much traffic the AT&T cell network can bear. In the meantime the Sprint system seems pretty good.
Just realized something in a new way.
I've been posting links to new blog posts on Twitter since I started using it two years ago. It's just a natural thing, another step in the publishing process. You can see very clearly where it fits in by looking at the button-bar in my editing window.
Here's the process.
Step 1. Write the initial draft. Organize. Edit.
Step 2. Save. This publishes the piece to scripting.com, both on the home page, and on its own story page. I repeat this step until I'm ready to have the story appear in the RSS feed. (I don't mind if readers see the interim versions, I imagine it's somewhat interesting, if not it doesn't seem to do much harm.)
Step 3. Build RSS. I know that many RSS clients will only read an item once, so I wait to rebuild the RSS that includes the new piece until it's pretty much finished. I might still add some pictures, or links or tweak up some wording, but by the time it goes out in the feed, it's not likely to change much.
Step 4. Twit-It posts the link to Twitter. I get to edit the link text before it goes out, but it does the work of creating a short URL and smashing it together with the headline before presenting it to me in a dialog.
This last step is relatively new, but its import is starting to settle in. In a real way a story isn't published until I've pushed it through Twitter. I expect over time, as more systems hook into Twitter, it will come to mean more. Of course I will, as long as Twitter has a 140-character limit, publish everything on the web and in RSS. This article so far has 2291 characters, or 16 tweets.
Another way of saying the same thing is that Twitter has become the newspaper of record. In a few years what's left of the news industry will call Twitter a parasite and demand royalties. Too bad they don't see this coming, and create an even better news system built around the principles of Twitter and instead of asking for alms they'd get a piece of the PE.
Sidebar to the Twitter bizdev people: Wish I had upside in Twitter, so I could be motivated to make these things work in your company's product. But I'm a greedy capitalist just like you, and with my "stock" in Twitter diminishing in value every day (through dilution), I have to look elsewhere for my upside. You might think of this as a challenge or a puzzle, figure out how to incentivize your users to make you even richer.
Tomorrow is another big Apple announcement day, and most people expect there to be a new iPhone. Maybe there will be more. But one thing that's likely to come is more of Apple's positioning relative to netbooks. And more sniffing from people who love Macs about how inadequate the current crop are.
I'm typing this blog post on a big iMac running Leopard. I like my Macs, but I also own three netbooks. One I bought early, a 901, it cost $600, sells for $280 now. I took it to the DNC in Denver, and it made a huge difference, blogging in tight spaces and often far away from a power outlet. Then I have a $450 H-series that runs Linux, and the workhorse another H-series that I bought for $350. See what's going on? I'm getting more for less.
People who don't think these are great computers must not have a sense of history. My first personal computer, purchased in 1979, cost $10,000, had two small floppy drives, 64K of memory and ran a very bare-bones OS. It weighed as much as a dorm room refrigerator, and generated as much heat as a dorm room hot plate. Yet it was a marvel -- a computer of my own, in my living room. Amazing.
At the time I would have told you that someday we'd have computers like the EeePC that weigh as much as a small textbook and run on batteries for 6 hours. I might have guessed they'd be as cheap as they are, but it's one thing to predict they'd be here someday, and another to hold one in your hand, to take it with you everywhere. These are machines that can be stamped out in the millions, they're Everyman computers. Yes, Macs are great, but they're great in different ways. People who sniff at the netbooks are missing something important.
FTP will be around for a long time, just as HTML and SMTP will.
I still deploy apps today that use FTP to transmit files. For example, I have a flow of AFP wire photos that streams through an app running in the OPML Editor. They send the photos via FTP. A huge number of bits are transmitted this way, every day. Billions of them, just to my server.
In turn, I transmit those bits to a server on wordpress.com, using XML-RPC. Another protocol, which, like FTP, will be around for many years to come.
I call a server somewhere in the world (probably in US) running software written by a man from Turkey, that turns these pictures into thumbnails. To do that transaction we use HTTP.
I have a River of News aggregator running on a server in my house. It reads feeds in a variety of formats and presents the results in HTML over HTTP.
I have an app running on my living room computer that reads the feeds of some of my friends, and when it detects a new item in one of them it posts a message to a Twitter feed. It uses RSS on the input side, and calls the Twitter API using HTTP in an XML-based format of their own invention which is sure to become a standard because of the popularity of Twitter. I read these results on twitter.com, and over 1000 people follow this feed, accessing it from a wide variety of Twitter clients, spread over the world.
I could go on and on. I've been programming similar apps for many years, they are deployed on lots of machines, and they use all manner of formats and protocols. None of them are going to "die." You can no more get rid of them than you can all the microbes that inhabit the human body. If you tried, you would die long before they did.
I once told a live oak tree on my property in Woodside, a very beautiful stately tree that had been there for decades, that I owned it. The tree laughed. "I've been here for decades, and I will be here decades after you're gone." That was 15 years ago. The tree was right. I left Woodside in 2003.
I moved to the Internet in 1994 from heavily controlled platforms, because it was and is the platform without a platform vendor. Every once in a while a company comes along that looks like, to some, that it will suck up all the energy of the Internet behind their domain. Don't bet on it. More likely, they will advance the art in some useful way, and then distribute their users the way a flower disburses seed.
If you're looking for something that will die, look at the companies, not the formats and protocols. They're like cockroaches and microbes, and the trees -- they'll be around long after the companies are gone. They get the last laugh.
I don't know to what extent Twitter archives my posts. For example, here's a blog post from January of this year. It links to a tweet, which is still there. Not sure if it keeps around older stuff, or how I would browse them if I wanted to see what I had written. The search command in Twitter stops at a certain point, exactly where that is -- I don't think anyone knows.
Uncertainty about what's backed up is a sure sign of a problem with backups.
Because I want a record of whatever I post to Twitter, I wrote an app that archives all my posts and those of people I follow. It's a very easy bit of code to write, since Twitter has an API call that returns all the recent tweets of all the people I follow, every client has to make this call regularly, so it has to be efficient, on both ends; and it is. I have shared the code, anyone can download it for free. It runs in the OPML Editor. But I think more developers should add this to their Twitter clients as a service to their users, as a competitive advantage, and as a way of making the work we do safer.
Right now we're all running without a safety net. Or more accurately, only Twitter knows how much of a safety net we have. And as we saw in the financial meltdown, it's not wise to assume that people we depend on to understand how complex systems work actually understand how they work.
In software, it's always a good idea to back up your work. And the people who make the popular Twitter clients could do a lot to help us there.
1. Some environments allow apps to write to local disks, and others don't. I don't know if Air, the platform many of the Twitter clients run on, allows this. If so, then I recommend that the clients simply maintain a calendar-structured folder of XML files containing each days' tweets, one file for each user. If not, then the backup has to be maintained in the cloud.
2. The size of these files is negligable in the age of MP3 and AVI. Text files are tiny and disks are relatively huge. Size isn't an issue.
3. Neither is performance. The file systems of today's computers are incredibly good at saving small text files.
4. It might add a little complexity to the Prefs user interface. At least it would require a panel that allows the user to choose a folder, and to enable or disable the feature. I would have it enabled by default.
5. You might want to allow the user to save his or her backup in Amazon S3 or to use FTP to upload to another server. Again, the overhead is negligable. I have the software running on my desktop system in the background. It's just an ordinary iMac. I don't notice any delays. Honestly.
6. What format to use? The simplest choice would be to use the XML-based format that Twitter itself uses. Other choices include RSS, Atom, OPML, or something of your own invention. I think RSS is the most rational choice, but I used OPML. I'm beginning to think that was a mistake, though I had good reasons for that choice at the time.
7. I also dereference short URLs and store both the long and short version. Wouldn't want to go to all the trouble of backing up the tweets only to find out the URLs broke because tinyurl (or whatever) went away.
8. The most basic reason to do this is backup, and that was the original motivation in my suggestion, in the summer of 2008. I suggested to the client vendors I could reach that they support RSS-based backup. That way, when Twitter went down -- as it was doing regularly then -- their users would not go down. But then Twitter started becoming more reliable so the urgency of this decreased
9. However, storing user backups first on the desktop, then in the cloud, those are the first steps towards an open, low-tech, simple form of federation that doesn't depend on a central node. If for no other reason, we as a community, should start down that road, asap. Murphy's Law says that at some point we will wish we had.
10. I'm sure there are other considerations, please post comments if you think of them, and I'll add to this list as I think of them.
Just read a wonderful post by Jackie Danicki, about how she chokes up in the presence of older people. Me too. I have tears streaming down my face as I read her piece, as I write this one, as I experience the memories it evokes for me.
A few weeks ago I was walking in my neighborhood in Berkeley, a hot day, and I came up behind an old man, all bundled up, walking slowly. The sidewalk was narrow, so I walked around him, and as I passed I said, gently "Excuse me." He jumped, startled and said reflexively -- "I'm sorry." In an instant I felt protective and sorry I hadn't found some other way to handle this. I did the best I could, and said something like Oh no, smiled and continued my pace.
He was skinny and fragile, exposed and vulnerable, and reminded me of my father, who will turn 80 this month. I felt protective for this man as I would if he were my own family. And was instantly reminded of something my father says often, that's worth remembering: Growing old isn't for sissies.
No doubt. But we, whose bodies still work, more or less, as they were designed to, can easily overlook that in every old body is a person who remembers well what it was like to be young. We are at a disadvantage, we don't know their experience, we get little inklings of it when we get sick, but we expect we'll get better. At some point, you no longer have that to depend on, and the quality of life must change.
Why do old people reach my heart this way? It could be their courage, or the inner strength it takes to compensate for the weak body. I don't know why they get to me this way. But they do.
Twenty years ago today -- the Tiananmen Square Uprising in China. There were no blogs that day, but things have certainly changed.
Now so many blogs and social media sites are blocked in China.
Here's a tool that helps you determine if your site is blocked.
If you discover anything interesting, feel free to report in a comment here.
When the Guardian ran a piece over the weekend on the Suggested User List, my interest, and others' re-stoked. Then I got an idea from Sarah Delman, pondered it, wrote some code, published the results. Now we know more about the source of the SUL. More back and forth, some of it heated, then it comes together for me -- we lack leadership. I can't provide it, I'm not trying to. I'm trying to be an provocateur, a role I often cast myself into.
As I've said before, I believe Twitter is a chapter in a story that's been playing out for a long time. It's both the best news system and the worst. No pictures, no video, limited metadata, and it has an increasingly confining 140-character limit. But it connects people like no system before has. It's both the backroom for journalism and the delivery mechanism. A lot of power there. But imho it's not the last word.
The remaining news organizations will move onto twitter-like systems over the next few years. The news system of the future is electronic and real-time. The stakes are huge now. If I were in their shoes, I'd be thinking very hard about how I want these systems to evolve as environments for journalism. I'd stop worrying about squeezing Google for a handout and start thinking about how to grab some of the PE-ratio for myself.
But the lions of the news industry lack imagination and chutzpah. Where are the strategists, the bizdev people of the news industry? They're plotting paywalls, when they should be creating and linking new conduits with graphics, sound and movies.
The business model? The same one that served Google in its early years. People are so excited about what you're doing that they pour cash all over you.
The SUL is a small piece of the big picture, but it's an important one. For all the reasons I've said. No need to repeat it. Now I'm going to let everyone else worry about this for a while.
Early-on in the life of Twitter I tried to figure out how it works behind the user interface, and was told I couldn't, that it was very complex, and I shouldn't try. This bothered me then, and it bothers me even more now. After all, I'm an experienced software engineer, with a number of products under my belt, some quite complex. If I can't understand it, who can? And if they say I can't, do they?
We know the networks we're building on Twitter are very fragile things, but they're more fragile than most users have been willing to comprehend, because we can't find out how it works. We have to trust that the company knows, and that they will continue to operate it as they have in the past. However we don't have any right to that service, and if they restricted it or took it away we would have no recourse.
Of course it would be a disaster for them as a company, but such disasters have happened before and they certainly will happen again. And what do we actually know about the people who run Twitter? Not all that much. They don't tweet a lot, and when they do, they tend to be short messages, and often cryptic.
So while Twitter is useful and fascinating, even intriguing, is it simple?
It it not simple.
Yesterday I posted the results of a study of who the people who work at Twitter follow on Twitter. This is important because it is the source of their Suggested Users List, which in turn determines who new users will follow. People and organizations who are placed on this list see a substantial surge in followers, often in the hundreds of thousands. How much this is worth was the subject of a recent Guardian story.
It was easy for me to adapt the crawler to also generate a table for the people of the NY Times. I don't know exactly why this is interesting, but it seems people are curious to know if they make it onto the radar of the people at the Times.
If there's continuing interest I'll refresh both these tables from time to time.
Jay asked, in a recent RTN podcast, why simplicity is so important. I don't think I had ever been asked this question, and it had been a long time since I considered any other way, so I didn't have an answer prepared. I fumbled.
It has bothered me ever since, but not as a front-burner thing -- I still accept the value of simplicity as a given. As I build layers of software, the simpler I make each layer, the higher I can build. If you don't design to hide complexity behind interfaces, you get overwhelmed by the complexity sooner, and your project can't do as much. Early in my career I often scrapped multiple levels and went down to the roots and rebuilt. Once I understood what the higher levels looked like, it exposed deficiencies in the lower levels. Reworking the lower levels allowed me to build higher. (It happens in other technologies too. For example, the design of office buildings changed radically once elevators had been invented.)
The ultimate interface is where you pass off to another user, and a whole new human brain comes into it. As a designer, I try to present a conceptual model to the user that's as simple and predictable as possible. This allows them to focus on their work without mine getting in their way. In the movies they call this "suspension of disbelief." The user gets so wrapped up in the story they forget it's a movie. It starts feeling like life. Even so, I like to create things that can be explored beneath the surface. If you want to "lift the hood" I know you'll see a puzzle, but I hope it's approachable.
As I discussed in this week's RTN, I'm working my way through Season 1 of James Burke's Connections series. It aired on PBS in 1978. Yesterday I watched the final episode in the series. It was even more fascinating than the previous nine. Its message is so important that it's worth taking time away from the innovations of our day and to talk about the process of innovation itself, and why it's dangerous. Yes, that's right -- dangerous. I've known this all along, and I've tried to publish warnings, but Burke does it with a breadth I've never approached.
There was a time, not too long ago, when anyone who wanted to could understand any invention that our lives depended on. Look around and look past the computer monitors and hard drives. Look at a door, for example, a hinge, a door knob, the door itself. All of this technology is transparently simple, and at one time, not that long ago, it was leading edge. I suppose paint might be hard to understand, but if you really wanted to understand how it was made, you could. It might take some time, and you would have to talk to an expert or read a book, but it wouldn't take much time to understand it.
But why bother understanding it? Burke asks. Isn't it enough to just use it and enjoy it and let it sustain your life? Yes, it is, until you have to make a decision about it. Or until it fails.
If you had asked me a year ago if we would live to see the financial system melt down, I would say yes, it's the one thing I'm most scared of. Like many others, I listened to experts talk about the aftermath of the collapse, reporters from famous newspapers, economists, radio show hosts. One thing I heard over and over was that no one actually understood how the system worked. Yet we had built our economic system, our lives, on it. Let me repeat that so it sinks in. No one understood how it worked.
It's only in its unraveling that we're learning how our economy works.
But that's far from the only system that no one understands.
Think about all the technology you depend on to live that you don't understand, and ask yourself if anyone understands it.
What if you went to the ATM and all your money was gone? What if everyone went to the ATM and all their money was gone? Think it can't happen? It did happen in post-Katrina New Orleans.
That's just one vector. Think about food supply, sewage treatment, water, transportation, education, law enforcement, health care, defense. We're making decisions about all these things every day whether we know it or not, and like the financial system, some people understand parts of these systems, but it's likely that no one understands the whole thing.
At the end of the series Burke lays out a set of choices we faced in the late 70s. We could continue along as we have been through all of history, or we could change course. It's now 30 years later and we didn't change course, and it seems we're not likely to. But if you're an innovator, you have a choice. You can create new things with an emphasis on being understandable to the people who use them, not just at the surface level, but under the hood too. It often takes more work to make it hard to understand. But even if it takes more work, it's worth doing it simply, helping create the sense that technology is understandable, because imho that's the key to moving forward in a way that might just work.
Update: Example of perfect simplicity. View Source in web browsers.
Here's the story...
We've heard that the SUL is like the list of books that employees of a book store recommend to customers. So this raises the question -- which "books" do the employees of Twitter read -- i.e. who do they follow, and how does that correlate to the list of Twitter users on the SUL?
I loved the idea -- it's my favorite kind of investigative reporting, because it involves programming.
Thanks to the Twitter API, it was actually possible to write a crawler that:
1. Generates a ranked list of who they follow and
2. Highlights the ones who are on the SUL.
I've written and run that crawler and the lengthy table on this page provides the result:
You can judge for yourself who influences the people of Twitter, and how that in turn influences the SUL.
There's a wealth of other information in the table, I've already spent a couple of hours pondering it, but it's time to share it with everyone else, and see what you all come up with.
Also thanks to Tom Reynolds for getting me the contents of the SUL late last night in response to a query I posted on Twitter. I didn't know where to find it, and it turns out it's hiding in plain sight on a page on twitter.com.
Also note that people come on and off the SUL. So there are people with high follow counts who are not on the list, who were at one time.
Update #1: There are four accounts on the SUL that have 0 followers at Twitter: Ali_Nejad, AstrobiologyNAI, LIVESTRONGCEO, PEOPLEPets.
Update #2: Who do the people of the NY Times follow?
Last night's podcast is up this morning, bright and early!
A theme from last week continues this week: Bug catching as a key practice in a re-booted system of news. Jay unfolds an example from this week: the AP's coverage of the Twitter TV show.
The TechGuardian asks How much is it worth to be one of Twitter's suggested users?
Dave discusses BitTorrent and why he put RTN 1-10 on it.
CheckBox News, Dave's mock-up of a re-booted user interface for television news where you can uncheck the streams you don't want and check the ones you do, and program your TV set that way.
For sources of inspiration (it's his turn) Dave returned to three: James Burke's public television series Connections, about the history of science and technology (inventions are usually the result of a synthesis of things created by earlier inventors); the Cluetrain Manifesto (ten years old and great); and VisiCalc, the first spreadsheet program for personal computers-- the demo for which was
Dave argues that the great news organizations should be the operators and originators of systems like Twitter. It's not too late, but soon it will be, he warns.
We close with a short reading from Barbara Ehrenreich's commencement speech to the UC Berkeley School of Journalism. "A recession won't stop us. A dying industry won't stop us. Even poverty won't stop us because we are all on a mission here."
Dave Winer, 54, pioneered the development of weblogs, syndication (RSS), podcasting, outlining, and web content management software; former contributing editor at Wired Magazine, research fellow at Harvard Law School, entrepreneur, and investor in web media companies. A native New Yorker, he received a Master's in Computer Science from the University of Wisconsin, a Bachelor's in Mathematics from Tulane University and currently lives in Berkeley, California. "The protoblogger." - NY Times.
"The father of modern-day content distribution." - PC World.
One of BusinessWeek's 25 Most Influential People on the Web. "Helped popularize blogging, podcasting and RSS." - Time.
"The father of blogging and RSS." - BBC.
"RSS was born in 1997 out of the confluence of Dave Winer's 'Really Simple Syndication' technology, used to push out blog updates, and Netscape's 'Rich Site Summary', which allowed users to create custom Netscape home pages with regularly updated data flows." - Tim O'Reilly.
Dave Winer, 54, pioneered the development of weblogs, syndication (RSS), podcasting, outlining, and web content management software; former contributing editor at Wired Magazine, research fellow at Harvard Law School, entrepreneur, and investor in web media companies. A native New Yorker, he received a Master's in Computer Science from the University of Wisconsin, a Bachelor's in Mathematics from Tulane University and currently lives in Berkeley, California.
"The protoblogger." - NY Times.
"The father of modern-day content distribution." - PC World.
One of BusinessWeek's 25 Most Influential People on the Web.
"Helped popularize blogging, podcasting and RSS." - Time.
"The father of blogging and RSS." - BBC.
My most recent trivia on Twitter.
© Copyright 1997-2009 Dave Winer.
Previous / Next