I'm pretty sure the Times implementation of WinerLinks doesn't have the all-important feature of allowing the author to insert, delete or move paragraphs without breaking external links.

This is just a guess, but I am inferring it based on: 1. The paragraphs have numbers like p1, p2, p3 etc. There's no evidence of reorganization, which would look like this: p12, p3, p4, p9. In other words, there would be no relation between the visual order of paragraphs and the permalinks. 2. They use WordPress, and its model can't support invariant paragraph IDs, because you're editing HTML source, not structures of paragraphs.

My content system can do it because I use an outliner to edit my blog posts. Screen shot.

Each paragraph is a separate entity. It gets a unique ID when it's created, when I press the Return key while typing. That ID does not depend on where it is in the outline. I can reorganize as much as I want and the paragraph IDs won't change. The permalinks stay the same no matter where the paragraph has been moved to.

As a demo, here's a list that has been reorganized:

1. Pepsi.

2. Ginger ale.

8. Orange soda.

4. Dr Pepper.

5. Coca Cola.

3. Fresca.

6. 7Up.

If you mouse-over the permalinks above you'll see that they're out of order. That's why this works.

There are outliners that work in a browser, so you don't need to use an external tool, it could be embedded in your browser-based blogging environment. But to make this work, WordPress itself will have to advance, I don't think you can do this in plug-ins (could easily be wrong). No matter what they will have to store two bits of text for every post: 1. The "source code" which contains the paragraph-level metadata. 2. The HTML rendering. I hope the source code is OPML, and they permit external editors like mine to be used to edit the posts. I asked Matt for a feature that would allow a bit of XML text to be stored in WordPress alongside the HTML. I don't think he understood the application at the time. This is the application (and there will be others as well, separating source from rendering is basically a good way to gain leverage).

BTW, here's the source code for this post. I store it in my CMS alongside the HTML rendering. If you change the extension from .html to .opml you'll get the source.

 11/30/2010; 11:13:43 AM.   .

I find the iPod software too cumbersome for listening to podcasts. I want something lighter with features that make sense for 1-hour-long programs, as opposed to 4-minute-long songs.

Nowadays the functionality I want could be done in software, either for Android or the iPhone. I carry both all the time. But I'd still prefer to carry a medium-size device just for playing podcasts.

In case someone is interested in making this product, here's how it would work.

1. The device plugs in through the USB port as a hard drive. I copy the MP3s into a folder which can then be played through the UI of the device. If this is iPhone software, I create a playlist in iTunes that contains all my podcasts.

2. At any time there are a bunch of programs in the podcast queue.

3. I can hit the pause button on a podcast, and it remembers the position. I can then skip to the next one, or choose the next one from a list.

4. When I come back around to the one I just paused the playback picks up where I left off.

5. When I make it to the end of a show, it goes in the trash. I can empty the trash later when I'm putting more programs on the device, or leave it there in case I want to listen to it again (I don't think I've actually done this even once, in six years of podcast listening).

6. If it's a standalone device, I don't want to pay more than $30. Apple's players are way too expensive, so are Sony's.

Ideally I'd like this to be a device, not an app, because even though both platforms claim to do multi-tasking, they behave unpredictably. I shot a movie the other day while listening to a podcast, and the podcast stopped, and when I resumed I had to start over from the beginning. On the original iPod I knew how to fast-forward to a new location, but they changed the UI, and when I'm out and about there's nothing dumber than having to stop on the side of the street for five minutes to get the software to behave itself. I had one moment where the iPod told me it had to rebuild something and I should be patient. I was standing in the middle of Houston Street with traffic all around me, and the programmers back in Cupertino programming want me to be patient. Hello.

I used a Sony Walkman for a while and loved it. It does almost all of the items on my list, but not the crucial one -- #4 -- picking up where I left off on multiple podcasts. This is a big feature. I can read three books at the same time, there's nothing so weird about wanting to "read" three podcasts simultaneously. The Kindle does this admirably. Hey maybe Amazon should make an audio book reader? Thinking out loud.

BTW, the Walkman that I love crapped out on me. It's stuck in a loop rebuilding its library.

I know people are going to tell me about products that kind of do some of what I want, or do something completely different. I think there's a $30 sweet spot out there that everyone is missing. Music and podcasts are very different forms and require different approaches.

 11/30/2010; 9:59:13 AM.   .

I was tempted to put a picture of Jimmy Wales in the right margin of the previous post. But I decided on a bigger idea. Why should I be the only one? One day, maybe next week, we should all put pictures of Jimmy Wales on our blogs. WIth whatever slogan you like, it doesn't matter -- just you have to have a picture of Jimbo there so everyone knows... uhh so everyone knows.. what he looks like! Yeah that's it.

 11/29/2010; 10:32:06 PM.   .

We did RSS before the Times did, but when the Times did it, everyone followed. That's good leadership, as ex-Timester Jonathan Glick likes to say (he now runs a stealthish NYC startup). Same thing is happening with paragraph-level permalinks.

The concept of a permalink on a paragraph existed before I implemented them here on Scripting News, but it got enough exposure here to get other people interested, and they've been appearing in a smattering of places on the web, notably in Jay Rosen's PressThink blog.

Then Daniel Bachhuber did a WordPress plug-in, and all of a sudden there are a few dozen sites doing it. He called it WinerLinks as an ode to me and my blog, and I won't discourage it. It's nice to get recognition!

Earlier today Zach Seward tweeted that the Times had done paragraph-level permalinks too. It's a different implementation, but the idea is similar. Instead of the link appearing at the end of the paragraph (seems more natural to me) the Times links appear at the beginning. And instead of appearing by default, you have to click the Shift key twice, with your pinky touching the tip of your nose. Actually it seems to work even if you don't do the pinky thing, but I like to do it anyway.

When you do the magic incantation, nice paragraph symbols appear, and if you mouse over them you get a tool tip and when you click on it, the permalink to the paragraph is placed in the address bar of your browser, ready for you to copy and paste it or shorten and tweet it.

It's good because you can point to the specific bit you want. No more making the inquisitive reader fumble around to point to the bit you're referring to.

Nice!!

I wonder how their editorial tools work. If the author adds a paragraph does that break all the links out there?

Also, is it on all their sites or just some?

Maybe we should have a meetup to discuss it. It's all happening in NYC (a big neener neener to my friends on the west coast) so we could meet at NYU or the Times or at the Starbucks at Astor Place.

 11/29/2010; 10:17:52 PM.   .

I had this great idea a few months back, create a top-level folder in my Dropbox hierarchy where each sub-folder would be the name of a host, like listings.opml.org and daveriver.scripting.com. Then one of my servers, on the same Dropbox net, would make the folder the root of the Apache htdocs directory. A script would run once a minute and look for new folders and update the vhosts.conf file. Then any file I dropped in any of the folders, once it synched would be available on the web.

I was using this system, but the server would flatline a lot, and so would many of the other machines on my Dropbox network. I didn't know what was causing the flatlining until I decided to split things up and make one server into three, so I could isolate which components were interfering. Lo and behold, the machine that flatlined perfectly was the one serving up these fancy folders. So I uninstalled Dropbox on that machine, and started replacing all the cute bits with FTP calls. Yeah it's not so much fun as the magic of Dropbox, but here's the A-B comparison:

Bottom-line: Dropbox is good for sharing files with your colleagues and across your devices, but it can't, at this time, be used to manage a static server running Apache. Too bad, cause it was such a neat idea.

 11/29/2010; 9:23:16 PM.   .

I've lost count of the number of days I've spent working on this issue. It's becoming clear as I rearchitect my web presence that building on Dropbox was probably not the best idea. There were elements of greatness, stuff to be excited about in theory, but in practice, maybe not so much.

First. I think I'm pushing it in ways that no one else is. When I get to the bottom of what's causing the performance issues here, it's going to be something I'm doing that you're not doing.

Second. I'm kind of sure I know what the difference is going to be...

I have Apache configured on one of my servers such that its root directory is a top-level directory in the Dropbox hierarchy that all my servers, desktops, iPads, iPhones, Droids and laptops share. I think it's the connection with Apache that's driving everything crazy.

I can't know because Dropbox is a black box that I can't see inside. There's no rule that says you can't give a folder to Apache to serve from. It's pretty neat when you do. It means that any node on the network can cause something to be published by writing a file to a folder. There was a time when this performed pretty well. Or so I seem to remember. It's like the Fog of War, keeping these servers running. The last week I've been desperately and slowly trying to lift the fog and get scientific about it. But the science is hindered by a lack of understanding of how Dropbox works.

Luckily, I'm getting special service. The CTO of the company is in the loop. So maybe we can figure this out. Maybe there's something I can tell Apache not to do to keep Dropbox from thinking it's always got to be doing something that not only flatlines the computer it's running on, but sends so much noise to the other nodes that they happily consume all the cycles on the machines they're running on.

That's my status report for near-mid-day on Sunday. Stay tuned for updates.

 11/28/2010; 11:08:20 AM.   .

After posting earlier about performance problems with Dropbox, I got an email from Arash Ferdowsi, the Founder/CTO of the company, saying there is a new version in the final stages of testing. He suggested it might help with some of the problems.

He wrote: "I recommend trying the latest release candidate of dropbox. it's been incubating for the last 9 months and we're planning to release it in the next 2-3 weeks. it's much better at handling large file sets and uses much less cpu/memory. additionally, it's a lot more aggressive about pruning the dropbox cache (the build you're on only clears it when the dropbox process reboots while the new build clears it every 3 days)."

http://forums.dropbox.com/topic.php?id=27419&replies=146

I asked if I could install the software on just some of the machines here, and he said yes, with a caveat. "0.8 now supports syncing of the executable bit and some other xattr data. if you're running a mix of 0.7 and 0.8 clients, this metadata won't sync properly."

I'm installing the new release on the server that's suffering the most, and on one or two other machines. I will of course report the results.

Update: I've tried installing the new software on two machines and both installs have frozen at this point. I'm going to leave the installer running on this machine for a while and see what happens.

Update: A couple of hours later, I was able to get it to install on one of the machines by first uninstalling then re-installing the app. This meant relinking the account, a slow process, but there was an upside -- I stumbled across the new Selective Synch function. This is going to make a big difference. It allows me to push content out through Dropbox that I might only need on one or two other machines, without adding overhead for the others. Nice!

Update: Why does Dropbox download so slowly? 5K/sec. I have no bandwidth constraints in the prefs. See the screen shot.

 11/27/2010; 4:12:11 PM.   .

I've spent the last week wrestling with my servers, dividing up what they do, factoring things so that maybe I can get the performance where it needs to be. It's one of the reasons I was exploring the possibility of using S3 to store my static sites (I gave up, couldn't make it work).

So now things are running in the new configuration, or I should say -- limping along in the new configuration. There is always lots of breakage when I do this kind of surgery. While doing it I wish I could integrate DNS, generating Apache config files and serving into one master outline, where I make a change and the new configuration automatically percolates where it needs to. That, and it would be great to have one server with infinite CPU, disk and net bandwidth, so I'd never have to logically split a server across multiple machines.

Maybe I'll get all this in time for next Christmas!

Anyway...

I think the mistake I made in the last half of 2010 is that I got too Dropbox-happy.

I built too many systems on it, without understanding that it could be brought to its knees, begging for less to do, all the while causing every other application on the server to be starved for CPU cycles.

When I saw how much work Dropbox was doing just to keep current in a relatively stable folder, I realized I needed to lower my dependence on it. It really wasn't doing that much for me, but it was taking up a very large portion of the CPU cycles on every one of my machines. (Only a few small text files were changing every few minutes, the really big stuff came once a night when the servers backed up).

Also it uses a huge amount of disk space. The cache is enormous. I figure it uses at least 1.5 times the size of the folder.

Now, Dropbox is a miracle product. Last year I called it an "idea factory." It gets you thinking about great things could be with a simpler architecture, and closer to the ideal of one big server without boundaries. It's a very easy-to-understand queueing system, and when you add XML or JSON to the mix, you get the magic that comes at the intersection of simplicity and power. But that's the virtuality of it, the reality is that it uses a lot of resources, so you have to use it carefully.

I'm interested in finding out what other people are learning about Dropbox.

 11/27/2010; 4:25:24 AM.   .

Interesting piece in the Telegraph quoting new Twitter CEO Dick Costolo saying that Twitter doesn't have a long-term vision, but it's his job to come up with one, and we should expect to hear something there shortly.

I think what he did here was admirable because it's so obviously true. It invites people like yours truly to pontificate publicly on what we think their vision should be. Maybe someone somewhere has an idea, and cares enough to share it with the rest of us. Maybe there will be something worth listening to in all the noise?

So what do I think Twitter's vision should be?

It seems, no matter how you slice it, they have the same two choices that all tech companies do:

1. An ad-based business, where the users get access for free, and they sell the attention of and aggregate data about the users.

2. Let the users run their own ads, if they want, and charge them to use the system. A freemium model, where the base product is free, but advanced features that help you sell stuff through Twitter, cost money. (Users might even come up with new ways of advertising, given a chance to be creative.)

In the first model, it's inevitable that not only will Twitter force developers off their platform (as they already are doing) but the same will eventualy happen for news organizations that use Twitter as their notification system. As with developers there's a naivete among news orgs that Twitter is somehow a public utility. This is not how Twitter views itself, nor is it in reality what it is (it's a privately held corporation).

In the first model they basically compete with everyone who uses their service as part of their business. In the second model, they compete with almost no one, and simply make money off the growth of the activity.

I think they've gotten so large so quickly that #2 is the only viable approach. It's the same advice I gave Microsoft before they hit the wall with the Internet, and the same advice I gave Google as they were hitting the Facebook wall. Stop hiring outside the basic area you already occupy, and shrink that if possible, and get ready to take 50 percent of all the growth, and share the rest with others. It clearly would have worked better for Microsoft, and not so clearly (yet) for Google. But it's a predictable cycle that Twitter is advancing through very quickly because they're growing so fast, and because unlike Microsoft and Google, they never had a golden age when they were technically best at what they did. They've always been ragtag, and I'm pretty sure they always will. It's in their DNA. And besides, companies as they grow never get more competent, they always get less.

But Twitter has been advancing fast into model #1 -- advertising and aggregation of user data. The purpose of advertising is to turn cash into attention. Companies will be diseconomic about this, for a while -- but long-term they can't afford to. A company that bases its business on buying attention must get more value back than the money they spend.

People seek out commercial information on the Internet all the time. They look for it. Think about your own habits. Last time you bought something, how much time did you spend finding out about it on the net. I won't even go out to eat at a restaurant that looks good from the street, that was recommended by someone I like, without looking it up on Yelp and seeing what other people thought. It's amazing to me how unhelpful most vendors' sites are in my buying decision.

Ads almost never influence my buying decision.

Companies that spend a lot on advertising would be well-advised to instead spend the money on making their websites more helpful for people wanting to buy their products. Long-term that works, imho. Unfortunately for Twitter, their service isn't very good for providing info about products people may want to buy. I can't even keep track of what cities my friends are in with Twitter, the pulse is so rapid and the flow is so flat (which is good thing for news, btw).

Further, more and more product development is going to shift to the net. That manufacturing companies are going to become fullfillment houses for communities of users who know what they want, and contract to have it made. So the flow of information the other way, the opposite of advertising, is very important. Not just the aggregated statistical info where we're all unidentifiable hamsters turning wheels around, the other kind -- intelligence, smarts, even brilliance and vision, that comes from not being immersed in the very limited vision of Silicon Valley which boils down to "we know what's right." What's great about what Costolo said is that he admits they don't know what's right. Hold on to that, and never let it go. The wisdom about your product is not in SOMA, it's "out there" -- the rest of the world that SV works so hard to abstract and simplify. It's not simple, it's rich and very very different from what you think it is.

What would be really smart is for him to say "Hold on, I'm going to take a few months here and really start listening to what the users think about this product, without any expectations." And really try to understand. Tht would do more long-term good for Twitter than what's more likely to happen -- a series of off-sites and talking with tech industry insiders, and more viral ad-driven money schemes.

Bottom-line -- Twitter's manifest destiny, if it's to be successful, is what Biz Stone said the other day in London -- to be the news system of the future. Now take a step back from that, and ask what Twitter has to do, what they have to change, to get there. It might be pretty radical, but it's still not too late, believe it or not. And when we look back in 10 years, with the benefit of hindsight, I'm pretty sure it'll seem to have been the right thing to do.

 11/26/2010; 1:27:52 PM.   .

I was telling a friend visiting from Berkeley that one of the nicest things about New York is restaurant delivery. You can have anything you want almost any time. It's like room service in a hotel, but better, cheaper and faster, with more variety. And with the advent of delivery.com and seamlessweb.com, you don't have to wait on hold to explain your order to some guy who doesn't care and doesn't speak English. It works because there's lots of competition. That doesn't mean, of course, that the food is good -- it might just sound good in the writeup. The other day I saw there was a new Italian place on delivery.com, at triple the price of my regular place. Decided to try it. After all -- maybe there's something really great about a $16 spaghetti and meatballs -- when the price at Pizza Mercato is $6. Turns out not so, the $6 plate is better and bigger than the $16 one.

Thanksgiving is when you give thanks. So thanks for food delivery.

Thanksgiving posts are a tradition here on scripting.com. I might have skipped a year here and there, but ever since the beginning, in 1994 (arrrgh!) the one holiday I usually didn't miss here was this one. It's a good holiday for American bloggers. It's a holiday we share with our friends to the north, but they for some reason think it should be a few months before ours.

Thanksgiving is an American thing, with a universal theme -- thank you! We're not celebrating one religion or political philosophy triumphing over another. There's something phony (to me) about the good cheer people show around Christmas. New Year's Day is good, but it's kind of arbitrary. But giving thanks -- it's a good thing to set aside one day just for that purpose. The end of November seems just the right time. It's a clear time of change and in the change, there is a lot to be thankful for. Sometimes we forget.

Recently I read a story about the inventor Buckminster Fuller, who at age 32, despondent about business failures, contemplated suicide. He decided that instead of dying, he would live the remainder of his life as if he had died. I wonder if this sounds like a foreign concept to you, because it's familiar to me. In 2002, I came home from the hospital to a house that felt like it belonged to a dead relative. I recognized the possessions as mine, at an intellectual level, but they seemed from some other person's life, as I had recognized the things in my grandmother's house after she died. These things are all familiar, but the person they belonged to is gone.

After such a shift your priorities change. Before that, I cared a lot about what people thought of me. I still do, but there's a twist. As I'm processing the insecure feeling that comes from disrespect, I remind myself that I'm gone, I'm dead -- the person they're dissing doesn't exist. Of course they forgot about me, I'm dead. I know it must sound weird, but that's where I'm at. If my Wikipedia page is wrong, well, no one is going to care after I'm gone, so ipso facto, no one cares.

Is ipso facto a real phrase? Have to look it up.

So how does one chart the course of a dead person?

The body has to do something every day. When a dead person wakes up in the morning, what's the first thing on his to-do list?

And the ego isn't really gone, but now it can be reasoned with.

Hard to write about, except on this one day. Because all there is to say is that I'm thankful for this day. I have no future and I don't remember my past. This is all there is. Finally those words make sense, in the way a mathematic theorum makes sense. No matter which direction you approach it from you get to the same place.

Funny, even though the past is a fiction, I still remember a lot of it.

Bob Hope, who lived to be 100 years old, sang about this in Thanks for the Memories.

So if we can be thankful for memories, let's try to keep the sweet ones, and keep the lessons learned from the not-sweet ones.

The great thing about memories is that every year there are more of them.

Moving across the country from a big house to a small apartment, from an ideal climate to one of extremes, time slows down. Thanks for that. It's hard to believe that we celebrated last year's Thanksgiving in Berkeley. This year -- Brooklyn.

Another year more to be thankful for.

I love the story about the guy who enjoyed his TSA patdown so much he wanted to do it again. That's the American can-do spirit. Awesome.

Thanks for the winter that's coming. I love walking in the city when it's cold. Bundled up with only my eyes and mouth exposed, each breath brings cold freshness into my core. My eyes see other bold explorers dressed as I am. Hello there. If there's slush on the ground (coming soon I think) every step is taken with thought. This let's your mind drift in interesting ways while your body is occupied with movement and safety.

Thanks for the bike riding of the summer. I thought I would do it through the winter, but it doesn't seem to be working out that way.

Thanks for living as if you were dead, I think that's not just a good place for a person to be, but it might be a good approach for the country, even the world, to adopt. It looks like the US is approaching the end of our empire. From here-on, we have many equals -- both in economics and in war. Maybe even "equal" is a lot to hope for. Even so, we flirt with disaster, seem to welcome it, judging by the leaders we elect.

But are the woes of one country anything compared to the brink we, as a planet, find ourselves on? It would be better if we gave credence to the idea that we live on a dead planet. If not, as a person who is on the brink would, we should prepare our papers and archives for the obvious and inevitable outcome.

But it's 2010 and we're still here.

Still here but dead. What a world!

 11/25/2010; 3:19:47 AM.   .

I moved into a FIOS-capable apartment in Manhattan a little over a month ago, and have had a chance to get accustomed to having FIOS-level performance from home. It's pretty nice!

I especially like the software in the Verizon router. As someone who has pretty much mastered port-forwarding and dynamic DNS, I appreciate the completeness of the UI. It can be hard to find some functions, but for most of them, it's all where you think it should be.

But then I wondered, as I started to set up a small server to run out of the house, taking over some of the work of my EC2 servers, why don't they give everyone a few gigabytes of static HTTP storage, right in the router. It would interface as a local-net-accessible folder. Any file you copy into the folder would be available over the net. Easy UI. A bit of the cloud right there on yer desk.

 11/25/2010; 2:56:17 AM.   .

I'm once again looking into using Posterous for podcasts. I see from the FAQ that they do them, but the question is, can it fit into our process for RBTN. Which goes like this...

1. Jay creates a post but it's not published. The post contains the items for this week's show. We go back and forth, then we do the show, and after it's over, he does a light edit to reflect what we actually talked about.

2. I prepare the MP3 that Adrian, our engineer, gives me. That means verifying that you can hear us talking, adding metadata, and uploading it to mp3.morningcoffeenotes.com. It's really important that all the MP3s be served out of the same folder. I've learned the hard way that if you don't do that, a few years later half your shows are somewhere else that has gone away or you don't remember where you put them. So I keep disciplined. This means that uploading the MP3 to Posterous is not an option.

3. Then I add the MP3 link to Jay's blog post so the CMS can scrape it. It's a totally wrong UI, but I didn't have any say in it, so don't blame me, I'd much rather just enter the URL in a dialog and have the CMS render it as an icon that communicates clearly to the user what's going on. Having an in-browser player is a pretty good idea too, and would just make our UI match that of the professional sites. All of these guys make us look bad, but meanwhile we've been upgrading the production of the show, and I think we've gotten a lot better at what we do. While this is happening our technical team (wordpress.com) has been making us look worse and worse. I know Matt's a business guy, this is a business issue. We care about our users, but we can't do it if the tools vendors don't care about theirs (i.e. us).

Anyway, I haven't really figured out how to or if I can make this work with Posterous.

Wordpress, Tumblr, Posterous.. are there other choices? Is anyone out there in PodcastLand happy with their CMS? Please let me know.

 11/23/2010; 12:32:00 PM.   .

As you may know, I do a weekly podcast with Jay Rosen called Rebooting The News.

This is a heads-up to people who subscribe to the podcast. There is a problem with the feed that will cause some subscribers to not get every episode. (It could be a majority of subscribers not getting any episodes, depending on how their clients are programmed.)

Technical: The problem appears to be caching. After a post is published, it builds the page, and caches the HTML. Somehow this is done before it realizes there is an MP3 attached. Eventually the cached feed expires, it rebuilds the feed, and now the MP3 is linked in as an enclosure. How long before this happens and how many clients have read the post and stored the guid and assume there is no MP3? Unknown. But a correctly written client that scans frequently will miss the MP3. My client does, for example

Until this problem is fixed, you should manually visit the site periodically to see if there's a new MP3 available and download it to to your listening device.

This is a repeat of a problem that hit our podcast in the beginning of the year. It was fixed, the feed was working properly for a few months, but it's broken again. The Automattic people are aware of the problem, and hopefully will have a fix soon. As I understand it, all podcast feeds on wordpress.com are broken, btw -- so if you're hosting a podcast there, you should be aware of the problem.

We continue to use WordPress because Jay is very comfortable with it. We checked out Tumblr, and it's not any better at podcast support than WordPress is.

A personal note, none of these programs work as well as the tools we had when podcasting was booting up in 2001-2004. Had our tools been this bad, there would be no podcasting, I'm sure of it. As these services have scaled, important features are breaking. This of course totally sucks.

 11/23/2010; 9:03:38 AM.   .

When I publish a picture from my iPhone, I:

1. Take the picture (that's the fun part).

2. Click the send icon at the bottom of the picture.

3. Choose Email from the dialog.

4. Type FU to get the first two letters of my flickrupload email address.

5. Wait because the iPhone is ridiculously slow at recognizing this sequence of clicks.

6. Click in the subject area of the envelope.

7. Type the title (a necessary step, no matter what method is used).

8. Click Send.

9. Choose the resolution from the dialog that pops up.

And that's just the beginning of the trip my picture is going to take. Now it has to get from Flickr to Twitter somehow. For that, I usually wait until I can get to a laptop with a net connection.

It'll be interesting to see how this compares to the way pictures are shared from the road in the future. Hopefully most of the steps will be gone.

 11/22/2010; 7:33:57 PM.   .

From the top of the Empire State Building.

 11/20/2010; 10:11:55 PM.   .

I've had a persistent problem with static hosting, going back to the beginning of scripting.com in 1995 or so. That's a long time to not solve a problem. While all the incubators start new tech companies with viral business models, one of the most basic ideas about the web, a folder that maps onto a website, still requires a degree in bicycle repair (not quite rocket science) to keep going.

I talk to my friend Chuck Shotton about this all the time. Chuck used to write web servers.

I say to Chuck one day we should knuckle down for a month and do an completely brain-dead simple self-hosting server, running on a Mac probably, cause it comes with Apache built in and turned on. We never seem to get around to it.

Meanwhile, Amazon made it so that if you stand on one foot while touching your nose with the pinky of your right hand and say Wild Beatnik Pie! three times you can host a static site in S3, which seems to have been made to serve static sites. My friend Joe Moreno has mastered this art, and posted a howto about it. I am going to proceed to follow his directions and set up a simple site in a S3 bucket, one that very rarely changes, to see what happens.

Joe Moreno: How to host a static site in S3.

1/2 hour later, I think I've set everything up right, but I get a "Sorry invalid request" when I try to visit www.smallpicture.com, a site I set up following Joe's instructions. Maybe it takes a while for things to show up?

When I tried to go to dsxuvulkqwac7.cloudfront.net it correctly linked to the index file in web.smallpicture.com, but I got an "access denied" from (what appears to be) S3. After changing the permissions the index file correctly displays, but I still get the invalid request message when I go to www.smallpicture.com.

I think the problem is in Joe's step 2. I changed it so that www.smallpicture.com is a CNAME for s3.amazonaws.com. I bet that fixes it and www.smallpicture.com will now correctly show the small Mona Lisa.

 11/19/2010; 6:35:20 PM.   .

Saw this headline roll by and had to remember to breathe. Until that moment I thought of Tumblr as a tiny company operating upstairs in a random midtown office. David Karp, its founder, is brilliant, and the site is growing at an incredible rate, and now the company is valued at $135 million. Oh to be young, talented and rich in NY in 2010.

Instant comments say its more bubble bidding, but it might not be. I hope Tumblr opts for a WordPress-style business model known as "freemium," where they give away the basic site, and charge for the extras. That will keep their interests nicely aligned with their users'. I think long-term, companies like Twitter and Facebook will have big problems as they have to narrow the margins for their participants, always competing with more members of their community, forcing them to either disappear or look for greener pastures. I don't see any clear line of places they can't go. But if Tumblr charges for sites, then its users are customers, and that's something that we're all comfortable with because we're either customers or have customers in the rest of our commercial lives. The user-as-hamster model is growing more uncomfortable all the time.

Anyway, it might not be obvious at first that this is not great news for the publishing business, as it's currently configured. Flush with cash, companies like Tumblr, Foursquare and others are mining tech talent in NYC, and bidding up the already-high prices for engineers. If you're a newspaper or magazine publisher in NYC, and you've been employing technical people, watch out -- you're competing with employers with newly deep pockets. And don't forget Google is (rumored to be) buying a huge office building in Chelsea, near the meatpacking district. It's going to be filled with technical people too.

Of all the tech wunderkinds out there, it's mainly WordPress that sees the media as customers. It probably would be a good idea if they had some competition.

 11/19/2010; 1:56:38 PM.   .

I've been hooked into the Hacker News firehose feed now for about a week and I love it.

I haven't seen any spam or abuse. It's a community that seems to value news and ideas. A lot of is opinionated, and some of the opinions are disagreeable to me, but the whole thing is very good. And I learn a lot from people I disagree with, anyway -- so it's all good.

I push a handful of the links I like from HN to Twitter, so you all are getting the benefit of my reading this new source.

Highly recommended!

 11/18/2010; 4:28:37 PM.   .

I saw this cartoon comparing SOAP to REST in the Hacker News firehose feed, and wondered why Fox-News-style politics has to be part of technology discussions. Where is the religion coming from? Don't people want to use the best tool for the job? Would people willingly take a big step backwards? The answer to the last question is absolutely yes. It happens so often.

Okay first -- let me get this out of the way -- SOAP is a mess. That's a strong statement coming from one of the guys credited with designing it. But I left the design process early because I didn't like the mess the big companies were making. I was quite vocal about it.

But the first versions of SOAP were really simple and easy to implement. You could do a full implementation in a weekend. The later versions were so complicated that you could never finish an implementation. No matter how complete you thought it was, someone could throw a call at your server that it wouldn't understand. A cynical observer might think the big companies liked this aspect of SOAP because they could claim to be standard-compliant without all that inconvenient interop!

Luckily the simple version of SOAP was published, in 1998, and a vibrant community developed around it. There are implementations in all the major development environments. It's baked into Python and the Mac OS. And a set of APIs for blogging tools was developed and were very widely deployed. All the major blogging tools supported the protocol, including WordPress which is so popular these days. I host the XML-RPC site, and to this day it gets a lot of traffic. A lot of people are, very quietly, using it.

So when people compare SOAP to REST, and not to XML-RPC I wonder if they know that REST is missing something that it should always have had, a standardized way of serializing structs, lists and scalars. That's something XML-RPC (and SOAP) have always had, and any discussion of these technologies should include that advantage relative to REST, which forces you to cook your own serialization with every API.

Because other developers like them, I implement REST interfaces all the time, and have been doing it for years. Every time I have to re-implement a layer that's provided built-into XML-RPC and SOAP. True, there is no single SOAP, and I would never, unless forced to, choose to implement an API in SOAP. But XML-RPC is a whole other matter. It's simple, easy, and it interops.

The need for a common serialization was so great that people are dumping XML in favor of JSON to get it. And when they do it, they talk about the incomprehensible complexity of SOAP. Too bad, because XML-RPC never had that complexity.

So come on engineers, stop being so Fox-like and open your eyes. Maybe we can do better than we have been doing.

 11/18/2010; 4:02:53 PM.   .

I'm a big believer in designers, programmers, writers, artists, news people all working together.

So it's in the spirit of a programmer looking for help from designers that I ask the following question.

What's the best way to display a River of News in a browser?

A river is a strict reverse-chronology of news items that conform more or less to the elements of a RSS 2.0 <item>. So you could have a title, link, description, publication date, link to comments, categories, or enclosures. There are a few other elements that can be in an item. For a full list, check out the RSS 2.0 spec. It's written so that a designer can make sense of it.

Here are some examples of rivers:

1. nytimesriver.com -- designed originally for Blackberries.

2. Arc90's rendering of the 2006-era river thanks to archive.org.

3. A screen shot of the Radio 8.0 river, in early 2002.

4. east-village.org, produced by River2.

5. Twitter is a river.

7. Les Orchard's news page (the descriptions would have to be initially expanded to make it fully a river).

8. Glowdart is very interesting. I'd like to experiment with a UI like this, but one where the user could give it any number of feeds to follow. In other words it's a nice start for a generalized RoN aggregator.

The design challenge is this. GIven the latest HTML techniques, do a mockup of a great River of News. If it's really something new, I'll put the software behind it and make it live.

 11/16/2010; 7:13:36 PM.   .

A very simple observation.

When the Republicans are in power and running up huge deficits, the issue of deficits is nowhere in sight.

During the last Bush presidency we should have been talking about deficits and the national debt and what we were passing on to future generations.

We were fighting two wars and had tax cuts at the same time. A very unusual confluence, and a really powerful way to accelerate debt. Usually wars are times of sacrifice, with rationing and high taxes, compulsory service. Even so we emerge from wars usually (if we win) with huge debts to pay off.

Now when the Democrats are in power, and the issue is (finally!) investing in America, all of a sudden the deficit is the big issue.

And who's making it such a big issue? The same deficit-loving Republicans (when they're in power).

Reality: No matter who's in power, what we as a country spend money on should always be an issue. Whether we go into debt or not is moot. There will be times when we will go into debt. It's as natural as the ebb and flow of the tides.

If the decision is spending our grandchildren's money on blowing up Iraq, or making them healthier or better educated -- I don't think it's much of a choice -- do you? Yet the Republicans seem to spend freely to kill Iraqis and fight against investing in America. We build new infrastructure in Iraq, and then blow it up, but we won't build new infrastructure in the US, at all. This is lunacy. We are collectively out of our minds if we let this happen.

Meanwhile our competitors invest in the health and education of their grandchildren. The next generation of Americans won't thank us for not investing in their future, they'll be the menial labor of their generation -- they will curse us.

"Running a deficit" is just another way of saying "investing." The hype is that we have to balance our budgets at home, so the government should too. But that's not true -- most of us run deficits, esp when we're young. If you have a mortgage, you're "running a deficit." And it's a good idea. When you're young you haven't had the time to accumulate enough money to buy a house, so the financial system lets you buy it a little at a time. Really, there's nothing imprudent there. If you're young and you're against debt, and don't have any -- you probably had rich parents.

Student loans are also a good idea if it means you'll earn $120K instead of $30K per year. You can pay off the loan with part of the higher salary. If you don't invest, the only way to grow is by windfall -- and our windfall in America has run out. We don't have great natural resources, we're a resource-importer. That means unless we want to spiral downward, we must invest. And that's why we have deficits.

We grew up thinking it was our birthright to run everything and own everything. The truth is, that was a bubble that had a lot to do with geography, and our advantage is largely over. That's not any party's fault, much larger forces are at work. Now we have to think to keep going, and think very well. And at just this time, thinking is something it seems people don't want to do.

PS: As with deficits, there's nothing wrong with earmarks. If the government decides to spend money, either the executive branch or the legislative branch decides how to spend it. (Occasionally the judicial branch makes spending decisions too.) If it's the legislative branch deciding, that's an earmark. If it's not an earmark that just means an employee of one of the departments, likelly not even an elected official, is making the decision. To campaign against earmarks, as the Republicans are preparing to do, is to assume the people have no idea how government works, and that no one (i.e. the press) is going to clue them in.

PPS: Love this story about the newly-elected Republican from Maryland, who campaigned to repeal health care reform, pissed off that his health care (free with his new job) doesn't start for a month after he's sworn in. It's okay for the rest of us to pay for his health care, but if we want help with the insurance companies, that's too expensive! These Republicans are such greedy hypocritical and heartless assholes.

 11/16/2010; 10:01:41 AM.   .

Just happened to stop in at apple.com this morning and saw this teaser.

Now what could it be?

I rolled it around for a bit, and decided to put a stake in the ground. I'm probably way wrong, but maybe there will some other interesting ideas.

I think we're going to see software for the iPhone that turns it into a remote control for iTunes. The two devices would connect over the LAN. And it will do things that a remote has never been able to do, because it has a software-controlled display and it knows what content is on your server that can be played. And it knows about your playlists.

Apple has always shipped fairly useless dinky remotes with Mac Mini's and Apple TVs (I don't think iMacs or laptops come with them) but I've never been able to use them. And likewise, the full-screen software for controlling iTunes is (I find) cumbersome and difficult to control. Seems like the UI for remote control belongs on an iPhone. And btw, there isn't an Android version of the software. Sorry.

That's my wild speculation for today!

PS: I like the idea that Apple is going directly to the customers instead of going through the press. No reason for a middleman here. They are perfectly capable of teasing on their own.

PPS: Names for such a product: iMote, airMote.

PPPS: As has been pointed out in the comments, they already have such a product. Never mind.

 11/15/2010; 3:01:52 PM.   .

I went to dinner last night with a couple of old friends who live on the Upper West Side. As I left my apartment, I thought I might text them to say I was on my way. As I walked to the train station I thought about an app I wished existed.

I'd like to connect my iPhone with the computer at my host's home, just for the next 1/2 hour, so they could watch my progress. That way if I was going to be ten minutes late they would know as soon as I knew. Obviously this is an opt-in thing on both ends.

Similarly if I'm going to visit someone who's going to pick me up at a train station. I used to ask people who were visiting me in Berkeley to text me when they got to the MacArthur station on BART. That was approximately when I would have to leave to meet them at the North Berkeley station. This could be automated.

Is it possible this app already exists?

 11/15/2010; 10:57:10 AM.   .

I woke up this morning to stories in my aggregator about path.com. I had heard about it before their launch, and was intrigued by the name. Turns out it's yet another instant photo sharing iPhone app. I have limited enthusiasm for them, I already use my iPhone, extensively, with Flickr and Twitter.

Even so, first thing this morning I signed up for a Path account on my iPhone.

After entering my name and email address, gender and password, it asked if it can use my location. I said yes. Then I went to the People section to start looking for friends to share my pictures with. I was astonished to see a list of suggestions, all of whom are people I know. I was confused. How could they know I know all these people? I jumped to an incorrect conclusion, they were all following me. I smiled -- it's really cool that all these people, some of whom I haven't spoken with in years, are following me on Path. After happily adding eight people (noting that Andrew Baron had signed up twice, with two different email addresses), I realized that can't be it. Some of these people are so totally offline they could never be using this app on its first day of public existence.

So I went to Twitter and asked if anyone knew how they were doing it.

fr8d had the obvious (in hindsight) answer: They looked in my iPhone's address book.

I never said they could. What else did they do with my contacts? Send a copy to their server for safe-keeping? Foolish me, but I thought that was my iPhone and my contact list. I paid huge money for the iPhone, so it's not like it could be anyone's "business model" to use that data. But now, as far as I know, some unknown startup in California has all my data.

As I fumed, I said -- The tech industry is a virus.

An analogy. I'm standing on a subway platform and someone behind me is reading my credit card numbers aloud. I turn around and see they have my wallet. At that point does it matter if they're going to use the info to buy some goodies at Crate & Barrel or is the damage already done?

It's like spammers took over technology, like the pet food guys did in 1999. Everyone has a scam. This year the scam is to grab all the user's data and resell it. It's gotten to the point where it's a risky proposition to try out a new iPhone product.

Another example. When I realized that any random Twitter app who you give your credentials to can download all your private direct messages, that was the end of me using Twitter apps that want credentials. Meanwhile the team at Twitter Corp has always had access to this info. Who's to say their interpretation of one of their terms of service is that they get to analyze and mine every bit of text I enter into the system even text that's only meant for one other person to read?

It's bad. That's the point of this message. I'm trying to end it on a positive note but I can't think of one.

 11/15/2010; 9:24:28 AM.   .

Great piece on Epicenter explaining, plausibly, how Facebook could grab Google's ad business by using data they gather better than Google does. Further, even though Google does accumulate some of this data, they can't use because they've promised not to.

I doubt a few years ago if Google thought this kind of route-around was possible.

The same thing happens on the journalism side of things as well. A few years ago CNET probably felt they could not be routed-around, but they were. It happens regularly, and will certainly happen again.

I have a theory that you can avoid being routed around if you don't try to erect the barriers at all. If you welcome all comers, and try to learn from them, including your competitors. In such a world, Google would have tried to do what Facebook and Twitter do as soon as they came around, without trying to use their installed base advantage to compete. I don't think that organizationally they could use their own people to do it, rather they would invest in the newcomers, aggressively.

One tech company that appears to get this is also a generally quiet company that doesn't bluster much, and tries like hell not to take sides in the technology wars. For 10 points, which company is that? Click on the plus next to this paragraph in the blog post for the answer.

 11/14/2010; 4:41:06 PM.   .

Fred Wilson has a post today about the explosion of platforms that developers may choose to support.

It's a race to see which one gets the killer app, the one everyone has to have, and then the other platforms will drop off, and there will be a consensus platform, the one "everyone" develops for. I hope, for the sake of all of us, that it's a platform without a platform vendor -- that no one gets to decide who has access to the market and who doesn't. Of course if that ever comes about, and the vendor throws around their weight too much, that's when the DoJ steps in, assuming the superrich haven't completely bought the government by then.

Anyway, it seems to usually happen that way. There were lots of personal computers until Visicalc came along, then the consensus was the Apple II. Until the IBM PC came along and everyone wanted to run Lotus. Then Windows and Excel, and the web and Yahoo, Amazon, Google, etc.

 11/14/2010; 3:34:55 PM.   .

Andrew Keen tweets: "While Facebook wants all of us to publicize our personal data, its strategy is to privatize its data from Google."

To which I responded: "Welcome to the tech industry! Welcome to your hamster cage!"

This isn't the first time I've written about hamster cages, but I haven't explained what I meant.

They make a wide variety of colorful and fun cages for hamsters that are designed to keep the hamster, and their human owners, entertained for hours. When you get tired of one, you can buy another. It looks great until you realize one day, that you can't get out! That's the whole point of a cage.

Remember how they used to say: "If it sounds too good to be true then it probably is?" They still say it.

Another one: "There's no such thing as a free lunch." Exactly.

When they say you get to use their social network for free, look for the hidden price. It's there. They're listening and watching. It's pretty and colorful and endlessly fun for you and your human owner.

 11/14/2010; 3:22:56 AM.   .

The feed is more or less done.

http://static.scripting.com/hackernews/rss.xml

Each item points to the article being linked to from the Hacker News entry. The link points to the HN discussion page.

The feed is realtime, it has a functional element, and the notification server works (at least for now, knock wood, praise Murphy).

I am following it in my personal river. It's a lot of news, but I like having it there.

You're welcome to poll it, but it only updates at most once a minute, so there's no point in polling more frequently.

The source of the flow is http://api.ihackernews.com/new.

PS: Lots of interesting comments about the feed over on Hacker News.

PPS: The Twitterfeed folk are using this feed to link the firehose to Twitter.

PPPS: Even better, they say they're going to hook into rssCloud!!

 11/13/2010; 11:16:32 AM.   .

Over 40 percent of all New Yorkers today were born outside the United States. This was true at the beginning of the 20th Century, but there was very little immigration after that. Only recently has the city re-opened for immigrants.

Immigrants are "colonizing" neighborhoods that, in the 60s, 70s, 80s and 90s, were in decline, thanks to urban renewal and redlining.

I can testify to that. The neighborhood I grew up in, Flushing, has become almost entirely Asian -- Chinese and Korean mostly. Much bigger than the Chinatown in Manhattan with lots of great places to eat, and what appears to be a booming economy. Russians have settled in Brooklyn, Indians in Jackson Heights.

I left New York in the 70s, and I never wanted to come back. I remembered big parts of the Bronx reduced to rubble, and the rest of it on its way. When I left, Brooklyn was in very bad shape. I always assumed my rejection of NY was more about me than the city, but it turns out that's not true. The city was in decline, it's only recently coming back. Things work now that never did when I was young. It's mostly safe, people go places that would never have been considered safe when I was growing up. Haven't heard anything about strikes. In the 60s and 70s all kinds of city workers, sanitation, transit, would strike.

You can definitely live in NY without a car. I've only driven my car twice since April, and that was just to move it from one place to another. I don't think there are any other places in the US where you can get by without a car.

However, I wish they could reduce the presence of cars in the city. I'm pretty scared to ride my bike here since the accident in September. That was caused by a car. I now live in the middle of the island, far away from the Hudson River bikeway. I know it sounds radical, but I wish we could give one avenue to bikes, and completely ban cars, buses and trucks. It would take a revolution to make that happen. But it would be transformative. Not proposing it, but thinking "what if."

Business-wise, there is a huge opportunity here, but the more I get to know people in business here, the more I hear them clinging to the ways of the past. It happens in California too. But the sooner the media, or some portion of it, fully embraces the new channels of communication and cuts its dependency on the distribution system of the past, the sooner New York can resume its central importance in the US economy. It could happen. It could be an extension of the rebound we're seeing here now.

 11/12/2010; 1:29:46 PM.   .

Got an email from a developer friend asking if RSS had rebooted yet.

Here's how I responded..

Well it never actually needed a reboot -- with billions of feeds updating all the time, it's pretty much like Facebook, without all the money.

 11/12/2010; 12:50:07 PM.   .

Bill Gates didn't invent the PC. Steve Jobs didn't invent graphic operating systems. Marc Andreessen didn't invent the web browser.

Mail lists existed long before Craig Newmark started Craig's List. Ward Cunningham implemented the first wiki, but Jimmy Wales built it up to world scale with Wikipedia.

Inventing isn't something that happens the way most people think it does. Or they use the word "invent" as a shorthand.

When people introduce me, they often say I invented RSS. Rather than argue, I smile and accept the compliment and thank them, and enjoy the glow.

But I didn't invent RSS. It isn't the kind of thing that gets invented. Not in the sense of Eureka! I just invented RSS.

If you've ever watched James Burke's Connections series, the invention of RSS was more a chain of events like that. News wires, CDF, XML, Microsoft, Adam Bosworth, Vignette, Netscape (with Wired, Red Herring, Salon and Motley Fool), My.Netscape, My.UserLand, blogging tools, NY Times.

RSS wasn't invented but it did become a standard.

Every step in that chain was necessary to get to the point where the Times could get on board, and there was enough software and users that it mattered. But looking back, the moment when RSS 2.0 came out, followed by the NY Times stories flowing through it, that was the point when the fighting stopped and mass-scale deployment began. That was the moment of standardization. And my contribution was that I marshalled the users, software, content and yes, the tech industry so they were all marching in the same direction. This, imho, was a lot harder than merely having an idea!

All this became clearer to me in a meeting yesterday with a major news publisher, in a 38th floor office overlooking the NY skyline. He said the publishing industry could never agree on anything like what we were talking about. I said "but they agreed on RSS." That turned the conversation in an interesting direction.

I always marvelled at how the publishing industry didn't fight the way the tech industry does. I learned yesterday that this is not true. The publishing execs probably didn't understand what they were doing by supporting RSS, but the Times was doing it, so it must be okay. (I guess, I'm an outsider to the publishing industry.)

The conversation continued. He asked why there was so much fighting over RSS. I said the tech industry doesn't like it when users prove they don't need them. That's probably why RSS has never been popular among the most powerful engineers at the big tech companies. It makes what they do seem less interesting. Of course that's why I like it, and that's why it made some important things possible.

And I think it's going to be important again, as the tech industry fights over who has the right to do what with the user's data. The problem they're going to have with RSS is that it lives outside their clouds, for the most part. It's been very interesting to watch TechCrunch covering the fight between Facebook and Google over user's contact data. I want to create data in places and ways that it isn't subject to that kind of manipulation. That is what RSS is and was all about.

RSS is a pretty good thing to have installed so widely. I think it's going to turn out to be a bit of a dormant seed, lying in wait, largely forgotten while we all learned about social networks. One day we're going to wake up and realize there's still a lot we can do with all those feeds that are still updating. And we're going to be glad that they aren't owned by Oracle, Microsoft, Google, Apple or Facebook.

Anyway to answer the question posed in the title of this piece, if you want to say I invented RSS, if it's a shorthand for all this michegas, go ahead. But it would be more accurate to say I made it a standard. That I think is the true story, the true accomplishment.

 11/11/2010; 10:51:29 AM.   .

I had lunch today with Ben Slivka, who I knew many years ago during the famous browser wars between Microsoft and virtually everyone else in the software business. Ben led the team at Microsoft that was working on Internet Explorer.

Back in those days a guy like me could have lunch with a guy like Ben and talk pretty frankly not only about what was going on in the technology, but also about the politics at Microsoft and elsewhere. Ben was his own guy, sort of a SWAT team leader, a programmer who more or less reported to the CEO of what was then a powerful company.

We went to Odessa where we both had a combo plate with stuffed cabbage, four perogies, a potato pancake, a big fat kielbasa and a pile of mushy sauerkraut. It was deliciously wonderful for a guy like me with a lot of Ukranian blood.

After that we went for a walk. He's from Seattle, so he saw things with his green-country eyes that my city-dweller eyes no longer see. He pointed out the razor wire on top of a fence, noting they don't have that in Seattle. I said that Pioneer Square seemed pretty grubby, and heard in myself a surprising chauvanism for NYC. I felt defensive on behalf of the city. I thought to myself, you could fit downtown Seattle in three square blocks of mid-town Manhattan (if you factor in the vertical dimension of the city). Seattle is small. Everything in NY is big.

We walked past Cooper Union through NYU-land, and straight into Washington Square Park where he observed there were a lot of people in the park. I was thinking the opposite, even though it was a sunny day, it was so crisp, bordering on cold, the park looked relatively empty to me.

He said people need parks in NYC because they don't have back yards.

This struck me as completely backwards!

I tried to explain how that was just one perspective, but I failed. Later I thought of the proper way of expressing it.

In Seattle they need back yards because they don't have parks. (I know they have parks, but stay with me here...)

Parks have different meaning in Manhattan than they do elsewhere. At first, I was really confused by this, but I've been reading about the history of NY, and working my way through the excellent PBS miniseries.

New York is a very planned city. Early-on, they planned a grid that went all the way to the top of the island, even though then, everything above where City Hall is now was country. And the city had hills and ponds, streams and forests, all of which would be levelled according to the plan. However in the plan they didn't allow space for parks. So uptown they created a huge 840-acre park, but it wasn't left to nature. They ripped the whole place up and created an interpretation of nature. The artist of the park, a man who had no design experience named Frederick Law Olmsted, was creative with the design of the park the way a composer is with a symphony or an artist with a canvas. As a result the park is as vibrant a part of city life as the sidewalks, streets and subways are. It was planned that way, as integral to the flow of the city. And on a huge scale you don't see anywhere else in the US. Nowhere in the US does it even approach the scale of New York.

The life of New York is both capitalist in the extreme and democratic in the extreme. When we use our parks here, we use them with everyone else, rich and poor, blue blood and fresh-off-the-boat immigrants. And the range of activities is mind-boggling. Visit the park on a summer weekend when the rest of the city is empty.

There's so much more to say.

I have had some wonderful back yards myself, and they have their advantages, for sure. Being by yourself is great, esp in the outdoors. Hot tubs, swimming pools, casual attire. You don't get any of that in the big city. But you sure don't get any of the big city in the small cities of the country.

 11/9/2010; 8:23:16 PM.   .

Following up on yesterday's post about the Hacker News firehose feed.

http://static.scripting.com/hackernews/rss.xml

Ronnie tweaked up his server and it's working again, and my app is polling his and updating the feed, above, once a minute.

When you read that feed you're not hitting a Ycombinator site, it's hosted in my space (on S3).

Turns out there are a lot of submissions, but that's why it's called a firehose.

 11/9/2010; 7:32:44 AM.   .

I love Hacker News, and appreciate that they have a feed for the stories that make it to the front page. But I want more. I want to get a stream of stories, as they are submitted. For that I'd need a feed that they don't have.

I asked about this on Hacker News, of course, and found that a clever developer in Texas, Ronnie Roller, had cobbled together an API by scraping the HTML of the website. It was a simple matter to create a tool that coverts the data from JSON to RSS, which is what I did in a couple of hours last night.

http://static.scripting.com/hackernews/rss.xml

That's the feed, but you'll note that it hasn't updated since early this morning. Apparently that's because Hacker News has blocked his API.

That doesn't seem consistent with the philosophy of the site. We're just playing with data and apps to see if there's something there worth developing. That's the only reason we like having the API, and Ronnie is running it entirely on his own dime.

So this is an appeal to whoever decides these things. Let us do this work. Or even better, provide us with an offical sanctioned API that connects directly to the database behind the HTML.

 11/8/2010; 5:52:10 PM.   .

First, I don't think we should call products like the iPhone and Droid "phones." Their phone functionality is fading quick. This probably would be happening even if the cell networks could keep up with the growth, but these devices don't really work as phones. I can call my mother on her land line, but a talk with Scoble, cell-to-cell is impossible. For the most part I, and many others, have given up. The devices we carry in our pockets are computers, not phones.

I'm coming to a similar realization about cable TV.

I hate the distribution technology, but I want (some of) the programming. I really want them to unbundle and let me buy only the services I want, and get the programming over TCP/IP, not through the settop box.

I just moved into a new apartment, and for the first time since I shut off Comcast in 2008, I have cable. Before that, I had EyeTV devices on my office and living room Macs, so I could watch digital programming, as long as it was broadcast over the air.

So here's my setup. I have a Mac Mini behind a 46-inch Sony display. I also have a new Apple TV, but I haven't used it yet. Too busy. I have a Motorola settop box, and a new Slingbox HD. I tried an EyeTV HD, but couldn't get it to work for more than 5 minutes. I returned it and got the Slingbox. But it sucks too.

I don't want to watch cable programming through the settop box.

I don't like the user interface, or the remote. I have become very accustomed to using the Mac desktop to navigate through EntertainmentLand. I want my cable programming to show up that way too.

Why should I pay Verizon a fee for being a middleman here? And why should I pay for all the services I don't use. Like anything but CNN, HBO, Showtime and whatever stations I need to watch the playoffs in baseball, basketball and football. Why should the channels I want share revenue with the ones I don't want, and with Verizon? Makes no sense.

 11/8/2010; 4:29:15 PM.   .

A very cryptic link just appeared on Hacker News.

http://twitter.com/#!/places/247f43d441defc03

Here's a screen shot with an arrow pointing at the interesting bit.

Looks like the Places feature is here now, turned on and ready to go.

All we need are some docs?

 11/7/2010; 6:52:15 PM.   .

I've been wanting a feed that listed all the new items posted on Hacker News as they came online.

I asked if such a feed exists, and was told that it didn't but there is an API provided by Ronnie Roller that would return the data.

I checked it out and it appears to work as advertised.

I wrote a little app that creates a feed from the API.

 11/7/2010; 5:21:09 PM.   .

A new browser, Rockmelt, launched in the last hour, and you can tell right off the bat if the software is anywhere near competent, and there's no reason why it shouldn't be (that's not the same thing as "is") that it's a winner.

Even though there are other ways to slice it, and maybe there isn't even that much demand for what it does.

No matter. It's going to get an installed base (with the caveat above about competence) and with an installed base comes momentum and value, and there won't be any shortage of money. They're rich overnight.

There should have been more high-level browsers.

And there should have been a browser with identity built in.

This browser is both high-level and understands identity.

But it's not a river! It's a hunt and peck style aggregator.

Okay, let's do the river.

 11/7/2010; 5:03:06 PM.   .

This is a followup to yesterday's piece about Obama and his shellacking.

I linked back to my first impressions of Obama's version of whitehouse.gov, back in January 2009.

"I want the White House to be a public space, where new thinking from all over the world meets other new thinking. A flow distributor. A two-way briefing book for the people and the government."

In other words: "People return to places that send them away."

Fast-forward two years, to late 2010. Now the hyperlocal web is booting up in amazing ways. I get inspiration from a dozen local blogs that are reaching into their communities and reporting on what people want to know. There are probably a hundred more. In two more years, there will be a million hyperlocals.

Applying a simple rule: "If you hear fire trucks in the night, in the morning you should be able to find out where the fire was."

In 2008 it was social networks. In 2012 it'll be the rebooted news system. And the new system of news will look a lot like today's hyperlocals.

Obama doesn't trust the press any more than he should, any more than Bush did. But that doesn't mean giving up on communication. The sitting President can't run campaign ads as an aspiring President does. But he has the ability to communicate more effectively than anyone else on the planet, if that power is developed. If you send people away to places that involve them.

The White House blog should be a daily link list of ideas and perspectives on what's happening in the world.

The White House blogger should be as independent as the Federal Reserve chairman or the head of the FDA. His or her job is to start small and build a network of information on how Americans can help America. Not fluff, not fear, but what's really going on. And to be controversial. Newt and Karl will say it's run by left-wing biased limp-wrist sissies. Let em say it. Link to them saying it.

Someone who is irreverent, idealistic and intelligent. Not a suit and someone who has never served in government.

 11/7/2010; 9:40:05 AM.   .

President Obama got a shellacking in the mid-term election. I hope health care reform isn't in jeopardy. I hope we find a way out of Afghanistan, and I hope if we hit another terrorist or financial crisis the Republicans and Democrats can find a way to work with each other.

But I think there's a bigger lesson here, and this is one of those moments when you can talk about lessons in politics. Most of the time people feel it's futile, and nothing can be done, but I think this is one of those moments that there is a message to hear, not from the insiders, but from the voters. This is what I hear.

First, I hear paradox. How can you make sense of a voter who feels the government should be doing more to create jobs, yet feels the government should stay out of things business does better. Who thinks the deficit is the number one problem but thinks we should spend more to create jobs. Who, like Reagan, thinks government is the problem, yet thinks the military is great. Or says keep your hands off my Medicare, while rallying against "socialized medicine."

Here's how you make sense of it.

Times are hard, and people think they're getting worse, and we feel powerless to do anything about it,. Except at one moment, when we pull the lever in the voting booth. That's the only time we have to be listened to. Otherwise, you and I have no way to make things better. So we do the thing that will be heard. Voting for the incumbent is a silent act. Voting for the first African-American president in 2008, was very audible. Voting for a Tea Party candidate in 2010 is the same kind of thing. That's how reconcile the two. Both are heard, both send a message. I want things to get better.

It was and will be especially bad for Obama because not only did he campaign on Change We Can Believe In (and in that he's far from unusual, most candidates run on change) but if you look at his skin color, and his youth, and his intelligence, and the way he organized using the Internet, you can see why this time around we thought things really might change. Not that the Republicans would all of a sudden stop trying to take money out of our pockets and give it to their friends in the defense industry and on Wall Street, that wasn't the change we were expecting. No, we were told that volunteerism was going to be the new norm. We would all have a way to pitch in to help. To make a difference. He wouldn't have been up against the Republicans if, on coming into office, he set up a Craigslist for volunteering, with meetups at a community level every month to talk about ways of making things work better. And during the rest of the time, no talk, just action. Keep the streets clean. Help out the homeless. Create new flows of information about how things are working, or not.

I think he had that opportunity, to carry the new way his campaign lifted him up and use the same energy to lift up everything. At least then, if it failed, it wouldn't have been his faillure, it would have been ours. But he didn't even try. Not even a bit.

I can point you to the exact moment of my disappointment. I was suckered by Obama's pitch. "The fierce urgency of now." I thought that when the new whitehouse.gov went live it would be about us, the people of the United States, but of course it wasn't -- it was about them -- those who took the standard places in the White House to do what they always do. Maybe slightly different. But nothing like the change we were promised, and some of us expected.

If Obama wants to know why he's being treated like a disappointment, the answer is simple, we are disappointed. Not so much that he's in the pocket of the bankers, and we're still in Afghanistan and Iraq for some reason no one understands, even though we were told that Iraq would already be behind us, and we (incorrectly) assumed that meant Afghanistan too. We need change, we need get our people involved, at least those who want to be involved, working to make things better every day of every year. Not just on the first Tuesday of every other November.

 11/6/2010; 1:35:50 PM.   .

I was talking tech at Andrew Baron's UWS apt the other night. Andrew is the founder of Rocketboom, and a pioneer of the art of short web news videos, going back to 2004 when that art was booting up.

Andrew told me that when he started Rocketboom, video content was distributed in QuickTime, Apple's video format. I remembered that well. Every weekday Rocketboom embedded a new QuickTime movie in their home page. Click play and the video would play. It was the same everywhere. QuickTime was the standard way to view video on the web.

Then something came along that took QuickTime out of its dominant role: YouTube. Instead of using QuickTime to display videos YouTube ran Flash. Almost overnight leadership changed from Apple to Adobe.

Somehow in the discussions about Apple banning Flash from iPads and iPhones and removing it from shipping Macs, this seems to have been overlooked. Apple's motivations might not be performance, the purity of the platform, or even battery life. It might simply be an effort to wipe out QuickTime's major competitor.

 11/6/2010; 7:52:52 AM.   .

 11/4/2010; 5:27:33 PM.   .

Thinking about switching from Verizon MiFi to Sprint to get some of that Wimax speed. But is it as good as advertised? The Sprint website isn't of much help. Is anyone actually using it? How fast is it? Reliable?

 11/4/2010; 4:20:25 PM.   .

This site, scripting.com, a blog that has been in continuous operation since 1997, is blocked to people whose computers are controlled by McAfee anti-virus.

Click on the thumb for a full-size view.

I hope someone can provide me with a list of the actual URLs of the files they're concerned with, but I'm pretty sure they're not compormised, and cannot harm your computer.

In the meantime, here's a list of the files in the screen shot. Some of them appear twice, not sure why. I've only listed each file once.

FrontierPsapiInstaller.zip -- here are the docs for this download. It's a DLL for Frontier, released (probably) in the late 90s, that allows certain verbs in Frontier's scripting language to work. I'm pretty sure the DLL was supplied to us by Microsoft. Never heard of a report of it causing problems. I have installed it many times, pretty sure it's included in the release of Frontier now.

snDecade.zip, snOpmlDecade.zip -- On April 1, 2007, I released a downloadable archive of the ten years of Scripting News from April 1997 to April 2007. The archive was provided in both HTML and OPML. These are those archives. HTML and OPML are text formats, and don't (can't) contain any executable code.

OPMLzip -- There is a snapshot of the OPML Editor release from April 2007 at this location. Pretty sure this is the opml.zip they're concerned about.

opmlArchive.zip -- a snapshot of the OPML of Scripting News taken in December 2007.

frontier423.zip -- the download of a major release of Frontier, which I provided as a download to show what blogging software was like in 1997. This is the software a lot of the early bloggers used. The blog post was called A link back to the beginning. I try to leave a trail behind, part of a philosophy I call Narrate Your Work. I'm glad I still have a copy of this ancient software, and I'm not taking it down or modifying it just because McAfee's software is out of control.

Update: Below is a list of URLs from the McAfee report.

 11/4/2010; 12:05:55 PM.   .

I'm on my way to Washington for the second time this week. Last time it was to have a discussion about rebooting the news at the Online News Association. This time I'm going down for a two-day meeting at the Library of Congress to talk about creating an archive of what they refer to as citizen journalism. I think that term doesn't capture what's going on, it's anachronistic, as horseless buggy seems today. Who would think you could hitch a horse to a car to get around, but people did used to do that. In the future, we'll get our news from the sources, so the question is how to create a record of what the sources are saying.

I'm one of the people who will kick off the meeting by saying how I came to be interested in this subject.

My story will be about various formats that seemed so pervasive that they were safe choices for archiving content. There was a time when the 5 inch Apple II floppy disk was ubiquitous. You didn't have to carry a computer with you, because you could be sure there would be an Apple II when you got where you were going. Today, such a disk would be useless. CP/M 8 inch floppies seemed the same way, and the hard-shell 3 inch disks used by the first Mac. Yet none of them have held up over time, and most of the stuff that was written on computers in the 80s is gone now, unless it was printed out. Printing turns out to be a pretty good way to back up digital content. Or it was. Today we create far too much material to rely on printing as a backup. We're going to have to come up with something else.

I came up against this after I left Berkman, when the RSS 2.0 spec, which was stored on one of their servers, became inaccessible. I was using a CMS I had written, and somehow the app had stopped running. The sysadmin of the Berkman site didn't know how to keep it going. That was a big lesson. If you want content to stick around, you have to take deliberate steps to make sure it survives. And there are some best practices. When I focused on this problem, I was able to arrive at a way to store the spec on a Harvard server such that now, six years later, it's still accessible. Whether it will be available next year is anyone's guess.

Academics have always had this problem. A university employs a scholar, sometimes for a lifetime. He or she creates a body of work, that then must be made available to future generations. That's why we have libraries at universities. But lately, as with all kinds of intellectual work, scholarship is being done on computers. So when a professor retires or dies, we are left with an array of electronic files and folders in a variety of formats. What use will they be in the future if the apps that can read them aren't maintained.

This blog and its related sites are another good example. As much as I don't like thinking about it, someday I am going to die. And when that happens, unless someone pays the ISPs, and someone relaunches the servers when they crash, and cleans out the databases when they fill up -- poof -- there goes Dave's online presence.

But we can do a lot better than we are doing, we just have to have the will to do it.

I've written about this many times, I call the topic future-safe archives.

A few bullet-points:

1. I want my content to be just like most of the rest of the content on the net. That way any tools create to preserve other people's stuff will apply to mine.

2. We need long-lived organizations to take part in a system we create to allow people to future-safe their content. Examples include major universities, the US government, insurance companies. The last place we should turn is the tech industry, where entities are decidedly not long-lived. This is probably not a domain for entrepreneurship.

3. If you can afford to pay to future-safe your content, you should. An endowment is the result, which generates annuities, that keeps the archive running.

4. Rather than converting content, it would be better if it was initially created in future-safe form. That way the professor's archive would already be preserved, from the moment he or she presses Save.

5. The format must be factored for simplicity. Our descendents are going to have to understand it. Let's not embarass ourselves, or cause them to give up.

6. The format should probably be static HTML.

7. ??

 11/3/2010; 9:03:41 AM.   .

I'm writing code while the CNN idiots are chattering about nothing behind me. They're talking such nonsense that it doesn't interfere with my concentration. It's relaxing.

If you understand how programming works that means they're not saying anything my cerebral cortex needs to focus on, and I have no expectation that they will say anything I have to listen to with my intellect. I can focus fully on the program runtime environment.

In the rebooted system I'll have Nate Silver and people he listens to, talking behind me. I won't be able to write code while that's going on. (Or write a blog post.)

Will Nate's live video stream come via the settop box or via the FIOS connection? Who cares? The net effect is exactly the same.

I'll be doing that for sure in 2012. We're almost there now.

PS: It's been a long time since I've watched cable news. I watched the World Series (love the Giants), and now I'm "watching" election returns. Only thing notable that's changed is the political ads appear to new fears about China. Maybe the Koch Brothers are preparing for World War III?

 11/2/2010; 7:16:28 PM.   .

There's been a trickle of other blogs picking up on the idea.

Great mockup on TiPb.

Why??

Why would Apple want to do this? To create as large a market as possible for IOS apps.

Why do they care about that? Well, they make a lot of money selling those apps. That's a pretty good reason all on its own.

But even more important, for Apple, any developer energy that's applied to an Apple platform is energy that isn't applied to a Google, Microsoft, Oracle, Blackberry, Nokia platform.

I probably left someone out. Oh yea. The web.

 11/2/2010; 4:32:51 PM.   .

Krugman is a Natural Born Blogger.

You can tell just from reading his blog. It's especially important because he's blogging at nytimes.com. In that sense what he's doing is very subversive because he's an expert writing without interpretation by a middleman, a reporter. There's no one between him, the expert, and me, a person who wants access to his expertise.

He's a Source that Goes Direct.

I especially go for his angry pieces (like this one). What makes Krugman angry is the same thing that gets me going -- ignorance.

Ignorance is the practice of ignoring. To be ignorant is not the same thing as being stupid. Albert Einstein was surely ignorant of some things.

We all do it. The world is too big and complex to pay attention to everything, no matter how smart you are.

Ignorance becomes negative, however, when you ignore things that are important. Ignore the pedestrians in the crosswalk as you drive through it -- that's seriously ignorant, for you and for the pedestrians. Ignore that a weak dollar must be balanced by a strong currency somewhere is ignorance. It's just math. There's no way around it.

The same kind of reasoning applies to various systems. The news system is rebooting. That's the premise of a lot of my thinking. The political system is not rebooting the way I hoped it would. We're still voting against our interest, we're still giving money to people who use the money to buy votes so that they can take even more money etc etc We'll see tomorrow how bad it is, but they say it's going to be pretty bad.

I think the systems are linked. The smarter the news system is the smarter the political system can be. And the converse is true as well, until the news system gets smart, the political system can't. That's why the smart anger at a leading news place like the Times is a very positive sign. Krugman himself is good news.

We have to care enough to not be ignorant of our interests. We're not there yet. But there are hopeful signs.

 11/2/2010; 11:13:32 AM.   .

I think Steve played a little sleight of hand with us the other day when they announced the Back to the Mac business.

It just came to me when I took an Angry Birds break with my iPad. Why can't this run on the Mac? Answer: it can. (Or it soon can, maybe next month or the month after.)

Everyone was wondering what kind of software would be sold for the Macintosh version of the Apple Store. Marco even said there would be a new class of little software, you know -- like the stuff we buy for the iPad and iPhone.

Close, but not quite.

The software we will buy from the Mac version of the Apple Store will be the actual software that runs on the iPad and iPhone.

In other words, they're teaching the developers, privately, how to write iPad software for an iPad with a keyboard. In other words, the MacBook Air.

Why didn't I see this? One of my first wishes when I got my iPad was that this software would run on a Mac. I forgot that, and Uncle Steve said it the other way. The store is coming to the Mac. The store is coming to the Mac. That's the sleight of hand. What he really meant to say is that IOS software is coming to the Mac. Or maybe it's the IOS hardware I'm writing this on is running Mac software, kind of the way Carbon ran old lifeless legacy Mac apps. Which one is the "real" OS and which one is running in a compatibility box? I have a funny feeling that right now, as I type this on an AirBook, I'm using the compatibility box. Right? (I imagine some of my IOS developer friends reading this, wishing they could tell me, but for an NDA.)

Put it another way, who has the patience to play Mac games now that we've been playing iPad games?

What else?

All the other iPad apps.

PS: Everyone who said I was full of shit when I said the trackpad was the harbinger of IOS apps on Mac hardware -- I owe you all an I Told You So.

 11/1/2010; 2:44:59 PM.   .

Two seminal pieces totally worth reading.

1. David at 37 Signals says Apple must use their new power more carefully. Been saying saying the same thing here. Not just in this loop but in previous ones. I urged Microsoft not to try to snuff out Netscape, to leave the web alone. My argument was that they'd get 60 percent of the growth no matter what, just selling more operating systems, word processors, etc. They needed the ideation process to run outside The Borg, because nothing creative comes out of the hive-mind. They can't hire the kind of people who build the stuff that makes a new layer of tech. Just kick back, I urged Bill, to no avail, and count your chips. Your pile is about to get much taller, and you don't have to do a thing. He completely went the other way, and hit two brick walls: 1. The reality of the hive. 2. The US government. My guess is that Steve Jobs didn't learn from Gates's mistakes, and now that it's his turn, he's going to fuck it up, probably in a much bigger Jobsian way. Because that's how he is.

2. Mike Arrington at TechCrunch is returning to his roots, and that's a great thing! Reflecting on his experience with TechCrunch, and obviously thinking about what's next for himself, he's asking the good questions. Who am I, what am I capable of, which direction should I head in next. I encourage him to take his time here, because he'll be living with the decisions he makes now for a long time. He's at a personal inflection point. And Mike is a powerful and super-smart guy. Expect something interesting from his corner of the world. Sent him an email of congratulations. Good work.

3. One more thing, today is Mitch Kapor's 60th birthday. In a tweet he says it's not 16. No doubt. I'm only 55, so how would I know, but I knew Mitch when he was not much more than 16. He's still the same guy he was then, only smarter and wiser. One of my best teachers when I was starting out, it was from Mitch that I learned the review process of software. That you have to take time out and put down the compiler and actually use the stuff you're creating, to solve the problem it was intended to solve. Without this reality check, done often and with thought and care, your chance of shipping something that works as a product are pretty slim. You'll miss the target every time. Seems so simple, but so often this is the step people miss.

PS: I used the word tweet in this piece. Sorry Twitter, I'm not going to capitalize the T. Nice try.

 11/1/2010; 12:46:23 PM.   .