Scripting News

The next toy

Tue, 23 Oct 2007 03:03:37 GMT

Something new in News

Mon, 22 Oct 2007 16:23:24 GMT

It's not very often that you see something new in News.

News is not exactly new technology, but when personal computers came along, and then widespread networking, it created a whole new playing field for news, that has shaken things up for most of my life. Change comes in fits at starts. First there was the web, then RSS, and now I think we're on the cusp of another bit of change.

If you want to see what I think it looks like, check out the home page of nytimesriver.com. But that's not the end of the story. A flat completely chronologic view of news probably isn't enough. And earlier this month at a meeting in NY, two engineers at the NY Times set me off in a new direction, with a very simple bit of advice. They told me to look in the HTML source code of their stories. When I did I saw they had applied a taxonomy to their news flow, and this opened the door to what I would like to show you today -- an outline view of the news.

http://nytimesriver.com/outline/

I believe it's fairly self-explanatory.

The topics are arranged in order of frequency in today's news.

In a previous rendering, the stories were shown in a histogram, but this view I think is much better. You can still see how many pieces relate to the indicated topic, but by clicking on the plus next to each topic, you can actually see the headlines and descriptions, and if you want more you can click through to the full stories. (Initially, the outline was sorted by frequency, with the most frequently occurring keyword appearing first. I changed it, based on feedback, to be alphabetic.)

Now, there's still more to do, I showed this to a number of people during the weekend and got some excellent clues on ideas to pursue next, and I will do that. Further, in the process of exploring this, I've been shown the work of other developers who discovered the keywords on their own, and one in particular is very interesting. I'm hoping that these projects will come public so I can show them to you and tell you what I think they mean.

This is what I live for, professionally -- the sense of being somewhere with great unexplored potential, a virgin landscape of the intellect. I'm never happier than when I get to play in such a place.

Dan Gillmor: "Dave Winer has been exploring a superb news resource, exploring the depth and breadth of the New York TimesŒ data-stream."

Bijan Sabet: Dave's River of News.

Scott Rosenberg: Remixing news.

Om Malik: A new way to view news.

Please comment on the screen shot page.

It pays to keep an eye on Comcast

Mon, 22 Oct 2007 19:30:02 GMT

Over the weekend there were several reports that confirm that Comcast is interfering with their customers' use of BitTorrent. This raises several more questions and concerns.

1. There are infringing uses of BitTorrent, for sure, but why is Comcast taking the role of enforcer against the interest of their customers. Just a question, but not likely to get an answer, because Comcast officially denies they're doing anything, even though employees (apparently) are confirming, not for attribution, that they are.

2. What about non-infringing uses of BitTorrent? Can their algorithms tell if someone is using BitTorrent to share mamterial that they have the legal right to distribute? If not, how do they justify interfering with their customers' use of the Internet?

3. And perhaps most disturbing, what does this say for the future? Perhaps someday it will be deemed inappropriate for people to publish content to the Internet, if so, could Comcast take steps to block that activity? How different is this from interfering with BitTorrent?

It pays to have a clean garbage disposal

Mon, 22 Oct 2007 17:44:57 GMT

Another nugget I thought would be good to share.

I came home from my trip to NY and there was a pretty bad stink in the kitchen.

Smelled like garbage.

It didn't take long to zero in on the source -- the garbage disposal.

I tried pouring all kinds of cleaning stuff down the drain, to no avail, the smell didn't go away.

Then I did a search on the Internet, found a variety of suggestions, and felt pretty sure that I'd have to call a plumber because they all seemed to assume skills and/or tools I didn't have. Instead I tried a very simple idea and amazingly it worked.

1. Put a drain stopper in the disposal so no water can flow out through the bottom

2. Fill the sink with hot water mixed with soap and chlorine bleach. The hotter the water the better.

3. With the water running put one hand on the stopper and the other on the switch, as you pull out the stopper, turn on the switch. (Be sure to do it in that order, to keep your hand from getting chopped up!)

4. Let all the water run out of the sink and leave the water running as long as there's suds.

5. Repeat two or three times. Wait a day or two. With any luck the smell will be gone.

The reason this works is that junk gets stuck on the walls of the drain, and since it's garbage, it rots and stinks. By immersing it in soapy water, the junk gets dislodged and goes down the drain and out of your life.

It pays to mull things over

Mon, 22 Oct 2007 15:52:03 GMT

I just figured something out, and it's the kind of thing that's best said publicly, even though it's likely to: 1. Be misunderstood and 2. Upset some people.

But since it's all about this blog it really is best to air it here.

First what triggered the epiphany.

I was over at Loic Le Meur's house in San Francisco yesterday having lunch with his family and friends. We were all drinking wine (very good wine of course), enjoying the view, and talking about this and that, when the subject turned to Mike Arrington. Loic said that Mike told him that we used to be best friends. I couldn't figure out what that meant, because our friendship was the business kind of friendship not the personal kind. What does it mean to be best friends in that way? And how does that relate to having a blog? It never occurred to me that friendship meant that (here's the epiphany) that I would only say positive things about Mike's business. It didn't occur to me until I heard Loic's side of a blog-fight that I saw happen from a distance, with Sam Sethi, Mike and Loic.

Complicated? You bet. Too complicated. An unspoken deal that I never agreed to.

When Mike was starting TechCrunch, I pointed to his blog all the time, with glowing praise, because I was truly impressed with what he was doing and because I wanted to encourage other people to do it too. I wanted people to write about technology products based on how they used them, not based on alliances, investment, posturing of execs, the crappy stuff that means almost nothing to users, and imho is just a substitute for actually understanding the technology. Mike was approaching products the way I felt they should be approached. Hence the praise.

Fact is, my opinion of Mike, as a person, hasn't changed much in the last couple of years. He has a personal charm and charisma that not everyone finds appealing, but I do. I like hanging out with the guy.

However, that doesn't mean that if my opinion of TechCrunch, his business, isn't uniformly positive (and of course these days it is actually fairly negative) that I will withhold it. But it's also part of Mike's way of dealing with people that he sees criticism as betrayal. I just don't see it that way. I've had the shit kicked out of me so many times, and as a programmer I understand that criticism is necessary to perfect a user interface, even to get the damned thing working, that even when it hurts, I have to push the hurt aside and listen to what people are saying, and try to respond to it. Professionalism demands it.

Anyway, one of the reasons I want to write this now is that I've written about Loic's business here a couple of times in very positive terms. I don't want anyone, esp Loic, to assume that this will always be so. If they get in the way of other creative people, or otherwise act as a poor example of entrepreneurship in technology, of course I will write about it, and will say what I think. I would expect Loic and people at his company to take what I write to heart, and consider it. My feelings won't be hurt if they don't do what I say. (People almost never do.)

Same as when I said Facebook sucks. Or when I criticize Techmeme. This isn't in any way meant to reflect on the quality of the people at Facebook (some of whom I know to be outstanding people of high principle) or Gabe Rivera who I know to be a very smart and competent and honorable person. It's possible to critcize someone's work and still admire the person. My epiphany is that a lot of people who thought were my friend, didn't understand this very basic thing about me.

8/17/07: Friendship and blogging.

A life lesson

Sun, 21 Oct 2007 16:24:03 GMT

It's not often you learn a life lesson without any pain, but that happened this week, starting with a blog post by Fred Wilson, that I've now applied twice, once successfully, and the second time, we'll find out.

Fred is a venture capitalist. An important part of his job is evaluating and deciding on opportunities to invest. For every company he invests in, he turns down many more. So how do you turn someone down without being personal? Well, you probably can't. So a lot of VCs side-step the problem and never turn anyone down, they just stop returning calls, or blame their partners.

Fred decided to tell people the truth -- not only that he's not going to invest, he'll also tell you why. I think this is a good idea (here's the lesson) because the person might be able to fix the problem, and Fred will get to invest, and the person's idea will get a chance to become a company.

I tried it yesterday in a negotiation at a furniture store, and it worked.

First, when buying furniture you're expected to negotiate, the sticker price is just a starting point. But I hate to negotiate, even though I know I have to. When I hesitated about whether I would make the purchase, the sales person said "Of course you get a ten percent discount." And if I said 15 percent? She said sure. I said to my companion, I bet she would have gone to 20. I looked at the sales person, she put a pained look on her face and said okay.

I didn't feel sorry for her, because they'll still make 40 percent of what I pay as gross profit, if the percentages are the same as when I sold software through retail in the 80s.

Then I decided to add a couple of lamps to the purchase. She said of course since those are accessories I would only get ten percent off. I grumbled to my companion, thought about it for a bit, and said "I'll pay, but I feel really bad about this." I thought some more and decided I wouldn't shop there again.

Then I thought of Fred and his policy of telling the truth, so I told the sales person that I'd not shop there again. She gave me 20 percent off. Telling the truth was the right thing to do because it gave her a chance to fix the problem and keep me as a customer. And I still feel a little slimed, knowing what I know about retail and margins, and I may not shop there again anyway. But that's another lesson.

Guy Kawasaki: The Top Ten Lies of Venture Capitalists.

College Ave furniture store

Sat, 20 Oct 2007 23:58:35 GMT

View Larger Map

Problems with expand/collapse

Sat, 20 Oct 2007 19:48:26 GMT

Last night I changed the way Scripting News is rendered in HTML, and while it works in Firefox on the Mac (the browser that I use) it is broken in a bunch of others. This afternoon I'm going to try to get it working everywhere.

The advice from readers, some of it quite confusing, starts here.

I've got MSIE 6 running in Parallels, so as I go I'm testing there and in Firefox/Mac.

Here are the changes I'm making, in order...

1. Apparently the <a name="xxx"> element is causing a problem, the purpose of it is to enable permalinks to work within the archive pages, Colin suggests making this the name of the <div>, so that's what I did. (This got today's elements expanding and collapsing in IE, but not older days. Very weird.)

2. In the stylesheet, added width:400px; to both .show and .hide and padding-left:15px; to .show. (That successfully widened the body of each post in MSIE.)

3. I eliminated the table I was using to indent the body text. (Now the older days expand and collapse. Hurrah!)

4. Added another 5 pixels of padding for a little bit more indenting.

5. At this point it appears to work in both MSIE 6 and Firefox/Mac. I will now download Opera and try it there. (Downloaded and installed, but I can't get it to display any web pages including scripting.com. Very very strange. If you have Opera installed, could you try clicking on the pluses and minuses on scripting.com and let me know if it works. Apparently it does.)

6. Removed some old CSS and Javascript includes from the head section.

7. Added "min-width:400px;" to the .hide style, per Colin's advice.

Validating the validator

Sat, 20 Oct 2007 03:21:20 GMT

Todd Cochrane noticed that feedvalidator.org is reporting problems with feeds it used to pass. I checked it out and verified the problems he reported. As far as I can see there's nothing wrong with Todd's feed, imho the validator should not be warning about the problems it's warning about. Please, would the maintainers of the validator check this out and make whatever corrections are necessary. Thanks!!

Thanks to Colin Faulkingham

Sat, 20 Oct 2007 02:57:47 GMT

I added his expand/collapse code to the HTML rendering of Scripting News, per my recent request.

It seems to work nicely.

Does it work in Opera? Please let me know.

LobbyCon 2.0 predicts the Newsroom of the Future

Fri, 19 Oct 2007 17:16:13 GMT

The lobby of the Palace Hotel was a hub of activity after lunch yesterday. The picture to the right hardly does it justice. A constant stream of friends, a wide variety of ages and backgrounds, flowed through.

It was unlike anything I had seen before, likely because this conference was held in downtown SF, and not San Diego or Phoenix, and a facet of what we'll have when the Newsroom of the Future is up and running. Every city will have one, but San Francisco will probably be first.

Here's a video I took last week at CUNY that gives another perspective. Lots of tables, video screens, a stage, radio and TV equipment. What you can't see is that the room was saturated with wifi, and while it didn't have a huge presence on the Internet, it could have.

Today's new toy

Fri, 19 Oct 2007 14:47:33 GMT

Today's new (developer's) toy is an OPML 2.0 rendering of the NY Times keywords list. Have fun!

A mighty torrent of news!

Fri, 19 Oct 2007 13:23:53 GMT

Doc: "A year from now every newspaper will have a newsriver."

I think so too because: 1. The idea is so compelling and 2. It's so easy to implement.

Thanks so much to Doc Searls for writing a great evangelical piece about the power of rivers. The stream is turning into a current, and soon really will be a river.

It¹s so weird to see rivers show up in Facebook, and Twitter is just a big river of all the people you¹re following.

The idea is actually a descendent of the teletype terminals that used to be in the movies (and for all I know in actual newsrooms). The news was printed on scrolls of paper, and when a new story came in it would push the older stories onto the floor. You could catch up on the news by scrolling back through the news. Cary Grant, Rosalind Russell, Katherine Hepburn did it. We¹ll all be doing it soon enough. And it really helps to get other people singing the song, esp from within the hallowed halls of Harvard.

The more attention we get focused on it, the more other developers will tune in.

And what may not be so clear from the narrative is that this project got its start from a meeting I had with some technical people at the Times last week in NY. Like most organizations, it's not of one mind, there are people who are scared of what comes next but there are others who know that the Times has to change. By opening up their internal data to me, all kinds of interesting stuff can happen. We've been here before. The Times are the unsung heroes of RSS, without them it never would have solidified, with the publishing industry falling in behind the Times. It was this consensus that created critical mass for RSS 2.0 in 2002 and 2003.

I really hope some of this stuff feeds back into the Times support of RSS. And as you have seen, there are now lots of new opportunities in user interface for news. This is what I do, when I'm in my "flow" -- we're there now again, with a new toy to build and then play with, every day.

Betsy Devine: "Dave Winer has been improving the New York Times for as long as I¹ve known him."

Phil Windley: "With more sources, who are themselves continuously updating, the keyword river could be as dynamic as you¹d want it to be."

The Boston Red Sox

Fri, 19 Oct 2007 13:56:14 GMT

If I were an American League fan there's little doubt that I would be a Red Sox fan. They have it all. Fenway Park. The Green Monster. The Curse of the Bambino. And an ancient legacy of sucking and when it looks like they're not sucking so bad, blowing it in the worst way possible at the last possible moment. The drama of the Red Sox, the agony of their fans. The only more hapless team in baseball is my beloved New York Mets. (And possibly the Chicago Cubs.)

Before last night's game they were down 3-1 in the ALCS, but they won, and now they're down 3-2. These are long odds, but with the Red Sox, you never know. (They were down 3-0 in 2004 and came back, amazingly, to beat the Yankees, a sweet wonderful humiliating defeat.)

For some reason, last night I thought of Aaron Copland's Appalachian Spring as an appropriate anthem for this moment in Red Sox time. The pioneers have their ups and downs, theres's still hope, but they've suffered greatly. I think of Dowbrigade, hunkered down, feeling sure his team will exceed his worst expectation of disaster.

With the Red Sox, as with Jerry Garcia and Scripting News, it's even worse than it appears!

A new view of NY Times news

Thu, 18 Oct 2007 14:01:06 GMT

After spending a day with the old keyword page, and getting bored with it, I came up with a new way to look at news, something I've not tried before, that might be fun and/or useful

http://nytimesriver.com/keywords.html

Since it's likely to change again soon, here's a screen shot.

How it works. Every hour, as usual, it does the nytimesriver scan. Every story is linked to in the database undern all the keywords it references. Then the report, in HTML, is prepared, with the keywords in the left column, and links to all the stories in the right colum. The list is sorted by number of references, the keywords with the most references appear at the top of the list.

So today, baseball is the top item, with 15 references. The teams, the Cleveland Indians and Colorado Rockies, rank high. For some reason (heh) the Boston Red Sox don't appear, even though they're still in it, and the Yankees, even though they've been eliminated (yay!) are near the top at position 10.

It's another leaderboard! (Oh shit.)

The stories age, and are removed after 24 hours. After all this is news, not olds.

If you have comments, post them under the screen shot, linked above.

PS: Note that since the list just started up today, the initial stories, even though some are already 24 hours old, will remain in the queue until tomorrow morning at this time. So the list will be artificially fat today, it'll thin down tomorrow.

North Berkeley BART station

Thu, 18 Oct 2007 23:16:34 GMT

Robert Scoble

Thu, 18 Oct 2007 20:37:55 GMT

Ted Leonsis

Thu, 18 Oct 2007 18:51:16 GMT

Help, I need really solid expand-collapse code

Thu, 18 Oct 2007 13:25:40 GMT

I really need a rock-solid expand-collapse display that I can integrate with Scripting News.

If you're reading this in RSS, flip over to the home page and have a look. See how the pluses and minuses work? There are a bunch of problems:

1. Doesn't work in Opera. Deal-stopper. Opera users are cool folk.

2. I don't like the indentation. I want the text flush-left.

3. Takes too long to display. I want it to be instantaneous.

4. Must be multi-level. I haven't tested the code I'm using with more than two levels.

I know it can be much better, because I see it done better in lots of places.

Looking for help from people who know their Javascript. Me, I'm into other things (obviously).

Mock up a Scripting News home page with your code, and post a pointer, and we'll test it out. When it's done, everyone will be able to use it.

BTW, I know there are people from the Radio community that have stuff in this area, I just don't know how well supported the stuff is these days. It's been a while.

Colin Faulkingham has a mockup that works in Opera, etc.

More NY Times digging

Thu, 18 Oct 2007 13:07:25 GMT

Before heading over to the Web 2.0 conference at the Palace Hotel in SF this morning, I couldn't resist doing some more digging into alternate user interfaces for news reading based on the NY Times keywords.

One thing I learned the hard way is that when you access the Times site from a script (not through a browser) if you try to read an article that's too old (not sure what the time limit is) it tries to redirect you to a login page (which is pretty pointless considering that there's no human being around to log in). I hit this problem yesterday, and then hit it again this morning, but couldn't remember what the problem was. So by writing it up this time I hope to remember.

No doubt some debunked hack journalist posing as a tech industry mogul will slander me for this, but the ~~asshat~~ asshole has no idea how blogging works, and who the fuck cares what he thinks anyway.

"cheesecake"