Monday, July 10, 2006

Okay, no Rocketboom today, but zere is a Ze Frank.  

Andrew Grumet: "We fixed the XML button linkage." 

Six years ago today: Why I like XML. 

Lance Knobel: "Maintain your critical faculties while reading Wikipedia." 

Did you make the list of desired speakers for the Next Web conference? Mike Arrington isn't on the list (he spoke this year) or Marc Canter, nor is Marissa Mayer. I didn't make the list. But Scoble did. Damn.  

Matt Mullenweg is hosting a WordPress user's conference in SF on Aug 5. That's an awesome idea. I'm scheduled to be in Montana on that day, but a shuffle may be called for. 

Washington Post piece on YouTube and politics. 

Mark Cuban says click fraud isn't self-correcting.  

Chris Heuer: "No gathering is immune from becoming a snooze fest unless the format demands participation and the facilitators can engage everyone present." 

Suggestion to Podshow 

Next time I suggest you do private reviews before doing a public launch. This will get rid of the concern that these mistakes serve your company's interest by drawing attention to your service via controversy. Also would appreciate you taking this at face value, I'm pretty fed up with attacks from Podshow people when helping debug this stuff in public.

builtins.fileSynch 

I'm working on builtins.fileSynch, a generalization of the folderWatcher functionality in the OPML Editor, and the successor to Radio's upstreaming. It's simple code now, and therefore easy to tune up and optimize, which is what I'm looking at now.

Paolo says he has a www folder with 5000 files. So I decided to test the performance of fileSynch with 12 files, then 112 files, then 1012 files, and finally 5012 files. The percent of CPU used is presented in this spreadsheet.

It's scanning the folder every 10 seconds for changes. Clearly it should scan less frequently, since with 5000 files it's using 40 percent of the CPU just looking for changes (even when it doesn't find any). This is not an acceptable situation, and provides a clue why Radio users were finding performance sucking after a few years of use.

Now I'm running the 5000-file test checking every 30 seconds. (Result: 33 percent.)

I know that both Mac OS and Windows have the ability to notify apps when there's a change to a folder, completely eliminating the need to poll. But the kernel doesn't have hooks for that functionality, so for now this is the only way to go. (Also, for sure, I'm still going to get emails from people telling me I'm stupid for polling and not using the OS functions.)

There's no structure to the test folder, it's just one folder with 5000 files. I will try experiments with different folder structures and see what happens.