Microsoft and XML-RPC
Friday, January 29, 1999 by Dave Winer.
Good afternoon DaveNet readers.. 
Lot's of things changing over here, sorry for not writing, I miss doing it, but it's been a bit hectic, with the growth of XML-RPC, new thinking on using computers in meetings, releases of alphas and betas of content management and workstation products, politics with a big platform vendor, and a visit from my Jamaican uncle coming thru San Francisco on his way to Thailand!
It's all very confusing. Like Bill Clinton I'm trying to "compartmentalize" as best as I can. I don't like to write public pieces when I'm confused, but there's so much to catch up on, I have to start somewhere, so here goes!
The Clinton Trial
I've written lots of unpublished stuff on the politics in Washington, then I got a pointer to a piece in the Austin Chronicle and I threw all my writing in the bit bucket. The author, Michael Ventura, said it much better than I had.
http://www.auschron.com/issues/vol18/issue21/cols.ventura.html
As I read this piece I felt stirred. He says it so well and with such clarity, it gave me goosebumps reading the piece.
One pet peeve, there's no publication date on piece, or in the HTML source code for the page. I feel every article on the web should clearly say when it was published. But no matter when it was written, this is great writing and clear thinking.
Technography
Switching gears.
I've reconnected in a very pleasing way with a longtime friend, Bernie De Koven.
Bernie is a game designer, he recently left Mattel, where he was part of the team that did the Barbie Fashion Designer software.
We've been friends since the early 80s when he was a user of the bulletin-board that I operated out of my Menlo Park living room, running on an Apple II with a ten megabyte Corvus hard drive.
Between then and now, Bernie has had lots of game-designing jobs and has explored a new idea called "technography" in the late 80s and early-mid 90s.
Technography is revolutionary. It's a new application for computers that's on the verge of being discovered by thousands, maybe millions of users. It's newly relevant in the age of meetings and net-connected executives and engineers.
How does technography work? Add a laptop and a smart person to a meeting. Project the laptop so other people can see the screen. Soon the screen becomes the "surface" of the meeting. People start pointing at text and say "Move this there!" or "This belongs here," and the technographer, with a few waves of the mouse, does what he or she is told.
People can log into the meeting thru a web browser. Decisions are made, resources allocated, committments given and taken. Technography increases the value of the organization in substantial, even huge ways.
The meeting can start on the web, days or weeks before the meeting, and extend days or weeks into the future. The meeting becomes a project, a process. Any conference or meeting, of any size, can benefit from technography.
A technography-managed meeting works differently from the meetings you go to today. No notepads. The purpose of the meeting is to create a document. One document, not twenty or two hundred. People who are shy are given ways to contribute safely. Technography is about tools for communication. And of course technology is central to technography.
I'm re-learning this from my friend Bernie. I've believed in the power of group idea processing since I started doing it with my team at Living Videotext in the mid-80s. Now fifteen years later, computers are widespread, the skills and knowledge are out there. The users are starting to do it, so it's time for me to tap my lovely friend Bernie on the shoulder and say, Bernie, let's have some fun!
And of course, my walking buddy Bernie says yes!
So here we go...
http://technography.userland.com/
I'll be writing more about technography, and sharing Bernie's writing. We'll do how-tos and Technography for Dummies. I look forward to the O'Reilly book, Technography in a Nutshell. It's like XML, but instead of being geek-level, this is totally user-level stuff, and of course it's connected to XML, but let me and my technical friends worry about that.
Alphas and betas
Along with our renewed interest in technography we've been making important progress in the core outlining software built into Frontier. This is happening in version 6.0, which is, in its various pieces, a workstation, a server, and a content management system. The kernel is in alpha, on the hard disks of our server testing group, and the core of the server, non-kernel stuff, is being tested more broadly.
One of the key features in 6.0 is borrowed from MORE, a product which many enthusiasts believe was the height of the art of outlining in the mid-late 80s. The new feature, fat headlines, allows Frontier outlines to be used for writing. I can testify to that, for the last three months I've been doing most of my writing in this outliner. Here's a screen shot..
http://technography.userland.com/screenShot
Finally, two groups are going to meet each other. People who write and think in structures. These are the technographers and idea facilitators. The other group are people who manage servers and websites, the people we've been working with since 1995, when we turned to the web.
Outliners meet webmasters. And vice versa!
MORE files rescued
At the same time, an engineer in the Macintosh division at Microsoft, Brad Pettit, has been working on a utility that extracts text from MORE binary files.
What? Why would a Microsoft engineer be interested in rescuing data from MORE? Well Brad is a generous fellow, and was one of the engineers who worked on MORE. Brad went to Apple, and then landed at Microsoft. Like me, he feels a bond with MORE users, and it's great that, thanks to Brad, there's now an easy way to get ideas out of MORE and into other text tools.
http://www.labmed.umn.edu/~john/more/index.html#MORE2Text
Of course Frontier 6 will be able to read the files that Brad's utility creates.
Search engines
The rest of this piece is about XML-RPC.
After gaining new implementations, in Java, Perl and Python, the next question was what to do with it.
Luckily, we had an idea waiting for this moment.
We do content management and group writing software for Windows/Mac. I see this as a single seamless platform, in our world you can intermix Macs and Wintel machines at will. You can have Mac servers or Windows servers, and any number of writing or content management workstations, all integrated thru HTTP and XML. We do the things that our host platforms are good at doing.
At the same time as Mac/Windows has been getting good at doing this stuff, Unix kept going, the Internet developed around Unix, and while the user amenities and tools on Unix are still behind those on Mac and Windows, they're better at delivering raw computing and storage power on the server side.
We've been working on a search engine that runs on Windows/Mac, it's perfect for the searching functionality of a large publishing site, or for a community composed of a several hundred smaller sites. However, I would not try to do an Alta Vista or Inktomi-level search engine with our software. It doesn't scale that high, at least not at this time.
XML-RPC to the rescue! We've documented the XML-RPC interface for our search engine, and hereby invite our friends in the Unix world to clone it. I envision a search engine running behind Apache on Linux, storing the info in MySql, all seamlessly connected to content systems, using XML-RPC. I put out this invitation to a smaller audience on Scripting News, and yesterday we got a bite! Stephen Judd, a webmaster at the University of Waikato in New Zealand, put up a search engine that supports our interface, and then I wrote a script in Frontier running on Windows that added my home page to his database. I then did a query, and my page showed up. All that happened in less than fourteen hours.
Now I want to evangelize this idea more broadly. Here's a link to the message, pointing to all the information you need to clone this stuff on Unix. Please please clone it. There's lots of business we can do here.
http://discuss.userland.com/msgReader$2454
Microsoft and XML-RPC
Finally, I want to very briefly tell you a behind-the-scenes story about how XML-RPC came into being. It may appear to be a solo effort on the part of UserLand, but it's not.
In February of last year I wrote a piece about using HTTP and XML to implement a cross-network wiring protocol. I write lots of pieces like this, here's an idea, does anyone think it's interesting?
http://www.scripting.com/davenet/98/02/rpcOverHttpViaXml.html
I got an email from Bob Atkinson, one of the lead architects in the COM group at Microsoft. I had met Bob many years ago at a series of Microsoft design reviews for OLE while it was in gestation. Bob was enthusiastic and asked if we wanted to work with them on the spec. Of course I said yes, and a week later I was meeting with Bob and a few other key people in the COM group, in a conference room in Redmond.
I did a little technography. On my laptop screen, projected, I brought up a Notepad file containing a sample RPC message. They said "change that" and I changed it. I learned from the messes at Apple that, if possible, agree with the platform vendor. These were smart guys. They let me keep some of my tags, I let them put in some of theirs. At the end of the day we had an agreement and a plan to proceed.
Over the next few weeks we went back and forth, and quickly the influence of the rest of Microsoft could be felt. I was surprised, my impression was that Microsoft moves in lockstep, that they execute quickly, but I found that this is not true. They interfere with each other, second-guessing and FUDing, internally, much more than they do with the outside world. As someone outside Microsoft I was much more empowered to move quickly than any of my collaborators at Microsoft.
So we decided to take a snapshot of the spec we were working on with Microsoft and make it public without putting Microsoft's name on it. This happened on April 4, 1998.
The spec worked. We were able to deploy, in Frontier 5.1, a robust underpinning for all kinds of distributed applications. Most of our users don't know that it's XML-RPC that keeps our search engine current or synchronizes their object database with the one on our support server.
I stayed in contact with the people at Microsoft. In the summer they did a design review with a number of big developers. I attended the meeting and felt very well supported by the Microsoft people. They encouraged me to speak, to explain the ideas, to respond to the feedback. My name is on the spec they circulated at the meeting. As individuals they have been wonderful to work with, and as a company, I am sympathetic, but they have been very frustrating!
As XML-RPC gained traction outside of the Microsoft-UserLand connection, we've had to face a tough choice. Do we negotiate with people developing on other platforms, or do we negotiate with our silent partner, Microsoft, or somehow keep trying to be the bridge connecting them? I eventually saw this as untenable. I already have a fulltime job, and I have certain strengths, but other people have other strengths. The system was highly inefficient and too centered around me.
So I privately lobbied Microsoft to join the public discussion group, to talk directly with the people implementing XML-RPC on other operating systems. I know that I made life difficult for people at Microsoft, but I also strongly believe it's in their interest to work in the networked development community. If they want Microsoft systems to be fully relevant in the new networking paradigm that they co-invented, now is the time to work. The train is leaving the station. It's happening now.
And here's a key point. I've been getting credit for excellent work done at Microsoft. Now I would like to correct that. There's a magic in that spec, the object serialization format, expressed in two simple tags, <struct> and <array>. It was designed by Atkinson and others at Microsoft. I am proud to have played a part in the design, and glad to do my part in promoting it, and happy that it's built into my product. But the design of XML-RPC is a gift from minds at Microsoft to the net community. Does this blow your mind? It should!
Anyway, after much lobbying and internal politicking, Atkinson posted public responses to questions raised by an engineer in the Internet Imaging Operation at Hewlett-Packard and a developer at the Swiss Federal Institute of Technology, Institute of Robotics and the lead architect at Allaire. You can see how far this stuff has spread, and Microsoft's involvement surely can help it spread even further.
I would like to stand up and cheer. Microsoft is doing what few would expect them to do. They're working within the confines of a delivery-oriented networked working group, developing a communicaiton protocol that's equally valuable to Windows, Mac and Unix. Because our spec is already deployed, there's no FUD possible. This is why, when we first started working with Microsoft on this last year, I sent a memo to Bill Gates, congratulating him for having more courage than I had ever seen in one of the gorillas of the computer industry.
Think of it this way. If I develop a search engine running on NT, it can be replaced by a search engine running on Solaris or Linux with a simple emulation of an open protocol. XML-RPC is far simpler than COM, CORBA or Apple Events. Time to deployment, from scratch, is measured in days, not years. We can move very quickly. And customers get choice.
In 1991 Gates said there's no value in plumbing. At the time he was lobbying Apple to let others build networks compatible with Macintosh networks. The great thing is now, with the shoe on the other foot, Gates and company have been living up to that promise. Out of deference to our working relationship, I've not said anything about this. Now that their involvement is public, I can.
Thank you Microsoft.