XML-RPC for Newbies
Tuesday, July 14, 1998 by Dave Winer.
Yesterday I sent you a pointer to a story about Microsoft and XML-RPC, which according to Microsoft, is in the early stages of development.
A quote from the piece: 'Microsoft officials said the XML-based protocol fits into its goal to interoperate with other platforms and vendors. "We're committed to interoperate as never before," said Vic Gundotra, director of platform marketing at Microsoft.'
Right on! It's taken fifteen years for this issue to rise to the top as an important agenda item for the software industry. I hope it stays right at the top until we get a solid Internet-based cross-platform remote procedure calling protocol.
But first... What is it?
XML-RPC for Newbies 
Inside every computer, every time you click a key or the mouse, thousands of "procedure calls" are spawned, analyzing, computing and then acting on your gestures.
For example, when you move your mouse over an icon, the computer calls a procedure, LocateMouse, to figure out what you're pointing at.
Is the mouse pointing at a menu? At a scroll bar? If so, which part of the scroll bar? Is it pointing at an icon? Or some text? Every possibility is considered.
OK, so let's say the mouse is pointing at an icon when you click the mouse button. What kind of icon is it? If it's a printer, call the procedure that prints things. What do you do when the user clicks the mouse on this kind of icon? A special procedure answers that question.
This is the kind of chatter that's going on inside your computer *all the time*, even when you aren't there. It's always asking questions. And the answers come from procedures. To get an answer, the software "calls" the procedure.
Parameters and returned values
Along with the call, the procedure might require some extra information, so it doesn't have to recompute things that other procedures might have already figured out. These are called "parameters".
The LocateMouse procedure might need to be told where the mouse is. The location of the mouse is usually expressed in a coordinate system like the Cartesian plane that you learned about in high school. An x coordinate and a y coordinate. Such a procedure would be said to "take" two parameters, an x and a y.
Parameters are important for three reasons. First, why do the work again when the caller probably already knows where the mouse is? And second, the mouse may have moved in the time it took to call the LocateMouse procedure. And third, a procedure may be called to do some computation, for example, to look up a record in a database. Such a procedure would require a user's name or account number as a parameter, to identify the record that's to be looked up.
And there's the motivation for the third part of a procedure call -- the returned value. It's the answer that the procedure sends back to the procedure that called it. A database-access procedure call might return a set of values, all the elements of the record indicated by the key identifier it received as a parameter.
What is a procedure call?
So, now we're ready to say, concisely, what a procedure call is.
A procedure call is the name of a procedure, its parameters, and the result it returns.
Why are procedure calls important? A very simple answer. Without them, there would be no computers!
Every program is just a single procedure called main, every operating system has a main procedure called a kernel. There's a top level to every program that sits in a loop waiting for something to happen and then distributes control to a hierarchy of procedures that respond. This is at the heart of interactivity and networking, it's at the heart of software.
What is RPC?
RPC is a very simple extension to the procedure call idea, it says let's create connections between procedures that are running in different applications, or on different machines.
Conceptually, there's no difference between a local procedure call and a remote one, but they are implemented differently, perform differently (RPC is much slower) and therefore are used for different things.
Remote calls are "marshalled" into a format that can be understood on the other side of the connection. As long as two machines agree on a format, they can talk to each other. That's why Windows machines can be networked with other Windows machines, and Macs can talk to Macs, etc. The value in a standardized cross-platform approach for RPC is that it allows Unix machines to talk to Windows machines and vice versa.
What is XML-RPC?
There are an almost infinite number of formats possible. One possible format is XML, a new language that both humans and computers can read. XML-RPC uses XML as the marshalling format. It allows Macs to easily make procedure calls to software running on Windows machines and BeOS machines, as well as all flavors of Unix and Java, and IBM mainframes, and PDAs and sewing machines (they have computers in them too these days).
With XML it's easy to see what it's doing, and it's also relatively easy to marshall the internal procedure call format into a remote format.
Why RPC is important
OK, now that we understand what XML-RPC is, let the XML part fade into the background. It's an implementation detail. Programmers are interested in XML, as are web developers, but if you're a user or an investor, XML is about as important as C++ or Java. The developers like it, or seem to, and that's the only major take-away from the XML part of XML-RPC.
But RPC is important, no matter what format is used, because it allows choices, you can replace a component with another one; and it opens possibilities, empowering advanced users to develop solutions with packaged software that the developers didn't anticipate.
Choices
For me, the trail goes back to the mid-80s when my company released an integrated product called MORE which was both an outliner and a presentation product. There was a lot of synergy in the integration, but a significant number of users just wanted the (new) presentation features without the outliner; and a significant number wanted the well established outlining features, and had no interest in presentation features.
Both groups wanted a choice, they wanted us to separate the two pieces and connect them together. If someone wanted the outliner they wouldn't have to buy the presentation software and vice versa. But if they had both, you'd get all the integration features. It was a great idea! I've told this story many times. It was a turning point in my career. The users were asking for something reasonable, yet there were no operating system underpinnings to allow such a separation.
A few years later, the needed technology was in the operating system, and we had built software, a database and scripting engine, around the technology. Application developers implemented interfaces that allowed our software to "call" them. We'd send some parameters to a procedure, implemented in another app, and the app would send back an answer. Every app that implemented this could now be viewed as a toolkit that an advanced user could build new applications thru. In this world, we could have done an outliner and a presentation program, and linked them together with "glue" scripts.
Choice is good for users, and for some developers, in some situations, it's not that good. So it was a mixed bag. In some cases, web servers for example, there were absolute standards that were adhered to. And in others, page layout programs for example, the "wires" were seen as a competitive advantage, and never got in synch.
Possibilities
Another advantage of RPC is that users may see opportunities for integration that developers don't understand, combining the features of two or more products. With the ability to glue apps together, an inspired user can do it. The developers may never understand or care what the users are doing, but it can happen anyway. Interfaces provide room for growth, open doors, allow a user-perspective to create new functionality. This vision says that users can be empowered to make integration choices without the support or help from software developers.
This vision was realized in some areas. Databases and web sites are a hot area today, but all the pioneering work was done by users! How many database vendors understood the web when it first started booming? But flip the question around, how many web developers needed databases? Almost all of them figured it out.
Because there are standard interfaces for databases, the web people could do what they wanted, even though the database developers didn't have a clue what they were doing.
Trade-offs
As with all software technologies, there are trade-offs. When you integrate, when the procedure call is within the same app, you get much greater performance and more user-oriented features because it's easier to do a procedure call within a single app than it is to go outside of the app with a remote procedure call.
Apps will not disappear. In the early heady days of this sub-industry, some people thought they would. But there's too much momentum behind the idea of a process implementing internal procedure calls. So, long live the app! It's a good thing. It's still here, for a while at least.
Back to Microsoft
According to Jeff Walsh's InfoWorld article, Microsoft is planning to open up their operating system using XML-RPC. Such a protocol could be deployed quickly in other operating systems that support HTTP, ranging from Perl scripts running on Linux, to Quark publishing systems running on Macs, to relational databases running on mainframe systems. It could put Windows at the center of a new kind of web, one built of logic, storing objects and methods, not just pages and graphics.
As you know, I'm a big believer in this kind of glue. I've been talking with Bill Gates about this since 1985, when we met to talk about interapplication communication for MS-DOS. Then in 1991 we revisited the idea as Apple was deploying the Apple Event Manager on the Macintosh OS. Apple wasn't willing to license the technology, so Microsoft did COM. But COM mainly runs on Windows, and although there have been recent efforts to popularize it on other operating systems, it'll be an uphill battle because most operating systems have their own native way of connecting systems together at a remote procedure call level.
So, by agreeing, at least at a philosophic level, that XML-RPC is an important way to go, Microsoft is putting out a big Welcome Mat -- come rule our world, they say. You can control Windows without adopting COM. You can replace an NT machine with any other machine that supports the same interfaces.
Even if NT should become a locked-in standard in the enterprise (far from an accomplished feat at this time) users will be able to substitute other operating systems for NT thru simple interfaces. I support that, I think it's a winning strategy, because I know how much value customers place on choices and new possibilities.
It's the lesson that the Internet teaches. It's a lesson that Gates learned so well that he's upping the ante. Instead of a hodge-podge of different wire protocols and payload formats, let's get something in place that can quickly be widely deployed everywhere. Why wait? That's what Microsoft is asking.
There's no reason to wait. Let's get just-enough technology in place and then let a thousand flowers bloom. I absolutely don't care what the wire format is, as long as I can develop simple scripting interfaces on top of it. I give up control, everyone does, and we let the messy cacaphony of competition take over.
That's the way to go.
Dave Winer
Updated 7/21/98: The XML-RPC spec is available. DW