Script Meridians
Thursday, January 22, 1998 by Dave Winer.
Addicted to Noise? 
I've become addicted to the Senate impeachment trial, the way many people became fascinated with the OJ Simpson trial. I lament the day it ends. I find it frustrating and enticing at the same time. It's frustrating when they miss a key point, and enticing because every once in a while they show me a new way of looking at things that I had never considered.
I know that most of my fellow Americans aren't tuning in, and I find that remarkable. There's so much to learn here! It's a crash course in the Constitution, a great structure for organizing a government. It reminds me that I don't live in an intellectually curious country. What to do about it? For now, nothing.
At the same time, the public opinion polls make the biggest headlines. In other words, we ask the people who know the least to make the decision for us. Wow. Takes your breath away and makes me wonder if me and this country are going to part ways sometime soon. And of course it a true appreciation for the First Amendment, a lot of people are going to tell me to shut up for excercising my right to have an opinion.
I work in my own world, letting the Senate worry about the issue of the disposal of the president, staying somewhat separated from the decision they must make. We elected them to do that, and I have no choice but to trust them to do the right thing.
And I guess that's the word I want to spread. The US Constitution says nothing about public opinion polls, and wisely so. I think the polls are too easily manipulated by television. This is a time to think and make hard decisions, for the people we elected, for the system we all agree on.
In the meantime, I'm writing for the Frontier community, sort of like Clinton's and his State of the Union speech, and I wanted to share some of that writing with DaveNet readers. Some of the most popular DaveNet pieces are the ones that reveal the process of our company. I am aware that there's a lot of knowledge assumed in this piece that many DaveNet readers don't have. Please allow me some leeway here. (I'm talking like Henry Hyde!) I hope I'm not an annoyance in the bosom of the Internet!
Venting
Last week, an important Frontier developer posted a controversial message on the Script Meridian community list, and it started a bit of venting, and instead of getting directly involved, I decided to unsubscribe from the list and read it thru the web-interface instead. It's good because the immediate temptation to respond is removed. I can't respond, I can only read. I've used this technique in the past to gain perspective and it works, for me at least.
As I read the messages I realized that we're largely in agreement, and we've been working quietly to address the problems for quite a few months. In this memo I hope to provide a update, a status report for Frontier developers on what UserLand has been doing in the latter part of 1998 and early 1999.
Our search engine is open
Our new search engine supports an interface that allows anyone to send an XML-RPC message containing the title of the page, the text on the page, its URL and the name of the website that contains the page. Within an hour of receipt of the message, the search engine adds the page to to the index. So, if you want to include your website in our search engine, it's a very small amount of work. Quite a few people have already done it. The call fits into your finalFilter script neatly, except for a small delay when rendering you hardly know it's being indexed, but it is.
This was a visionary thing for us to do for a couple of reasons. First, we're the first search engine to do this! Think of the possibilities, virtually instant indexing with no crawling. I truly believe that someday all search engines will work this way. And second, since the search engine we're developing is an integral part of Frontier 6, this open architecture will be an optional part of every Frontier-hosted content management system (or CMS). So there's no lock-in happening here. As soon as we release Frontier 6, anyone will be able to operate a distributed search engine.
We see the search engine as being the key idea to create a unified web presence for all information about Frontier, regardless of what website it's stored on. And then it can branch out and be the search engine for other focused communities, or it can be a standard-bearer, leading the way for Perl or Python-based search engines to implement compatible XML-RPC interfaces.
This is what I meant when I said on the Frontier-users list that there may be pieces of the puzzle that weren't evident at the time. It still may not be evident to many people. New ideas take a while to settle in and become ordinary. At one time or another every part of Frontier was this new and hard to understand. That's the way it goes.
It's hard to find stuff
But only if you don't use the search engine. We're providing two primary interfaces for content on our website, time and searching. The News pages and the calendar interfaces are the time interface, and search.userland.com is the search interface. If you remember you once saw something on one of our sites, now you can find it. You don't have to worry about where things are stored. Focus on the search engine as the primary way of finding things. That's how we're developing our docs as we go forward. As a web analyst and software designer, I watch where other sites are going. This is where they're going, and rightly so, and so are we.
The docs are incomplete
We have a formula for working with the Support Associates to fill in the holes. A major part of that process is the question and answer cycle on Script Meridian and Frontier-users. Just by asking a question on either of these lists you're helping to improve the docs. So please keep doing that.
Emmanuel has pointed out that Perl has better printed docs than Frontier. It's true. If we want our printed docs to get better, and we do, the first step is to organize and fill in the online docs. You can only get so far by lamenting the lack of docs. You've identified the problem. Now please help us solve it. All you have to do is search and ask questions when you don't find things. Let us worry about how the printed docs come about, that's part of our job.
The Discussion Group interface
Some people don't like the Discussion Group interface, but I do, because I see it differently. It's not at all like a mail list. It's a web interface into our content management system. This is a very popular idea for a lot of good reasons. Web browsers are ubiquitous and convenient. I have an Edit this Page button on my pages so I can go back and forth very quickly. And we have an XML-RPC interface in the works that connects from the DG to the new Frontier 6 outliner, so you lose no fidelity in editing tools, in fact you gain quite a lot. Think of the DG as a framework for group writing for a web presence you'll get the idea of what we're doing.
To think that somehow we're trying to replace Script Meridian is to completely miss the point of the DG. We're developing new software for you and your teams. It wouldn't work without the DG. And if you don't like the interface, cook your own. Make it work thru email. Go where ever you want. We want to establish a guest-database format for storing documents while they're in development by a web team. Content stored in the DG can be displayed thru any template you want to use. Users of your site may have no idea that your team is coordinating thru a web-interface. But having a collaborative web interface is key to running a dynamic site. All our competitors are moving this way. It's the right way to move!
Why are we using it as forum software on userland.com? To burn it in, get user feedback, and to bootstrap our own internal workflow. We've already started to use it internally to manage our own web content. This is the necessary precursor to others using it in this mode.
No breakage
One thing that hasn't come up in this round of email is the subject of breakage. This is a big issue here at UserLand. Where there was a lot of breakage going from 4 to 5, the transition from 5 to 6 will be relatively simple and smoothe.
We've added a lot of features to Frontier server-side, but the new features are all implemented in a single responder that runs alongside the responders you already have. And when you decide to move a .wsf site from 5 to 6, you'll find the path is well-paved with simple instructions and a troubleshooting checklist, and the experience of twenty or thirty well-known Frontier gods and godesses to help you as you get started.
It's part of our slow-down-and-get-it-right philosophy that has been a key part of our development process since we went commercial last summer.
Working with outside authors
We can avoid discontinuities with book authors by slowing down and carefully specifying the software interfaces. That's exactly what we've been doing. We know that these problems are not insurmountable, in fact we know how to do it, having shipped commercial software continuously for the last twenty years.
We've worked with a dozen book authors in that timeframe, every one is different, but they all have common needs, to know where we're working and where we're going. And vice versa, we also need to know where they're working and where they're going.
So our process is open to anyone who has a book contract, or who knows Frontier deeply and wants us to help them sell a book. We will open our process and coordinate our work with book authors.
But I want to stress this, the number one thing we can do is to ship complete online docs for new functionality. We're doing that with Frontier 6. When we ship it, you will find that the howtos are already written, and that detailed specs are available. There won't be guesswork. You'll know how the software works if you read the docs.
You can get in the loop if you want to
Here's where you make a decision about how smoothe a ride you want.
If you want a really smoothe ride on Frontier 6, then wait, and we'll let you know that we're confident that it'll go smoothly for you.
However, if you want to get an idea of what we're doing, read the docs. They're on the developer's site. Start on the home page and then dive into each of the subjects. Like all new things, it may take two or more readings for the ideas to sink in, to see how everything is related, how the search engine connects to the content, as does the calendar, and guest databases, membership, profiling, etc.
But we still have some time before Frontier 6 ships. Ask questions, ask for clarification, use the search engine (all the material on the developer site is indexed), and use any communication medium that you're comfortable with, except we ask that you not use the Frontier-users list because this is outside its charter. Once this stuff ships, that will the appropriate time to discuss it on Frontier-users. That's why we started the Frontier-server list to support the initial group of testers for Frontier 6. And, as far as UserLand is concerned, you may if you want use the Script Meridian list to ask questions about Frontier 6. If the members of SM decide that's OK, then go for it, otherwise use the Frontier-server list or the Discussion group.
That's it for now. I hope this makes sense. Thanks for listening!
Pointers
This article is on our Developer's website, with pointers to many of the documents referred to in this piece.
http://developers.userland.com/stories/updateForScriptMeridians
Finally, I know that a lot of the stuff in here has no background in the DaveNet flow. But there are important new ideas in here, and opportunities for working with other scripting communities, so I wanted to get the ideas out in order to foster more cooperation across operating systems and scripting environments.
We're still diggin!