Part of the scripting.com web server. San Francisco CA USA. 9/4/96.

---

Serving directly from the Object DB with WebSTAR

For Frontier magicians only -- Now you can serve directly from Frontier's object database. It's a performance and power breakthrough.

By Dave Winer, dwiner@well.com, 6/28/96

Background

We believe the Macintosh file system is a major performance bottleneck for high-traffic websites served from a Mac. Earlier in June, I posted the results of a performance benchmark that indicated that if we could set it up so that web text was served directly from Frontier's object database, performance might improve by as much as 7-to-1.

That was the green light that we needed, so in collaboration with Chuck Shotton, Doug Baron and I have set up a new system for deploying websites with Frontier to be served directly by WebSTAR. There's no script execution in this system, and no Apple Event communication. So there should be a performance improvement over earlier methods which involved both script execution and Apple Event messages.

Try it out

The new server software is running on this server. I've loaded a copy of the DaveNet website into the object database server. All the gifs referenced by this site are also stored in the object database. Here's the starting point:

http://ws3.scripting.com/davenet.default.odb

If you want to compare performance, check against the "official" file system based version of this websites:

http://ws3.scripting.com/davenet/default.html

How it works

There's a new environment, called WebSTAR.root. It's a Frontier object database, but it's mostly empty. This means that there are no suites, no glue scripts, an empty menu bar.

It's refreshing, a clean start! But it can be awkward to work in this environment. If you aren't already a Frontier geek, stay tuned, it'll get easier. If you have to have a real need for higher performance from your web server, or to want to understand the technology, go for it.

The bottom line -- this technology is brand-new, and the docs are sparse.

How to Install

Download the WebSTAR.root folder by clicking on the following line:

ftp://ftp.scripting.com/userland/webstar.root.sit.hqx

The download includes a new WebSTAR plug-in, odb-plugin. The plug-in requires WebSTAR 1.3fc2 or greater. Quit WebSTAR on your server, copy this file to the plug-ins sub-folder of your WebSTAR folder, and relaunch WebSTAR. You should see a message in WebSTAR's log window indicating that it was successful at opening WebSTAR.root.

Working on WebSTAR.root

There are four items at the top level of WebSTAR.root:

1. odb -- a table containing the website to be served. As it ships, it contains the material in the davenet, docserver and gifs folders on www.scripting.com.

2. odbScripts -- a table containing scripts that load root.odb from folders. This is just sample code. How the root.odb table gets populated is entirely up to the sysop. The scripts in this table load up the sample configuration for WebSTAR.root from the html and gif files stored on one of my server machines.

3. readme -- a wptext object.

4. system -- a virgin system table, as created by the New command in Frontier's file menu.

The central script in the odbScripts table is odbScripts.loadSite. It takes two parameters, the path to a folder on your LAN and the address of a table in Frontier's object database. It builds a database hierarchy that reflects the hiearchy in the file system.

odbScripts.dwloader is a custom script that I wrote to load my websites and then install the resulting WebSTAR.root on the server machine. You can use it as an example of a working installation, or modify it to suit your needs. Regardless, you should study the script to see how the system works.

URL patching

A URL in this system looks like this:

http://www.yourserver.com/xxx.yyy.zzz.odb

The odb extension is the key that tells WebSTAR that this is a reference into the object database. This URL refers to the object named zzz contained in the table root.odb.xxx.yyy.

As the loadSite script runs, it patches URLs contained in HTML files. The patches are specified in an outline at odbScripts.urlpatchoutline. Each top-level headline in this outline contains a URL prefix to be searched for, its only subhead contains the replacement string. In my setup, the first conversion specifies that all pointers into the DaveNet website should be converted to pointers into the DaveNet ODB site. My outline also converts all references to gifs and into the DocServer website.

Two examples:

1. http://www.scripting.com/davenet/default.html becomes http://www.scripting.com/davenet.default.odb.

2. http://www.scripting.com/gifs/cactus.gif becomes http://www.scripting.com/gifs.cactus.odb.

Any URLs that do not begin with one of the prefixes are left untouched by the patcher script.

The structure of a website, Futures

For security reasons, WebSTAR will only serve out of the root.odb table. This allows you to store scripts and private data that cannot be accessed thru the web.

Each folder becomes a table. Each file is also a table containing at least two cells. The "data" cell contains the contents of the data fork of the file. If it's a HTML file, the URLs are patched by the loading process, as described above. The second cell is called "mime" and it specifies the mime time for the data.

In the future, these tables will contain other attributes, and the architecture will be open, it will be possible for developers to add their own attributes or structures of attributes to objects served thru HTTP.

In the first release, we store all data as ram-based strings. This will change when Frontier 4.1 is released, it can manage disk-based strings as well, trading off a bit of performance (a small amount, we think) for a more efficient memory model.

New version of plug-in, Fri, Jul 5, 1996 at 12:20:45 PM

From Chuck Shotton, cshotton@biap.com: "Here's a new version of the odb-plugin. The significant new feature in this version is support for the if-modified-since request header. Clients and servers use this header to determine whether or not a file in a client's cache matches the file that the server has. If they match, the server tells the client the file isn't modified instead of having to retransmit all of the data again.

"This will be a huge boost in the performance of the odb-plugin because it means that the server will only send ODB data to the client once. As long as the current data is on the client, the server never needs to retransmit blobs of data out of the ODB. So now not only do we avoid the file system hit, we avoid the network hit as well.

"Since the current ODB Engine library allows only read-only access, I use the modification date of the root file as the determining factor in deciding whether or not to tell a client an object has been modified. When/if we allow updates to the root file, that logic will have to change, but it's internal to the plug-in so no impact on users."

You can download the new version of the plug-in from ftp.scripting.com.

Pointers

• See Mac Web Server Performance Tests for the results of the earlier performance benchmarks.

This page was last built with Frontier on a Macintosh on Wed, Sep 4, 1996 at 11:55:39 AM. Thanks for checking it out! Dave