The XML Files
By Jake Savin.
Good morning XML fans! 
Intro by Dave Winer. At UserLand we've always had a philosophy of no-lock-in for our users. We believe we make the best Web content management software, so we are happy to let our users move the content where ever they see fit. And we sleep better if they can back up their own work, and we're excited about the possible ways of flowing content from Manila sites into other formats such as PDF, eBooks, DocBooks, Wikis, or whatever.
Manila is a great groupware environment for working on Web publications, by opening it up for easy coversion to other formats, Manila can be used to write books, publish magazines or even flow news out to wireless devices.
Manila is the first Web application to be completely scriptable via XML-RPC and SOAP, now we're moving even more broadly.
With release of The XML Files tool for Radio, now you can flow content from a Manila site into a folder full of (you guessed it) XML files. Yeah it's a cute twist on the name of the TV show. Designed to be memorable.
Now I'll turn you over to Jake for the rest of the presentation.
What each folder contains
As a demo we downloaded this site to a local hard disk, ran The XML Files tool on the database, and it produced a folder of XML files. The whole folder is here.
At the top level of the folder, there are four sub-folders for the discussion group, pictures, templates and gems, and several top-level XML files.
The discussionGroup folder contains a folder of messages, and a calendar and stats file, both in XML. The pictures folder contains all the pictures from the Manila site. The templates folder contains the HTML templates used to render content in the Manila site. The gems folder contains XML descriptions of all the site's gem files, and a sub-folder which contains the gems themselves.
glossary.xml is the XMLization of the shortcuts table in the website, also known as the site's glossary.
hierarchy.xml contains the site hierarchy. homePageCalendar.xml is a calendar structure with the discussion group messages of all the home pages. members.xml is an XMLization of the membership database, with passwords removed.
navigator.xml contains the site's navigation links. siteprefs.xml is a very rich XML structure of all the site's preferences. Lots of information here. userPrefs.xml is an XML structure representing the schema for user preferences information.
Requirements
1. To use the XML Files tool, you must havea fully updated copy of Radio UserLand 7.0 or greater.
2. A Manila site.
How to download your Manila site
First, download The XML Files Tool here:
http://static.userland.com/sh7/gems/radiodiscuss/TheXmlFiles.root
Save it into the Tools sub-folder of the Radio UserLand folder, and launch Radio UserLand if it's not already running.
Radio 8.x users, bring the Radio application to the front. (On Windows, right-click the Radio icon in the system tray, and choose Open Radio.)
You'll see a new sub-menu named TheXmlFiles in the Tools menu.
Downloading and exporting your Manila site
In Radio UserLand, choose the Download Site and Export as XML... command from TheXmlFiles sub-menu of the Tools menu.
A dialog will appear, asking you to enter the URL of your Manila site -- enter the URL. After that, you'll be asked for your email address and password. Enter them and click OK.
You can watch the download/export progress in the About window. You will see a series of messages telling you what's happening in the process of exporting your site.
Once the download and export have completed, the folder containing your exported Manila website will open on your computer's desktop.
What's in the XML files
All of your site's textual content is in the messages sub-folder of the discussionGroup folder. Each message is contained in a single file, whose name is the message-number. Here's an example.
Discussion group message serialization
<msg> is the top-level element in a message file. It has one attribute, msgNum, whose value is the message number. Inside a <msg>, there are the following elements:
<authorEmail> -- The email address of the message's author.
<authorName> -- The name of the message's author.
<inResponseTo> -- The number of the message that this message is a response to, or 0.
<postTime> -- The date/time that the message was originally posted.
<lastUpdate> -- The date/time that the message was last edited.
<subject> -- The message subject.
<body> -- The text of the message itself. The body element has one attribute, type which has one of the following values: text/plain, text/x-outline-tabbed or text/x-opml.
<ctReads> -- The message's read count.
<responses> -- Contains any number of <response> elements. A <response> element contains the message number of a message which is a response to this message.
<alsoListedIn> -- Contains any number of <category> elements. A <category> element contains the name of a category that this message is listed in, such as mySiteStories, mySitePictures, or mySiteHomePages.
<image> -- Describes an image (gif or jpeg) stored in the site. An <image> element has six attributes: url -- the file:// URL of the exported image file, height/width -- the height and width of the image in pixels, shortcut -- the shortcut name for the image, src -- the image's URL on the Internet, and mimeType -- the image's MIME-Type. Here's an example of a message with an image. The actual image files are in the pictures sub-folder.
<enclosure> -- Describes an enclosure (of a Frontier object). Attributes are: url -- the file:// URL of the exported enclosure, type -- the Frontier type of the enclosure, and address -- the Frontier object database address for the enclosure. This element is present to support customized sites which contain enclosures of Frontier objects. Most Manila sites will not make use of the <enclosure> element.
Discussion group calendar
The calenadr.xml file in the discussionGroup sub-folder contains the discussion group's calendar.
<calendar> is the top-level element in a calendar.xml file. It contains any number of <year sub-elements.
<year> has one attribute, name, which is the year. A <year> element contains zero to twelve <month> sub-elements.
<month> has one attribute, name, which is the month number. A <month> element contains zero to 31 <day> sub-elements.
<day> has one attribute, name, which is the day. A <day> element contains any number of <msg> sub-elements.
A <msg> element has one attribute, msgNum, which is the number of a discussion group message. The character data enclosed in a <msg> element is the subject of the message.
Message stats
The stats.xml file in the discussionGroup sub-folder contains stats about messages in your site.
The top-level element in stats.xml is the <stats> element. Inside the <stats> element, there's a <rankings> sub-element, and possibly a <changes> sub-element.
The <rankings> sub-element contains a <msg> sub-element for each message in the site. A <msg> element has one attribute, msgNum, which is the number of the message. The character data enclosed in a <msg> element is a number, which is the number of reads for a given message.
The <changes> sub-element contains a <msg> sub-element for each message in the site. A <msg> element has one attribute, msgNum, which is the number of the message. The character data enclosed in a <msg> element is a date, which is the date/time that the message was last changed.
Membership data
The members.xml file contains information about the site's members. Here are the elements in members.xml:
<members> -- The top-level element in members.xml. The <members> element contains any number of <member> sub-elements.
A <member> element has a single attribute, email, which is the member's email address. A <member> element may contain any number of sub-elements.
Because the managing editor of a Manila site can specify the information they want to store about their members, a <member> element contains a nested structure of sub-elements which store the member's information.
All sub-elements of <member> are typed name-value pairs, where the name is the element's name, and the value is the data enclosed in the element. The type of a <member> sub-element is specified by the element's type attribute.
The type attribute is one of:
string -- A string of characters.
number -- An integer number.
float -- A non-integer number.
boolean -- A boolean with a value of true or false.
date -- A date as specified by RFC 822, expressed in Greenwich Mean Time.
struct -- A struct of values of any type.
array -- An array of values of any type, except struct or array. All sub-elements of an array are item elements -- array sub-elements do not have unique names.
Here are sub-elements of <member> which will be most common:
<bulletins> -- A struct containing two sub-elements: <enabled> -- a boolean specifying whether the user wants to recieve email bulletins, and <inHtml> -- a boolean specifying whether bulletins sent to the member will be formatted in HTML.
<discussionGroupInfo> -- A struct which in turn contains <discussionGroup>, another struct. <discussionGroup> contains a number, <highestMessageRead> which is the number of the highest message which has been read by this member.
<hitCount> -- A number, which is the number of pages that have been served to this member.
<lastVisit> -- A date, which is the date/time that the member last visited the site.
<personalInfo> -- A struct containing <name>, which is the member's name.
Keep in mind that while most membership records will contain only these elements, that the <member> element may contain very different information than what's described here.
Home page calendar
The home page calendar is exported to the homePageCalendar.xml file.
The top-level element in homePageCalendar.xml is <calendar>.
Like the discussion group calendar, there are <year>, <month> and <day> elements nested within the <calendar> element. All three elements have a single attribute, name, which is the year, month or day.
A <day> element contains the following:
For a home page day which is a message (as opposed to being news items-based), <day> contains a single <msg> element, which has one attribute, msgNum -- the number of the discussion group message which contains the home page content for the day in question.
For a home page day which is news items-based, the <day> element has an attribute, flNewsItems, whose value is true. Days which are news items-based do not contain a <msg> element.
Glossary (a.k.a. Shortcuts)
The glossary.xml file contains the site's glossary, also known as Shortcuts. The top-level element in glossary.xml is <glossary>.
A <glossary> element contains any number of <item> elements.
An <item> element contains some text for the glossary item, and has a single attribute, value, which is the HTML-text that is substituted whenever a rendered page contains the enquoted glossary item text. For most Manila sites, glossary items specify image tags or link-HTML which replaces the text enclosed in the <item> element, when the page is served.
Gems
The gems folder contains XML files describing each Gem file, and a sub-folder, files, which contains the Gem files themselves.
The Gem XML files have numeric names corresponding to the Gem number.
Each Gem's XML file has a top-level <gem> element.
A <gem> element has a single attribute, num, which is the Gem number. A <gem> element contains the following sub-elements:
<sourceUrl> -- The URL from which the Gem file was downloaded when downloading the site for export as XML.
<postTime> -- The date/time that the Gem file was posted.
<mimeType> -- The MIME-Type of the Gem file.
<ownerEmail> -- The email address of the member who posted the Gem.
<flDeleted> -- A boolean, true, or false, specifying whether or not the Gem file has been deleted.
Site prefs
The sitePrefs.xml file contains all of the Manila site's preferences. If you've ever edited a Manila site in Frontier or Radio UserLand, you'll recognize that the elements in sitePrefs.xml are from the site's #newsSite table. There are two additional sub-tables: urls (the site's #urls table) and prefs (the site's #prefs table).
The sitePrefs.xml file is serialized in the same way that membership data is serialized. Here's brief a summary:
At the top level, there's a <sitePrefs> element. The <sitePrefs> element contains a nested structure of typed name/value pairs, encoded as sub-elements of the <sitePrefs> element. (See the section on Membership data for an explanation of the serialization format.)
Other XML files
The following files are exported directly from the downloaded Manila site, with no change, since the data's original format was XML:
hierarchy.xml -- This file contains the XML from the Site Structure section of the Advanced Prefs page. It specifies how messages are mapped to URLs in the Manila site. See this page for details.
navigator.xml -- This file contains the XML from the Navigation section of the Advanced Prefs page. It specifies the links which are rendered by the {navigator} macro. See this page for details.
userPrefs.xml -- This file contains the XML from the Custom Prefs section of the Advanced Prefs page. It specifies the information that people enter when signing up as a member of the site, and data that a member can set via the member prefs page. See this page for details.
How to Update
You can use the Update TheXmlFiles.root command to get the latest features and fixes for the Tool. Here's how:
1) Bring the Radio application to the front. (Windows users right-click the Radio icon in the system tray, and choose Open Radio.)
2) Choose TheXmlFiles.root from the Window menu.
3) Choose the Update TheXmlFiles.root command from the Tools menu.
Change notes
5/13/02; 2:49:17 PM by JES -- Updated the Tool to work with Radio 8.x and Frontier.
|
