Monday, August 9, 1999 by Dave Winer.
Late last week, rain! Summer rain is so rare here. Big New Orleans thunder clouds in our little desert. In its aftermath the air is left bright, as if all the colors were turned up, like seeing with new eyes."Everything's so pretty!" I said so many times.
As I was driving around the Bay Area this weekend, I thought about the Deep Linking issue raised in a lawsuit by Universal Studios, and reported by Wired News last week:
What is a deep link? It's such a new term, there is no accepted definition. So I'm going to offer one. A deep link is a publicly accessible HTML "anchor" tag that points to an off-site web page that is not the home page of the site being pointed to.
I am fairly sure that Wired will not sue me for linking to their story. In fact, I imagine that they are pleased to have their story pointed to because the page contains several advertisments that they make money from.
Their whole purpose, from a business standpoint, is to get eyeballs to see those ads. Further, the page contains links to other Wired stories, and to the Wired home page, if you want to know more about them, or even bookmark them.
Universal's website is different from Wired's. Among other things, they use their site to serve movie trailers, short segments of video that promote their movies. It's a form of advertising. But they are huge files, often several megabytes, and serving them costs a lot of money because you need really big pipes to carry all that video.
Enter Jean-Pierre Bazinet, the owner of the Movie-List website, which he calls "the net's most updated movie trailer site":
Movie-list is a very simple weblog, a news oriented website. People who want to keep up on the latest movies, or research older ones, tune in to Movie-List. He filters out the corporate stuff, and just shows you the product (this is what Universal objects to).
Bazinet was pointing to the Universal trailers directly, without pointing to the Universal home page, or even the page with the description for each movie. Universal says this is unfair and illegal.
It's difficult to understand why Universal is objecting, since the trailers sell tickets and video rentals of their movies. However, the trailers are their intellectual property, so they certainly have the right to raise the question. And it's an interesting question because there is a technical solution, this does not have to be resolved thru a lawsuit.
In the early days of the web the Robot Exclusion Protocol became the accepted way for a website to declare its willingness to be scanned by a search engine or other kinds of web "crawlers."
If you don't mind being crawled, you don't have to have the robot file, robots.txt. But if you have one, all robots are supposed to read it and and thru the honor system, respect it. If they say they don't want to be crawled, then don't.
I propose a file called deepLinks.txt. It says whether or not links are permitted into the website, and further, if linking is permitted, it says which sub-directories may be linked to.
It's a clear statement of intention -- anyone linking into a not-permitted location is clearly wrong. Further, the policy could be implemented by software running on your server. With a few technical tricks, Universal could make it impossible for Movie-List users to watch the trailers. But the deepLinks.txt file would be the professional and respectful way to let the outside world know that you don't want their flow.
If you don't have a deepLinks.txt file, then any and all linking is permitted. This would allow the web, by default, to retain its current open-linking tradition, which is essential if the web is to retain its charm.