Home > Archive >  2009 >  August >  19

How to fix URL-shorteners

Wednesday, August 19, 2009 by Dave Winer.

First a few notes as a preamble. Permalink to this paragraph

1. URL-shorteners are bad for the Internet. They centralize linking, and make it more fragile, and more controllable. Wait till the Chinese govt finds out about them.  Permalink to this paragraph

2. When bit.ly breaks, it will be an outage that may be bigger than Twitter going down. Not only do we lose the present, but we lose the past too. One big URL shortener that dominates the others is itself a dangerous thing. Permalink to this paragraph

3. Twitter could and should obviate the need for URL-shorteners. Yes I know SMS messages are limited to 160 chars. So shorten the URLs at the SMS gateway and leave them long for communication over pathways that are not so limited. Any engineer could see this obvious solution.  Permalink to this paragraph

4. For now URL-shorteners are a fact of life. Permalink to this paragraph

End of preamble. Now to what is needed in URL-shorteners to work around the various issues they present. Permalink to this paragraph

It's not so different from the problem with Feedburner, and the solution they used, and implemented quickly once it was known. Permalink to this paragraph

1. CNAMEs. It must be possible for the user to own and control the domain his or her URLs live at. Technically, this means I register the domain name, and map a sub-domain to the URL-shortener site with a CNAME record. Anyone who knows how to use Godaddy can do it. I would be happy to write a howto that explains.  Permalink to this paragraph

2. Shared data. The URL-shortener and the user share a space where the data is stored. Joe Moreno at Adjix, who I have been working with, has figured out how to do it on Amazon S3. I have mapped a domain to an S3 bucket, and given his software permission to write to that bucket. Here's the key point. At any time I can revoke the permission and my URLs still work. Or Adjix could disappear, and the shortened URLs would still work. With this method the only way there is linkrot is if S3 goes down.  Permalink to this paragraph

Here's a URL that links to a Flickr picture: Permalink to this paragraph

http://tmp.loose.ly/jmxe  Permalink to this paragraph

Obviously the sub-domain, tmp.loose.ly, is temporary. But if you're a techie, I encourage you to do a DNS lookup on tmp.loose.ly. You'll see it's a CNAME to s3.amazonaws.com. And get the contents of the file to see how it works. It's static. Yet it still gathers statistics. Yes, it's unusual. That's why Joe was the only one to crack this nut. He's a creative guy. A picture named sidesmiley.gif Permalink to this paragraph

It's such a clean implementation that if I decide later to move the files to an Apache server on Linux, no problem.  Permalink to this paragraph

I think basically Adjix has solved all the problems with URL-shorteners. I hope other engineers poke at this and verify my conclusion or disprove them. Permalink to this paragraph

   Recent stories

A picture named dave.jpgDave Winer, 54, pioneered the development of weblogs, syndication (RSS), podcasting, outlining, and web content management software; former contributing editor at Wired Magazine, research fellow at Harvard Law School, entrepreneur, and investor in web media companies. A native New Yorker, he received a Master's in Computer Science from the University of Wisconsin, a Bachelor's in Mathematics from Tulane University and currently lives in Berkeley, California.

"The protoblogger." - NY Times.

"The father of modern-day content distribution." - PC World.

One of BusinessWeek's 25 Most Influential People on the Web.

"Helped popularize blogging, podcasting and RSS." - Time.

"The father of blogging and RSS." - BBC.

"RSS was born in 1997 out of the confluence of Dave Winer's 'Really Simple Syndication' technology, used to push out blog updates, and Netscape's 'Rich Site Summary', which allowed users to create custom Netscape home pages with regularly updated data flows." - Tim O'Reilly.


Dave Winer Mailto icon

© Copyright 1994-2009 Dave Winer Mailto icon.

Last update: 8/19/2009; 11:50:49 PM Pacific. "It's even worse than it appears."

Click here to view blogs commenting on  RSS 2.0 feed.