I've spent the last week wrestling with my servers, dividing up what they do, factoring things so that maybe I can get the performance where it needs to be. It's one of the reasons I was exploring the possibility of using S3 to store my static sites (I gave up, couldn't make it work). 

So now things are running in the new configuration, or I should say -- limping along in the new configuration. There is always lots of breakage when I do this kind of surgery. While doing it I wish I could integrate DNS, generating Apache config files and serving into one master outline, where I make a change and the new configuration automatically percolates where it needs to. That, and it would be great to have one server with infinite CPU, disk and net bandwidth, so I'd never have to logically split a server across multiple machines.  

Maybe I'll get all this in time for next Christmas!  

Anyway... 

I think the mistake I made in the last half of 2010 is that I got too Dropbox-happy. 

I built too many systems on it, without understanding that it could be brought to its knees, begging for less to do, all the while causing every other application on the server to be starved for CPU cycles.  

When I saw how much work Dropbox was doing just to keep current in a relatively stable folder, I realized I needed to lower my dependence on it. It really wasn't doing that much for me, but it was taking up a very large portion of the CPU cycles on every one of my machines. (Only a few small text files were changing every few minutes, the really big stuff came once a night when the servers backed up).  

Also it uses a huge amount of disk space. The cache is enormous. I figure it uses at least 1.5 times the size of the folder. 

Now, Dropbox is a miracle product. Last year I called it an "idea factory." It gets you thinking about great things could be with a simpler architecture, and closer to the ideal of one big server without boundaries. It's a very easy-to-understand queueing system, and when you add XML or JSON to the mix, you get the magic that comes at the intersection of simplicity and power. But that's the virtuality of it, the reality is that it uses a lot of resources, so you have to use it carefully. 

I'm interested in finding out what other people are learning about Dropbox.