More servers, bigger servers, longer servers, and 10x of that!

As a rapid growing start-up we obviously keep a close eye on where and how the growth is occurring and year after year it keeps amazing us. We just published a press release that showcases some of the trends we’re seeing, but given how press releases are I thought it would be nice to go into a bit more detail and background behind the numbers.

Maybe the most surprising growth to some people is the size of the servers. EC2 now has 8 instance sizes so we decided to look at them as xlarge vs. smaller to make it easy, i.e., m1.xl, c1.xl, m2.xl, m2.2xl, m2.4xl vs. m1.small, c1.medium, and m1.large. For normal people, that’s machines with 4+ cores or 16+GB vs. smaller ones. In june 2009 just 12% of the fleet was on xlarge servers but by june 2010 this grew to 56%, yikes! Some people’s reaction to this is “I thought the cloud was about horizontal scalability” and while horizontal scalability may have been a requirement in 2006-2007 it no longer is. It’s a wonderful option, but not a requirement. There are a number of simple observations:

• If you’re starting out building a web app on one server, it’s often smart to postpone the hassle of scaling it horizontally for as long as you can stand it by growing to a single larger server first. This simple principle applies pretty broadly and in particular to the database tier. The catch is that you need to build things right so you don’t have to redo everything to scale horizontally, but that’s not rocket science anymore.
• If you need 200-300GB of memcache I’m sure you would rather operate 10 32GB servers over 200 1.7GB servers (the only server available in  ’06-’07). And these are not the largest such clusters that we see. (One catch with this calculation is that the large servers can’t handle 10x the request rate due to an approx 100k pkt/sec limit, see our load balancing whitepaper that details some of this.)
• More and more types of apps are being moved to the cloud, it’s not just scalable web apps and many of the “legacy apps” don’t scale horizontally, so the larger servers are very needed.

At the same time we’ve also seen more and more servers in the cloud. More customers, each launching more servers. At the end of June 2010 almost 3x more servers were running than at the end of June 2009. That’s more companies moving more computing to the cloud and others “simply” growing their cloud footprint. Many companies start with proof-of-concepts or peripheral projects to test the waters, and that’s definitely what we always recommend. Once they get comfortable and experience the benefits the projects grow and multiply.

Lastly servers run longer. We looked at the proportion of servers running full time, as defined by running for more than 30 days and as opposed to more transient servers. The upshot is that of the servers launched in June 2009 3.3% still ran 30 days later and for June 2010 this grew to 6.3%. This growth is significant (don’t be fooled by the small percentages: they’re to be expected given what we measure) because production servers tend to run full-time while tests and prototypes tend to run for a few hours to a few days. Of course this is not an exact science because some production server arrays grow and shrink on a daily basis and some test servers are left running all the time. But nevertheless there is a clear trend that shows a continued move of business critical computing to the cloud.

The upshot of all this is that in dollar terms the spending we’ve seen on cloud computing servers has more than ten-folded between June 2009 and June 2010. We’re living in interesting times where the way computing is consumed is changing dramatically, whether at the personal level with smartphones and iPads or at the server level in the cloud!