The cloud is your install script

Why people jump from shared to dedicated hosting

In the beginning, there was “shared” hosting. And that was all the hosting there was. A UNIX beard would give you an account on their big UNIX server, and set up Apache for hundreds of users to put their web pages, CGI scripts and PHP. The Apache server invented hosting, it still works the same today as it did 15 years ago, and it’s still everywhere.

But demanding customers eventually hit a problem – a shared hosting platform only has one Perl or PHP version on the system, and eventually the UNIX beard has to upgrade it to make new users happy. Older customers suddenly “feel the beard” behind their hosting and their stuff breaks. That makes them grumpy when the beard hadn’t needed to intefere for years. It happens all the time, and gets us a steady stream of new customers.

So for a few years, the solution was to go Dedicated – pick a big company who would buy dirt cheap servers and rent them, whole, by the month. You can ditch the UNIX beard who broke your site, hooray! He also probably backed the server up, helped solve your programming problems, and worried about the server’s up time, but you won’t notice that until your server goes wrong in 2-3 years time.

This is about where we started our business in 2002, and the very clever User-Mode Linux project. We used it to offer dedicated-style hosting at £15 per month, where much larger companies were charging 3 times that price. And because it was grungy and unpolished and you had to be a kernel wizard to use it, the big hosts stayed away from our business a good long time!

We even went into Dedicated when we had the money, because we wanted people to be able to grow beyond VMs (It also helped that Pete didn’t mind driving 400 mile round trip to London – a lot).

Why people want to jump from dedicated

So we’re really proud to still have some customers running Redhat 9 systems, doing old, important things for them. (But hey – you two guys – I’d check your firewall is nice and tight). And we’ve never had a “throwaway” attitude to dedicated servers. Lots of hosts tell you there’s nothing they can do for your dedicated server when it breaks, and here, have a new one.  We occasionally spent hours scraping a half-dead hard disc for a customer’s un-backed up data. Keeping the same system going is really important to us.

But that’s the reason customers run screaming from Dedicated servers: they break and fail and you suddenly miss the UNIX beard who might have got your files back from last week. So our dedicated servers have always been come with “added beard” when required, just to keep them going a little bit longer.

Help! Back to shared hosting again!

Sometimes customers add more dedicated servers, or hire their own beard. But plenty feel that there’s something new in “the cloud”, which means either:

  1. A “platform-as-a-service” cloud. Or our old friend shared hosting. It’s not as simple as the old days with Apache, but you are sharing web servers or databases with other customers, you’re back to relying on our old friend the UNIX beard. This time he has brought an complicated, proprietary set of hosting scripts with him. You can’t have them of course, but he promises that even though you’ll pay more, you’ll save by not employing a system administrator, or needing to worry about performance. Unlike most shared hosting, he may not even own or have access to the hardware he’s deploying for you.
  2. If you still like your own sovereignty, you might pick an “infrastructure-as-a-service” cloud. They provision virtual or dedicated servers whenever you ask for them, and route you IP addresses and run shared services for things like DNS. That’s what I used to call a “hosting company” and I reckon they’re selling, well, dedicated servers again.

There are no hosting services out there that don’t fall into one of these camps, but it’s a basic distinction as to how complicated your infrastructure is, and who can screw it up for you.

The UNIX beard running his university Apache server in 1995 was the original “platform as a service”. And anyone renting UK2’s cheap servers ten years ago, when they were the cheapest option, were “infrastructure as a service” pioneers.

But no no, my application is really in the cloud

You’ve got up-to-the minute live replicas of your important data? Backups too? Your install is scripted, and rehearsed such that when your hosting provider goes down, you can simply redeploy elsewhere at a moment’s notice? You use chef or puppet or cfengine and you never make any manual changes to your server, that you haven’t tested and pushed via a source control system? And you’ve got accounts with multiple service providers?

If so, well done, you must be 1 in 1000 hosting customers. You realise that the real cloud isn’t your hosting provider, it’s every hosting provider, and your ability to both use the best of them, and depend on none of them. The cloud is your install script.

In reality, most companies without a dedicated 24/7 operations team just aren’t ready. They bed their services down in a reliable hosting provider, have a critical process or two here and there, graft on a server for a new business function and pay for 24 hour support. Sometimes they can even run their entire business successfully for years on a server, or a single redundant pair, and nothing goes wrong!

And the reason isn’t because they’re slapdash, or lazy, or stuck in the past, it’s because most businesses aren’t Amazon and don’t ever need that kind of scale, and preparing for it reduces your overall reliability and up time.

Maciej Ceglowski, creator of antisocial bookmarking site pinboard.in said this about modern hosting practices in an interview a few months back:

I believe that relying on very basic and well-understood technologies at the architectural level forces you to save all your cleverness and new ideas for the actual app, where it can make a difference to users.

His site runs on a handful of very large servers, and managed to handle a huge influx of users (caused by the bungled announcements around the rival delicio.us bookmarking service being closed) without needing any fashionable just-in-time hosting provision.

And John Kozubik of rsync.net wrote a lament two years ago about the failure of a rival caused by overcomplicated architecture:

When you don’t have stars in your eyes, and aren’t preparing for your IPO filing and the “hockey sticking” of your business model, you can do sensible things like keep regular files on UFS2 filesystems on standalone FreeBSD systems.

This is, of course, laughable in the “real world”. You couldn’t possibly support thousands and thousands of customers around the globe, for nearly a decade, using such an infrastructure. Certainly not without regular interruption and failure.

Except when you can, I guess.

…and illustrates with examples of two of his servers that have been up for 350 and 950 days respectively.

BigV: proudly built for worst established practices

These are the people for whom we built BigV (not literally those two people, that would be a bit stalker-ish).

With BigV we might say it’s built for worst practices first, because we know they last longer than you might want. We know that relying on single servers for a while is more reliable than trying to build for Google-size overnight. We want to make it easy to throw up a server, back it up, back out of configuration mistakes and “push the walls out” when the server gets too small for the load. Right now you can go up to 120GiB RAM and 40TiB disc in a single server, and after the beta we’re not stopping there.

We can do that because we’re a lean company with lots of nice customers, not a huge number of staff, and we have plenty to drop on HP kit. HP do some very nice kit, and we want you to be able to build monster machines, and not be stuck having to glue together 8 or 16GiB machines with a cheaper hosting company. We want you to be able to use private VLANs, SSD storage, huge amounts of RAM, arbitrary disc snapshots, and all the toys that used to need expensive dedicated servers.

It’s not that you can’t build a minute-by-minute scalable cloud with lots of CPUs if you need them – you can! Look! But we recognise that most users won’t ever need to, and so we’re trying to replicate that “zen garden” isolated feel of a Dedicated server, without any of the overheads. We’re still doing the “big cloud” stuff, but looking at it from another angle: that you’ll start your site small and simple, and only maybe need to grow it big and complicated.

Our beta is still open for signups, and we’re both expanding our cluster and accelerating the pace at which we send out V-Keys. We’re looking forward to seeing what our customers build when they can shake off the vanity of scaling before they need to, and deliver their sites with a single cloud hosting company they trust.