Matthew Bloch
Aug 04

by 
in  Bytemark Cloud Servers

What is BigV? The Unveiling

Note: Since the original publication, BigV has been renamed Bytemark Cloud and offers many new features since the platform launched in 2011.

BigV is Bytemark’s very own cloud hosting platform, built in-house. It builds on virtual machine (VM) technology to provide hosting that is both flexible and resilient.

  • Flexibility: You can change your servers’ RAM or disc space instantly. Plus, only pay for what you use. If you need to fire up a server for a few hours, that’s all you’ll be billed for.
  • Resilience: We have the capability to shift customers’ servers around our cluster of hardware if we think any of it is going to fail, needs maintenance, or an upgrade.

What Inspired BigV?

You might be shocked to hear that Bytemark’s existing VM product is a set of Ruby, shell & Perl scripts. It was originally written in about eight weeks back in 2002. The scripts got passed around in “maintenance mode” for many years and have survived various attempts to “rewrite them properly”.

Despite their simple beginnings, these VMs do a lot. They offer long uptimes, permanent discs and easy upgrades. This is still what we wanted, we just wanted to do it better.

Features Added:

When we began building BigV, the six most important things we wanted to add were:

  1. Reliable live migration – so we can upgrade our hardware without the laborious work of emailing customers and spoiling their up times.
  2. VM snapshots – so a customer can “checkpoint” their whole system before a major upgrade, and back out if it goes wrong.
  3. Access to all of KVM’s great features – graphical consoles,  installations from CD, direct network access, and anything else they’d be able to do if they had the server in front of them.
  4. A handy tool for provisioning, server upgrades and maintenance – a uniform interface to the software, rather than the 1980s text console you get at the moment (I should say “as well as”).
  5. Really flexible storage – so that servers could use terabytes, not just a few tens of gigabytes.
  6. A sane software development process and test rig – so we could add features to our live system without errors.

Architecture

The promise that “it’s a magic cloud, and you don’t need to worry” is not helpful in understanding how BigV works. So we thought you may be interested to hear about our architecture in more detail.

In creating BigV, we turned our simple system into a distributed one. To do this we used three types of server instead of one:

  1. Brains – these servers hold the database of all virtual machines in a BigV cluster, and run the gateway for customers to issue requests for servers.
  2. Heads – packed full of CPU cores and memory, and run the KVM processes, aka virtual machines.  They don’t have any storage.
  3. Tails – high-spec servers, but with RAID cards, a normal amount of RAM, and lots of directly-attached discs which can be hot-swapped.

What do brains, heads and tails do?

The heads and tails are always connected to the brains, and one of the brains takes on the role of the master brain.  That’s the one that keeps a complete list of every virtual machine and disc in the cluster, and that’s what you (the customer) talk to when you ask to provision a new VM.

The heads and tails are also connected to a 10-gigabit storage network so that the KVM processes can talk to their discs really quickly.

The brain can decide to move either virtual machines or discs between any pair of heads and tails, without having to reboot affected systems.  So that gives us our hardware nirvana –no live customer system need ever be tied to a piece of hardware again.

Security

Through the development and testing stage of BigV, we decided to add a final security feature before launching. V-keys. These will allow us to use two-factor authentication.

One-factor authentication usually means you supply a secret to prove who you are. A username, password and certificate are all just data, and have to be stored in a file. If a fraudster can quietly copy that file they were stored in, they can pretend to be you straight away.

With a physical token like our V-Key, you can’t copy it. You will need a V-Key to use BigV (at least for the early stages of the beta), and only the person with physical access to it can use your account.

V-Keys are easy to use. You retain a username and password, but our high-security accounts (which will likely be all of them, to start with) will require you plug in your key and press its button. It acts as a keyboard, and “types” a one-off password to our servers every time you activate it.

Our First BigV Cluster

So with the beta test of our first BigV cluster live, testers are able to do all of this:

  • create virtual machines up to 120GiB;
  • attach as many discs as you want (up to 500GiB normal, or 50GiB fast storage);
  • use IPv4 and IPv6 (one of each per server at the moment);
  • VGA & serial console access;
  • server installation from CD-ROMs;
  • online memory shrinking (but not growing); and
  • offline disc and memory addition.

Take a look at our up to date Cloud Hosting page to see how much the platform has developed since this post was written back in 2011!

Matthew Bloch