AMD’s new fight, and why we love Bulldozer

The commentary around Bulldozer, AMD’s latest processor line, is that it’s disappointing, a catastrophe, absolutely positively awful and so on, for miles and miles. And it’s a shame for AMD that Ars, Extreme Tech and the usual suspects have no imagination beyond their benchmarks when it comes to judging processors.

The numbers don’t lie of course – the benchmarks show that Bulldozer’s best is slower than Intel’s, by a long way, and most of those benchmarks care about absolute amount of data processed, numbers crunched and so on. Ars Technica concludes:

AMD compromised single-threaded performance in order to allow Bulldozer to run more threads concurrently, and that trade-off simply hasn’t been worth it.

But everyone’s performance tests make an assumption: that the processor is working for one person at a time, and that person wants it to crunch through as many numbers as possible for their benefit. Those numbers might be rendering frames of Battlefield 3, editing a huge photo or ripping a CD. Benchmarks sum it up the same way – however many ways a single job can be split to use a processor’s cores, they’re only interested in who gets to the finish line fastest.

And Intel still wins, I get it. Even AMD admit in a recent interview that their older processors still perform better for a lot of people:

We understand our customers make purchase decisions based on how they use their PCs, and in many cases our AMD Phenom™ II processors are a great (purchase).

They are struggling to pitch themselves against Intel to the gamers, power-hungry desktop PC users and the benchmark sites. They know they can’t get the same excitable press any more, not for years and another generation of processor. That’s probably why they’re temporarily giving up this fight on pure brawn, laying off hundreds in their PR and marketing departments a couple of weeks ago.

But instead of doing one thing for one person, let’s instead assume that a processor is under siege from 300 warring factions, all wanting to run separate and unpredictable work loads. The benchmark that interests us is: assuming we are running 299 “hostile” jobs, how quickly does that 300th job complete? If we vary those 299 jobs in nature, how reliable is the performance of that 300th? The time of that one lonely, slow, job is what I’m interested in.

For BigV where we are running a massively “multi-tenant” system, we really are planning to put in the order of 300 different customers on a single server. It’s far more important to have a reliable average performance for a virtual machine than the absolute fastest possible performance. I’ve not (yet) seen any benchmarks that test in this way, but it seems to us that more separate hardware cores must achieve this goal better than a single software-switched core.

If that weren’t the case I find it hard to understand investment in massively parallel server systems like the 768-core Atom server or super low-power 480-core ARM systems. Big multi-tenant systems don’t need to be the fastest, they just need to perform consistently.

For BigV, we just want more cores. The speed is almost irrelevant. While AMD’s performance is within the same ball park as Intel’s, they’ll work out very nicely for us virtual machine-mongers. And the low price of Bulldozer chips is just icing – a 32-core, 128GB system is extremely affordable and helps us keep our pricing to customers low.

AMD are gearing up for a different fight, and they don’t need press from the benchmark sites to prove them the winners. So while my gaming PC will stick with a Phenom for a while yet, BigV is going to be using an awful lot of Bulldozer chips in the near future.

The cloud is your install script

Why people jump from shared to dedicated hosting

In the beginning, there was “shared” hosting. And that was all the hosting there was. A UNIX beard would give you an account on their big UNIX server, and set up Apache for hundreds of users to put their web pages, CGI scripts and PHP. The Apache server invented hosting, it still works the same today as it did 15 years ago, and it’s still everywhere.

But demanding customers eventually hit a problem – a shared hosting platform only has one Perl or PHP version on the system, and eventually the UNIX beard has to upgrade it to make new users happy. Older customers suddenly “feel the beard” behind their hosting and their stuff breaks. That makes them grumpy when the beard hadn’t needed to intefere for years. It happens all the time, and gets us a steady stream of new customers.

So for a few years, the solution was to go Dedicated – pick a big company who would buy dirt cheap servers and rent them, whole, by the month. You can ditch the UNIX beard who broke your site, hooray! He also probably backed the server up, helped solve your programming problems, and worried about the server’s up time, but you won’t notice that until your server goes wrong in 2-3 years time.

This is about where we started our business in 2002, and the very clever User-Mode Linux project. We used it to offer dedicated-style hosting at £15 per month, where much larger companies were charging 3 times that price. And because it was grungy and unpolished and you had to be a kernel wizard to use it, the big hosts stayed away from our business a good long time!

We even went into Dedicated when we had the money, because we wanted people to be able to grow beyond VMs (It also helped that Pete didn’t mind driving 400 mile round trip to London – a lot).

Why people want to jump from dedicated

So we’re really proud to still have some customers running Redhat 9 systems, doing old, important things for them. (But hey – you two guys – I’d check your firewall is nice and tight). And we’ve never had a “throwaway” attitude to dedicated servers. Lots of hosts tell you there’s nothing they can do for your dedicated server when it breaks, and here, have a new one.  We occasionally spent hours scraping a half-dead hard disc for a customer’s un-backed up data. Keeping the same system going is really important to us.

But that’s the reason customers run screaming from Dedicated servers: they break and fail and you suddenly miss the UNIX beard who might have got your files back from last week. So our dedicated servers have always been come with “added beard” when required, just to keep them going a little bit longer.

Help! Back to shared hosting again!

Sometimes customers add more dedicated servers, or hire their own beard. But plenty feel that there’s something new in “the cloud”, which means either:

  1. A “platform-as-a-service” cloud. Or our old friend shared hosting. It’s not as simple as the old days with Apache, but you are sharing web servers or databases with other customers, you’re back to relying on our old friend the UNIX beard. This time he has brought an complicated, proprietary set of hosting scripts with him. You can’t have them of course, but he promises that even though you’ll pay more, you’ll save by not employing a system administrator, or needing to worry about performance. Unlike most shared hosting, he may not even own or have access to the hardware he’s deploying for you.
  2. If you still like your own sovereignty, you might pick an “infrastructure-as-a-service” cloud. They provision virtual or dedicated servers whenever you ask for them, and route you IP addresses and run shared services for things like DNS. That’s what I used to call a “hosting company” and I reckon they’re selling, well, dedicated servers again.

There are no hosting services out there that don’t fall into one of these camps, but it’s a basic distinction as to how complicated your infrastructure is, and who can screw it up for you.

The UNIX beard running his university Apache server in 1995 was the original “platform as a service”. And anyone renting UK2′s cheap servers ten years ago, when they were the cheapest option, were “infrastructure as a service” pioneers.

But no no, my application is really in the cloud

You’ve got up-to-the minute live replicas of your important data? Backups too? Your install is scripted, and rehearsed such that when your hosting provider goes down, you can simply redeploy elsewhere at a moment’s notice? You use chef or puppet or cfengine and you never make any manual changes to your server, that you haven’t tested and pushed via a source control system? And you’ve got accounts with multiple service providers?

If so, well done, you must be 1 in 1000 hosting customers. You realise that the real cloud isn’t your hosting provider, it’s every hosting provider, and your ability to both use the best of them, and depend on none of them. The cloud is your install script.

In reality, most companies without a dedicated 24/7 operations team just aren’t ready. They bed their services down in a reliable hosting provider, have a critical process or two here and there, graft on a server for a new business function and pay for 24 hour support. Sometimes they can even run their entire business successfully for years on a server, or a single redundant pair, and nothing goes wrong!

And the reason isn’t because they’re slapdash, or lazy, or stuck in the past, it’s because most businesses aren’t Amazon and don’t ever need that kind of scale, and preparing for it reduces your overall reliability and up time.

Maciej Ceglowski, creator of antisocial bookmarking site pinboard.in said this about modern hosting practices in an interview a few months back:

I believe that relying on very basic and well-understood technologies at the architectural level forces you to save all your cleverness and new ideas for the actual app, where it can make a difference to users.

His site runs on a handful of very large servers, and managed to handle a huge influx of users (caused by the bungled announcements around the rival delicio.us bookmarking service being closed) without needing any fashionable just-in-time hosting provision.

And John Kozubik of rsync.net wrote a lament two years ago about the failure of a rival caused by overcomplicated architecture:

When you don’t have stars in your eyes, and aren’t preparing for your IPO filing and the “hockey sticking” of your business model, you can do sensible things like keep regular files on UFS2 filesystems on standalone FreeBSD systems.

This is, of course, laughable in the “real world”. You couldn’t possibly support thousands and thousands of customers around the globe, for nearly a decade, using such an infrastructure. Certainly not without regular interruption and failure.

Except when you can, I guess.

…and illustrates with examples of two of his servers that have been up for 350 and 950 days respectively.

BigV: proudly built for worst established practices

These are the people for whom we built BigV (not literally those two people, that would be a bit stalker-ish).

With BigV we might say it’s built for worst practices first, because we know they last longer than you might want. We know that relying on single servers for a while is more reliable than trying to build for Google-size overnight. We want to make it easy to throw up a server, back it up, back out of configuration mistakes and “push the walls out” when the server gets too small for the load. Right now you can go up to 120GiB RAM and 40TiB disc in a single server, and after the beta we’re not stopping there.

We can do that because we’re a lean company with lots of nice customers, not a huge number of staff, and we have plenty to drop on HP kit. HP do some very nice kit, and we want you to be able to build monster machines, and not be stuck having to glue together 8 or 16GiB machines with a cheaper hosting company. We want you to be able to use private VLANs, SSD storage, huge amounts of RAM, arbitrary disc snapshots, and all the toys that used to need expensive dedicated servers.

It’s not that you can’t build a minute-by-minute scalable cloud with lots of CPUs if you need them – you can! Look! But we recognise that most users won’t ever need to, and so we’re trying to replicate that “zen garden” isolated feel of a Dedicated server, without any of the overheads. We’re still doing the “big cloud” stuff, but looking at it from another angle: that you’ll start your site small and simple, and only maybe need to grow it big and complicated.

Our beta is still open for signups, and we’re both expanding our cluster and accelerating the pace at which we send out V-Keys. We’re looking forward to seeing what our customers build when they can shake off the vanity of scaling before they need to, and deliver their sites with a single cloud hosting company they trust.

Bytemark hosting the Open Rights Group

For the foreseeable future, Bytemark will be helping to sponsor the Open Rights Group with as much hosting as they need to support their many and important campaigns for civil liberties in the UK.

The Open Rights Group have been campaigning for 7 years, funded by 700 seed supporters who joined up under the pledge: “I will create a standing order of 5 pounds per month to support an organisation that will campaign for digital rights in the UK but only if 1,000 other people will too.” (they had some leeway in that target!)  Today the Open Rights Group have full-time staff and make a major contribution to the national debate on digital rights, putting journalists, campaigners and policy makers in touch with one another.

As a leading hosting company with a direct interest in the digital economy, we at Bytemark know that the Open Rights Group‘s campaigning works in our interests, and those of our customers. They led the charge against the the pointless Digital Economy Act, and exposed its many holes. They maintain constant vigilance against government snooping, and they continually oppose the entertainment industries’ demands to shackle our digital futures in the name of preserving profit. Their work is continual, important to lots of UK internet users who’ve never heard of them, and supported only by donations.

We’re glad to be associated with the Open Rights Group through our contribution, and look forward to working with their technical team.

Pardon me, some redeployment in progress

We’ve always been more of a… console-y kind of company. But I’m finally hauling our 20-ish web applications up to a newer, simpler deployment. In fact if you’re reading and you lurve web application deployment, you can become our next systems administrator. If you prefer programming we’re also hiring a Ruby programmer.

But that’s by the by – I had a blogging renaissance in mind, and also needed to haul the old site off a hateful blogging program that ate 2-3 gigabytes of RAM every time I pressed “post”.  I might have posted more often, but it took a month each time for the swap storm to die down.

OK OK not quite true. But WordPress conquers everything, and does the business. What’s not quite done is our new design, but it’ll look a bit like this:

Apart from the bland default theme, does anything look broken before I carry on?

 

On V-Keys, not trusting your security vendor

I know, I know, we’re still polishing BigV and it’s nearly August. Thanks to the folks who came and saw us last Friday – we had a marvellous party and briefly showed off some of BigV’s tricks.

While we’ve been hammering away at stability, documentation and bug fixes, I wanted to explain why we’re adding one last security feature.

BigV accounts will have access to shedloads of computing power – even in the beta, that’s hundreds of gigabytes of memory, and multi-gigabit network connections. And our beta is free-of-charge, so our users aren’t limited by how much money they might want to spend. Because of that, we didn’t want to allow our users accounts to be compromised - from day one. So we bought a load of these:

vkey shot

They are V-Keys and give us the magic of two-factor authentication.

One-factor authentication usually means you supply a secret to prove who you are. A username, password and certificate are all just data, and have to be stored in a file. If a bad guy can quietly copy that file they were stored in, he can pretend to be you straight away. With a physical token like our V-Key, you can’t copy it. You will need a V-Key to use BigV (at least for the early stages of the beta), and an attacker would need to physically steal it from you to "borrow" your account. 

V-Keys are easy to use. You retain a username and password, but our high-security accounts (which will likely be all of them, to start with) will require you plug in your key and press its button. It acts like a keyboard, and "types" a one-off password to our servers every time you activate it. The password proves to the server that you have a particular V-Key in your possession. The key can’t be copied, even if you’ve plugged it into a computer controlled by a bad guy. Each password can only be used once.

If you know about computer security, you’ll recognise these as Yubikeys with a sticker on. But they’re not factory-fresh. We’ve reprogrammed them to only work with BigV, because they’re more secure that way.

Yubikeys are a great product, but they trade off the best possible security they could offer for convenience. Instead of demanding that their customers manage their own keys, Yubico program them for you, so that they can verify them later over the internet, and save you the bother of setting up your own servers. That means Yubico keep a giant database of every key they’ve ever manufactured. They also have the albatross round their necks of keeping the giant database online at all times; if it goes down, none of their customers can log in. Worse, they also need to keep it secure from hackers. The more successful Yubico becomes, the more hackers will be interested in their database. If a hacker got a copy of Yubico’s database, he could fake any Yubikey that was ever issued.

It seems likely that when RSA, the market leaders in security tokens were broken into (only a few months ago), their own database of keys might have been stolen. RSA must have had a pretty good reason to offer to replace any token with a new one, but have still not publically admitted the depth of the breach – as a customer you have to take their word for it that they have responded in your best interests.

So if veteran security vendors can’t keep their customers’ secrets, I’m not going to trust anyone else to do the same.  Yubico’s servers are as big a target, and eventually we would start to worry that another company has the ability to compromise our customers’ hosting.

I’m not bashing Yubico – their keys are easier to sell when they do the programming, and it’s still more secure than a simple password. But the reason Yubikeys are a better product than RSA’s SecureID is that 1) Yubico tell you exactly how the keys work, and 2) they give you tools to reprogram them. We don’t need to trust the people that sold them to us as much. And that is something security vendors should compete on.

Posted in Uncategorized | 1 Reply

Small holiday hold-up

I said June, I just didn’t say when in June. Clever, slippery me! Apologies, between holidays and illness we didn’t make it for the start of the month, but I am hoping we’ll get our first invites out by the end of the month, and the proper site up. We are going to leak invites out a few at a time, so we can watch the first few of you like lab rats.

When that happens you’ll also know our final price list; but we’ll still keep some capabilities under our hats for a little bit longer.

Here are the features that are working right now:

  • instant creation, deletion & changes to disc / memory quotas (offline only at the moment);
  • IPv4 and IPv6 access;
  • multiple storage grades (SATA and SAS at the moment);
  • imaging from our usual images, and CD-ROM / DVD installations;
  • graphical console access;
  • serial console access (via SSH).

These are features where much of the work is already in place, and which we’ll push out one by one during the beta (not necessarily in this order):

  • clever online resizing;
  • disc snapshots;
  • Windows Server 2008 support;
  • private networks & IPv4/IPv6 ranges;
  • mixing dedicated servers;
  • billing!

A few people have asked about an programming interface / API. The whole thing is controlled by web services, which is no big surprise. But we’re putting our effort into a single front-end, our command-line client, and not worrying about nailing down the API. The command-line client is designed to make automation easy. So I don’t really consider it a "feature" that we will (at some point) document the API properly an promise not to change it incompatibly, but I’m not in a rush to do that.

The "beta" tag will apply to BigV while 1) we might decide to break the whole cluster and take all the VMs down at once for a few minutes, 2) it might happen anyway and 3) we’re not charging for it. When I can cancel all those conditions, I’ll take the beta "apology" away.

What I can be confident of is that BigV is not going to eat your data, we’re not going to take you machines down on a whim, and we’re not going to change our pricing once it’s announced, at least we won’t review it more often than once a year.

So if you’re looking to quickly build your own hosting infrastructure on our network, and pay for only what you’re using, it’s nearly time. Thanks for hanging on.

Posted in Uncategorized | 1 Reply

Back to the future (of hosting)

Thanks to the all the folks who’ve signed up for our beta at http://bigv.io/

We’re polishing BigV to a shiny finish before the beta software goes out in June.  It’s not the finished product, but we think we’ve got enough to show off how the future of hosting should work (cue trumpets).  We have 512GB of RAM to play with, 64 CPU cores, 12TB of regular discs, and 3TB of SAS discs.  We may be pushed for space to accommodate all the beta testers, because we want those people to have the run of the hardware.

I say "the future of hosting", but unlike lots of other hosts, Bytemark are still going to be selling servers, just like the old days.  But BigV will give you flexible billing, so you can fire up and pay for a server for only a few hours – if that’s all you need. It will bring flexibility, so you can change your servers’ RAM or disc space instantly too.  And it will bring resilience: so we will have the capability to shift customers’ servers around our cluster of hardware, if we think any of it is going to fail, or needs maintenance, or an upgrade.  If we’re feeling particularly clever this might become automatic.

Too clever by half

But as Amazon’s monstrous outage shows (an outage that would bury any other host’s reputation for reliability) it is possible to overthink failover protocols.  We recognised this as the biggest risk with BigV and I thought you might be interested to hear about our architecture in more detail.  The promise that "it’s a magic cloud, and you don’t need to worry" won’t persuade anyone for much longer. You’re going to want to know the risks you’re taking by subscribing to a big virtual hosting platform.

I will shock you to your core by telling you that Bytemark’s current virtual machine product is a set of hairy Ruby, shell & Perl scripts. It was originally written in about eight weeks while I was being under-stimulated in a temp job in 2002.  The scripts got passed around in "maintenance mode" for many years and have survived various attempts to "rewrite them properly", including one to use Xen (we dodged a bullet there).

But they do a lot, and we all know how they work, and how they fail. More importantly their virtual machines’ uptimes are mostly in the hundreds of days, spoiled only by the occasional hardware upgrade.

The same, but better

This is still what we want – long up times, permanent discs, and easy upgrades.  And we still think that’s what our customers will want.

The most important things we wanted to add were:

  1. reliable live migration – so we can upgrade our hardware without the laborious work of emailing customers, and spoiling their up times;
  2. VM snapshots – so a customer can "checkpoint" their whole system  before a major upgrade, and back out if it goes wrong;
  3. access to all of KVM’s great features – graphical consoles,  installations from CD, direct network access, and anything else they’d be able to do if they had the server in front of them;
  4. a handy tool for provisioning, server upgrades and maintenance; a uniform interface to the software, rather than the 1980s text  console you get at the moment (I should say "as well as");
  5. really flexible storage, so that servers could use terabytes, not  just a few tens of gigabytes;
  6. a sane software development process and test rig, so we could add features to our live system without errors.

With BigV we’ve turned our simple system into a distributed one, full of features and with (maybe) the minimum possible complication.  To do this, BigV has three types of servers instead of one:

The Brains hold the database of all virtual machines in a BigV cluster, and run the gateway for customers to issue requests for servers.

The Heads are packed full of CPU cores and memory, and run the KVM processes, aka virtual machines.  They don’t have any storage.

The Tails are high-spec servers, but have RAID cards, a normal amount of RAM, and lots of directly-attached discs which can be hot-swapped.

The heads and tails are always connected to the brains, and one of the brains takes on the role of master brain.  That’s the one that keeps a complete list of every virtual machine and disc in the cluster, and that’s what you (the customer) talk to when you ask to provision a new VM.

The heads and tails are also connected to a 10-gigabit storage network, so that the KVM processes can talk to their discs really quickly.

The brain can decide to move either virtual machines or discs between any pair of heads and tails, without having to reboot affected systems.  So that gives us our hardware nirvana - no live customer system need ever be tied to a piece of hardware again.

How it’ll screw up

I’m still mapping out the ways in which this system will break, and may find out a few more during testing – the main one is network segmentation.  We rely on a lot of different local networks between the heads, tails and brains.  If those get misconfigured, the worst hazards might be bringing down every VM at once or freezing disc I/O.

If a head is disconnected, the brain can simply spread its VMs around to other heads.  The rule at the moment is that this happens after an unexpected disconnection period of two minutes.  If the head gets back in touch, nothing happens.  If it doesn’t, the head is under instructions to kill all of its VMs, and the brain will assume this has happened.

With tails, if your data is stuck on a "broken" tail, your data stays disconnected until that gets fixed.  However there are humans in this system too, and well-tested RAID setups, and redundant power supplies, and mirrored memory, and SAS switches.  We know how disc-based systems break, we monitor them, and we’ll fix them.  We’d rather run the risk of a couple of hundred VMs going down in rare circumstances, for a few minutes, than risk people’s data with an automatic recovery mechanism.

In a situation where random cables are pulled out of a BigV cluster, and then put back again, we expect affected VMs (which could be all of them, depending on which cables are pulled!) to do one of two things: freeze or reboot.  Nothing worse.  And it should be a stable system – once it’s put back together, everything will reboot the way it was. There are a few safeguards to prevent two copies of a customer’s VM from running, but I’m still trying to imagine the kind of failure that would make this possible.

Soon, soon…

That still sounds really complicated, but still as simple as I could make it while fulfilling our objectives.  Assuming it works well in testing, BigV will open a lot of doors for us commercially, letting us offer servers that nobody else is.  Plus we have all the benefits of an in-house technology.

The beta is still on for June, so if you’ve not already expressed an interest, head to http://bigv.io/ and do it!

$ bigv vm new

Finally we launch the site for our new hosting platform - bigv.io is looking for beta testers from June. If you think you might want some very cheap, powerful servers around that time, go to the site and tell us how you want things.

We want to give our customers the absolute best of KVM – all of its lovely features exposed, and geared towards providing flexible virtual machines with the highest possible up time.  All our experience in virtualisation, dedicated servers and network design in the last few years are being rolled into the consistent platform that we ummed and ahhed about, but haven’t had the resources to build until now.

Expect support for any sensible OS, expandible RAM & discs, a low starting price, flexible networking and a few other goodies. We have a couple of new ideas for both high-availability and high-scalability servers, and our lovely beta testers will get to try them first.

I promise that’s the last of the teasing. Actual specs and prices will follow.

Ruby Conference in Edinburgh this week

Bytemark is proud to sponsor the annual 
Scottish Ruby Conference, which takes place at the Royal College of Physicians in Edinburgh from the 7th to the 9th April – starting this Thursday!

The Scottish Ruby Conference 2011 is being run by the same team which oversaw the events in 2008, 2009 and 2010. The conference features a wide variety of notable and prominent speakers in the Ruby programming world.

This year’s sell out event differs from previous years, as on Thursday 7th April, a Tutorial Day will take place, where eminent developers Chad Fowler and Keavy McMinn will cover the fundamental building blocks of Ruby.

A free alternative to the Tutorial Day on Thursday 7th April 2011 is The Ruby Festival. The Festival will serve as an unconference event, providing a hacking space to encourage users to share their knowledge and experience in the Ruby programming language.

The Ruby Festival is open to those not attending Thursday’s Tutorial Day or the conference itself on Friday and Saturday. Festival and Tutorial Day attendees are invited to donate to the Children’s Hospice Association Scotland. Scottish Ruby Conference has raised over £15,000 for terminally ill children in previous years.

We’ve built our business on Ruby since 2002 so we really should have sponsored a Ruby conference sooner! Our new hosting platform is built on Ruby, and we’re glad that our favourite language is supported by a vibrant and clever community. We’re very excited to see what new ideas will come of these three days in Edinburgh.

What if the web browser comes second?

The web has the largest reach, and is the front to most mass-market software projects – but have you noticed that even the newest web browsers still can’t do some really basic things?  I think that developers who focus only on the web browser are skirting round some great ideas just to avoid that old-fashioned, native software development.

What can you do in a browser?  Video, some basic games, a word processors, endless to-do lists; most of what you could do in the 1980s but without having to wait to load anything from a casette tape, and not worrying as much about how you were going to save your work.  That’s quite good, that’s a bit of an improvement (from the 80s).

Most advances in the 90s haven’t been implemented very well by web browsers yet, and so lots of software developers live without them.  Here are some things that my Acorn A3000 withcould do in about 1993, compared to web-based software now:

  • multitasking – Chrome and Firefox multitask about as well as in 1993, i.e. just fine until one program (tab) goes crazy, and all my programs (tabs) freeze up.  At least I only need to restart my web browser when that happens, and not the whole computer.
  • 3D graphics – we have WebGL! The linked demos worked on my bleeding-edge browser.  For a bit.  Then the browser froze and the tabs crashed.  Nobody has done as ambitious or exciting a 3D game on the web as Star Fighter 3000 (that’s not to say Chrome can’t render 3D graphics, but nobody has chanced their arm on a whole game – yet).
  • multiple programming languages – pick any one you like, as long as it compiles to Javascript! Google have NativeClient but it’s not a sure bet to succeed yet. Or you can run your own code on your server, and only have to do the “fast” bits in Javascript, which is pretty ungainly.
  • complex creative tools - if you want to lay out pages for print, or compose music, or draw complicated diagrams and multi-stage artwork, and have these programs work together to produce a finished product between them the Acorn still wins hands-down on quality and depth of software.  Even cut & paste between Google Docs is fraught; even when you use it in Chrome.

I’m 90% certain the web will be the way to go, in the long run. All this stuff is being worked on, reinvented.  But in the mean time software developers need to consider looking beyond the web.  Maybe the web browser can come second to a native application, not first?

By thinking about the actual computers people use, and not the wobbly Hypercard clone that sits on top of most of them, there’s still masses to be done. There’s rich, brilliant services to be delivered that haven’t even been invented yet because of the perceived difficulty of writing “native” software.  But look at companies who roll their sleeves up and got on with it:

  • Dropbox is making a killing on their file sharing service, which is delivered through a native client on stodgy desktop PCs and Macs.  That can’t work without little hooks all over a user’s PC, but they need those hooks to make the magic folder work. There’s no other way.  Off the back of this peerless integration, they effectively become a hosting company, photo sharing service and a data synchronisation API for programmers.
  • Skype‘s VoIP software and network have not been copied by anything on the web, at least not as well. Again, look at the necessary integration – webcams, subtle notifications, nice snappy interface – none of these things can be done cleanly in a web browser.
  • Mozy (and others) install backup software on your computer and slurp off every file so they can give it back to you when you lose your PC. Web browsers don’t let their downloaded programs near your local files, let alone allowing them to create shadow volumes or filesystem wizardry.

Windows & Mac specific knowledge is scarce compared to knowledge of PHP, HTML and Javascript.  But if you have an idea that can only function with some desktop integration, native software is still the only way. There are cross-platform toolkits, languages, and build tools. Sure you have to test a lot more. And learn a lot more. But that’s the price of your software being able to do a lot more.

While development for mobile phones is big business if you’re selling anything “social”, the laptop & desktop computer is still where any truly creative and complex art is produced. Those creatives with massive screens, massive drives and fast connections want the best software; while they’re relying on the web browser to deliver to them, they’re often being sold short. There are still big opportunities outside of the web browser.