root ain’t what it used to be
My own groundhog day is a debate with customers about what constitutes “typical” Linux security precautions. If you’re a Linux administrator of any experience, you might find some of the following statements rather familiar:
- My server is secure because Apache runs in a chroot.
- My web site is secure because I have a hardware firewall! Nothing can defeat a hardware firewall!
- I have 67 non-standard kernel patches to ensure that normal users will ever get root access.
I’m sure my weary tone suggests the above all contain misunderstandings about computer security. The assertion “I am secure because X” implies that security is just a checklist item around an important computer system – the falsehoods are that 1) either it’s secure or it’s not, 2) security is something you can outsource to a clever person or clever tool, while you get on with the “real” work, and 3) that preventing root access should be the overriding focus of any computer security system. Because computer security is meaningless out of context – exactly what are you trying to protect, and from whom?
From a survey of our support tickets, the number one backdoor into their servers is a PHP web application. It goes like this: 1) customer installs a popular message board / content management system on their server, 2) customer starts using it and months later hasn’t updated it or even checked back with the authors for security problems, 3) hacker finds customer’s server through Google, applies a canned exploit and gains shell access to the customer’s system. The hacker then starts to use the system for sending out spam, hosting a phishing site, a base for further hacking attempts or high traffic denial-of-service attacks. Usually we receive a complaint about the customer’s server very quickly, see the signs of compromise, and shut it down.
For the majority of our Linux-savvy customers, this generates no more than a sigh as they kick the hackers off, close the security hole and restart their system. They’re often not surprised that their system has been compromised, and it doesn’t usually happen again to the same person. As far as these hackers are concerned, every system on our network is equally valuable to them (i.e. not very).
And really, I don’t think this scenario is a big deal, even though it happens once every couple of weeks for us. The customer only puts themselves and their data at risk, and the hackers are only really looking for a jumping-off point for their petty criminal activities. Once we spot the hackers and close them down, they’re gone.
So my security advice to most customers is simply:
- for every web application you’re running, listen for security updates and apply them. That’s it. That step will almost certainly stop your system ever becoming hacked, because enough other people won’t bother with it. The hackers you should worry about are only interested in an easy hack! If you don’t provide that to them, they will go elsewhere;
- don’t set stupid passwords and try to make sure your users don’t – dictionary words, same as the user names, and so on (this is the other very likely route in);
- (advanced, but easy) set up a simple outbound firewall to block outbound TCP connections from the web server. That way, even if you do get compromised, your server is useless to the hackers because they can’t do anything they want to do once they’re in.
(Our vhost system tries to do help with the last two items, as well as ensuring that your base system is updated with security patches)
So here are two reasons why root access doesn’t matter – if your system does get compromised, there is no point worrying about whether they “got root” or not. What harm would it do to assume that they did, and how would you tell the difference anyway? If you can imagine someone who might be interested in sticking around on your system and snooping at its data, don’t take the chance, and rebuild your system sooner rather than later. Linux’s history of local-root exploits in the last few years suggests that even if you don’t know it yet, a local user can probably escalate their privileges.
Or from another perspective – the root user is often irrelevant in a hosting environment because for the last 15 years, Linux distributions run all their important web applications as the same UNIX user, in the same stinking process! Once that user is compromised, typically every database in every application is accessible to the savvy hacker, and the longer they had access, the more of your users’ login details they can harvest through sniffing login data.
I’m not saying that I like this state of affairs – you might standing a fighting chance if you take the trouble to set up every web application in a separate UNIX user account. Or if you like masochism, you can create separate virtual machines for each application you want to run! But until distributions help with splitting up web applications, the majority of them will stay hosted in a single process, and so a compromised web server needs to be treated as severely as a compromised root password.
(Most of our customers who are serious about security cope with this by, errr, buying more hosting for each “important” application. Or asking us to design their hosting architecture. Either of which is a perfectly good solution – but I still care about everyone else 🙂 )
