It’s nearly four years since we began building Brightbox, so we thought we’d write a series of blog posts detailing some of the design decisions we’ve made over the years. To kick it off, I’m going to talk about how our Cloud IP address allocation policy protects our customers’ security (and the reputation of our address space).
Cloud IPs are public IPv4 addresses that can be instantly mapped to, and moved between, Cloud Servers, Load Balancers or Cloud SQL instances. They help you treat your servers like “cattle not pets”, by not tying your public facing addresses to your back-end systems.
When you “create” Cloud IP addresses, they are assigned to your account from a pool of unused addresses. You’re then free to map and unmap them as you wish. They’re completely in your control until you “destroy” them, at which point they return to the pool and can be reused by other customers.
It’s worth noting that Cloud IPs aren’t necessarily required for each Brightbox Cloud Server, since each comes with its own public IPv6 allocation and a private IPv4 address so they’re immediately accessible via the Internet on IPv6 and from other Cloud Servers within the same region via the private IPv4 address.
Once an IP address has been used for something, particularly via DNS, it’s out there in the Internet’s “consciousness”. It could be hanging around in badly behaving DNS caches or HTTP caches, it could be in host files, or log files, or just sat on someone’s command line :)
So if a Cloud IP is allocated to a different customer soon after being returned to the pool, it could end up receiving unwanted packets.
Obviously, any IP address on the Internet inevitably receives packets it doesn’t want, but in this case we can mitigate some of the risk with a simple “first in first out” policy (FIFO). If you destroy one of your Cloud IPs, it returns to the pool but we’ll delay re-allocating it as long as possible.
Avoiding reusing Cloud IPs as long as we can also helps us with abuse reports. If we receive a report about abuse associated with a Cloud IP it’s much easier to link to a given customer. We obviously have an audit trail of who owned which Cloud IPs and when, but it still reduced the overhead if they’re not rapidly changing between customers.
An abusive user could easily tarnish a Cloud IP’s reputation by, for example, participating in activity which causes it to end up on a blackhole list. In this case, a blanket “first in first out” policy would work against us as the user could just keep destroying and creating new Cloud IPs, tarnishing a large number of them.
So, one of the exceptions to the FIFO policy is that we try to reallocate Cloud IPs back to the same customer, to prevent them churning through a lot of them. If you tarnish an IP, you’ll need to sit in your own filth for a while (or use one of your other Cloud IPs). We’re able to make manual exceptions to this though, on a case by case basis.
IPv4 addresses are in short supply and whilst we have plenty, we don’t want to encourage waste. So, each new account has a default limit on the number of Cloud IPs they can have allocated at any one time. Customers can get the limit raised, but need to justify their request.
The quota also helps make it even harder for an abusive user to rotate through and tarnish lots of addresses.
If we re-allocated Cloud IPs in a predictable order this could be exploited by an abusive user to gain control of specific targeted Cloud IPs, perhaps as part of an attack. So we add some randomness in there to make it difficult. So it’s not quite “first in first out”, but more “early in late out”.
As you can see, we’ve put quite a bit of thought into the allocation of Cloud IPs in order to protect our customers and our reputation.
If you’re not already a customer, you can sign up in about 2 minutes and get a £50 free credit to play with. You can have a server booted in 30 seconds.
