Where did all those IP addresses go?


In April, ARIN, the (North) American Registry for Internet Numbers, announced that it had reached "phase 4" of its IPv4 countdown plan, with fewer than 17 million IPv4 addresses remaining. There is no phase 5. APNIC, the Asia-Pacific registry, reached the 17 million (one "/8" or 2^24 IPv4 addresses) threshold three years ago, and the RIPE NCC in Europe less than two years ago. LACNIC, the Latin American and Caribbean registry, reached a similar threshold of a little more than four million remaining IPv4 addresses earlier this week. APNIC and the RIPE NCC will give ISPs and other network operators one last block of 1024 addresses, the rules for LACNIC are similar, and ARIN is tightening the address supply but still allows ISPs to come back for more. Only AfriNIC in Africa is continuing to supply IPv4 addresses as needed to network operators in its service region.

How did we get here?


It could have been worse. In the 1980s, there were several widely used networking protocols, such as DECNET, AppleTalk, IPX, and CLNP. DECNET had 16-bit addresses, AppleTalk used 24 bits, IPX 80, and CLNP a maximum of 160. The newly invented TCP/IP held the middle ground at 32 bits. However, unlike most of the other protocols, which were never intended to underpin global networks, IP is the Internet Protocol, designed to interconnect all kinds of smaller networks into a unified, global one. As such, making the addresses a meager 32 bits was a big failure of imagination. That's only ten digits when written down as a regular decimal number.

The result was that it took only a decade before IP address numbering ran into trouble. Originally, IP addresses came in three classes: A, B, and C. Class A consisted of 128 networks with room for 16,777,216 connected systems (hosts) each. Class B was 16,384 networks with 65,536 hosts, and class C 2,097,152 networks with 256 hosts each. In the early 1990s, more and more universities connected to the Internet. Universities typically had more than 256 computers—or at least the potential to grow beyond that number—so they tended to get class B networks, which quickly started to run out. Giving them a dozen or so class C networks was much more efficient, but now routers had to keep track of ten times as much information, and routing tables started to explode. The Internet Engineering Task Force was barely able to avoid disaster by abolishing the class system so an organization that needed, say, 3,000 addresses could get a "/20": a range of IP addresses sharing the same 20 bits (the prefix) with 32 - 20 = 12 bits left to number hosts within the network.

Under the new classless regime, the deployment of new IP address space slowed down to a much more sustainable pace as the Internet boomed and then busted (a little). Around the turn of the millennium, more and more people got broadband always-on connections, and a few years later the mobile era dawned, where untold millions of smartphones were continuously connected to the network, too. Surprisingly, these developments only produced a small uptick in the IPv4 address usage rate. The reason for this is probably that by now, NAT was seeing broad adoption.

Network Address Translation

Before there was Voice over IP, there was IP over the voice network; i.e., in the 1990s we used modems to encode digital data such that it could be transmitted as screeching noise over the analog phone network. (Actually, the core of the phone network was already almost exclusively digital by then.) Apart from cute noises, tied up phone lines, and mind-numbing slowness, dial-up connections had the property that a user only required an address as long as she was connected. So an ISP with 10,000 users may have had a modem bank with 1,000 modems and thus needed 1,000 IP addresses.

As dedicated ADSL or cable broadband connections became available, the notion of making a connection, doing your online business, and then disconnecting, quickly went away, and we became always-on. So now 10,000 users required 10,000 IP addresses. However, in the early days, a cable or ADSL modem was still connected to a single PC. That PC thus held the IP address provided by the broadband ISP.

It didn't take long before people wanted to use more than just one PC with their broadband connection. ISPs were, of course, happy to provide more IP addresses—for a small extra fee. However, a cheaper solution is to share a single address among multiple computers. This is what Network Address Translation accomplishes. With NAT, computers and other IP-capable devices get an address from one of the IP address ranges set aside for private use: 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16. A home router that implements NAT then translates between the internal address and the regular, public address given out by the ISP. By modifying the TCP port numbers where necessary, the NAT device can avoid conflicts between the TCP sessions from different internal systems when they're active at the same time. (And much the same for UDP.)

The Stanford vs. China story

In articles from a decade ago that forewarned the looming predicament we find ourselves in today, it was frequently noted that Stanford University (or MIT) holds more IPv4 addresses than the entirety of China. However, by 2006, organizations in China held a total of 98 million IP addresses, nearly six times as much as MIT's class A block. Stanford also used to have a class A block, but had returned this in 2000. Before that, the story was actually true, as China held fewer than eight million IP addresses by the end of 2000. However, China (and some other Asian countries) used up a lot of IPv4 addresses while the getting was good: about 50 million addresses a year between 2008 and 2011, for a total of 330 million today. This makes China the second largest holder of IPv4 addresses, behind the US with 1.591 billion.

If the numbers were reversed, both countries would have about one address per resident. But in the current situation, it's about a quarter of an address per person in China and more than five per person in the US. In Africa, the number of addresses often dips below less than one address per ten people. Only in the US, Canada, parts of Europe, and a few selected countries, such as Korea and Australia, is the number of addresses per person larger than one. And with just over 3.7 billion usable addresses, the average for the entire planet isn't going to be better than 1:2. Out of a possible 4.295 billion IPv4 addresses, 268 million are set aside for multicast. Another 268 million are marked as "reserved for future use," and many operating systems don't allow them to be used, unaware of the fact that the future has now arrived.

So what now?

In a statement to Ars, John Curran, president and CEO of ARIN, stressed the need to adopt IPv6: "This issuance of IPv4 space in accordance with global policy has been expected for some time (and will occur several more times in smaller amounts) but doesn't change the need for ISPs and websites to move to IPv6." LACNIC echoed that tone in its announcement: "Today, the need to deploy IPv6 is now more pressing than ever. It cannot be delayed any longer if connectivity providers still wish to meet the demands of their customers and those of new users."

It's true. There is no plan B. During the past 10 years, 1.6 billion IPv4 addresses have been given out. It's inconceivable that the Internet as we know it today can continue to grow at a meaningful rate over the next decades with pretty much no new addresses being added, even as addresses are now traded. Even if no additional addresses were required, when one ISP grows and another loses business, the contracting ISP is left with a Swiss cheese-like address space full of holes while the growing one needs to find new addresses in the form of reasonably sized blocks to avoid exploding routing tables.

IPv6 is a new version of the IP protocol that increases the address length to a mind-boggling 128 bits, solving the problem, if not forever, then at least for many, many decades. Unfortunately, IPv6 is not compatible with IPv4—it only helps once everyone has upgraded. However, as Geoff Huston, chief scientist at APNIC, observes:

Yes there are some countries and some ISPs that are doing amazing things with IPv6 over the past 12 months: The United States at 7.5% continues to move quickly, as does Germany with 10%, but many many other countries appear to be sitting on their hands. LACNIC has now run out, but the level of IPv6 penetration in Brazil is 0.04%, which is better than Argentina (0.01%), or Mexico (0.02%) but not by much. 13 countries are above the average of 2.2%, while the other 190 or aren't. Given that networking is a matter of everyone working roughly at the same thing at the same time, things are still not looking good.
These are, of course, numbers of individual Internet users that have IPv6, (almost always) in addition to IPv4; see Google's measurements. It's also important to get websites and other services on IPv6, but those only use a tiny number of IPv4 addresses—it's the consumer ISPs that get the bulk of new IP addresses, which means that they're also the first ones to run into trouble when that's no longer possible.

And it gets worse: deploying IPv6 doesn't solve the short-term problem, as IPv6 users can't talk to IPv4-only services or other users who still only have IPv4 connectivity. Current operating systems can all use IPv6, but they don't always work as expected in an IPv6-only environment. And some applications and, especially, networked devices simply don't work with IPv6. The most notable example is Skype. All of this means that ISPs really have no other choice than to keep IPv4 running in some way for now.

NAT to the rescue—again

When broadband users couldn't get extra IPv4 addresses from their ISPs at a price they liked, they adopted NAT. So now that ISPs can no longer get IPv4 addresses at a price they like, they're also turning toward NAT. Of course there is a difference between a $50 home router that can handle the NATting for a single home and a NAT that can handle an entire neighborhood. These are called a Carrier Grade NAT (CGN), but they basically do the same thing. To avoid problems with the private addresses in the 10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16 ranges, there's a separate semi-private address block that ISPs can use between the CGN and their subscribers: 100.64.0.0/10. If you get an address in the range 100.64.0.0 - 100.127.255.255, that means you're behind a CGN. According to Geoff Huston, at least 3 percent of Internet users are already in that situation:

"A lot of CGNs are being deployed. Some recent work I have been doing shows that upward of 3% of IPv4 users present on a different source IPv4 address within 10 seconds—i.e. a minimum of 3% of users lie behind CGNs with relatively aggressive address lease timeouts. Secondly, a lot of shared Web hosting is being deployed. It is evidently commonplace to see upward of 10,000 Web host environments co-existing on a single host IP server address."
ISPs are playing their cards close to the vest, but it looks like many of the ones that are planning to start rolling out IPv6 soon will be deploying IPv6 along with CGN-based IPv4 for new users. They're reluctant to change anything for existing users, because the first rule of being an ISP is "don't generate support calls." Providing broadband Internet access is a very profitable business, but the profit generated by a customer evaporates faster than you can say "have you tried rebooting your router?" when said customer calls for support. ISPs that started deploying IPv6 in past years had access to enough IPv4 addresses to give users their own along with a range of IPv6 addresses. That is no longer true, or will no longer be true as soon as ISPs use up their own stashes of IPv4 addresses.

The downside of NAT is that it only works well in one direction: from the inside to the outside. When connections must be set up from the outside to the inside, such as in the case of peer-to-peer audio or video conferencing, additional logic is necessary to find a way to the right internal system through the NAT. This is bad enough when two users are both behind their own home NATs so that two NATs must be bypassed, but it gets much worse as ISPs deploy CGNs, so now four NATs can be in the path. CGNs also can't open up ports as easily as home NATs. As long as it's not firewalled too severely, IPv6 has none of these issues; with 2^128 addresses there is no need for NAT. So it makes sense for ISPs to deploy IPv6 along with CGN-based IPv4. However, there are still ISPs that pooh-pooh IPv6. Huston again: "The pessimistic view is that so far nothing much has broken in IPv4-land, so there is still some more time left to do nothing!"

Unless the ISPs that have been ignoring IPv6 plan to just keep their existing customers and not sign up any new ones, those ISPs are still going to be bitten by the IPv4 address exhaustion and will almost certainly be forced to deploy CGN at some point. With no pressure relief valve in the form of IPv6, all user traffic will have to flow through the CGN, which can then easily become a bottleneck and a single point of failure. As a result, the quality of service delivered by different ISPs will diverge more and more, with the ones providing unshared public IPv4 addresses as well as IPv6 doing the best and the ones using CGNs with relatively many users per public IPv4 address and no IPv6 doing the worst.

The good news is that so far, the Internet has always managed to adapt just before collapse was imminent. In the late 1980s, TCP congestion control saved the Internet from massive congestion. In the 1990s, classless interdomain routing and route flap damping kept the routers going. This time we only have to turn on a feature that's been in our operating systems for a decade and maybe replace an aging modem or two. Call me an optimist, but I think it can be done. But only at the very last moment, of course.