Why net neutrality repeal will drive us to the edge

VPNs, Wi-Fi Meshes, pods, supernodes, and intelligent distributed cloud services will become the lingua franca of the new Biased Net.

It seems that with every passing administration, the issue of Net Neutrality pops back up.

The tug of war between broadband consumers, content and application services providers, large broadband and telecom providers and what sort of traffic will be permitted to transit their networks -- and what quality of service (QoS) policies they eventually will apply to them -- seems to be a conflict without resolution.

Our previous administration was one of permissiveness and open access to the internet for everyone, and for all services. The current administration, under President Trump and FCC Chairman Ajit Pai appears to be positioned to give internet bandwidth providers the power to revoke or throttle that open access in an upcoming landmark vote next month.

Regardless of what the FCC vote hands down in December, the implementation of Net Neutrality (or lack of it entirely) will be the kind of hot-button issue that could very well alternate between Republican and Democratic-controlled administrations for decades.

If we accept that there will probably never be a guarantee about what kind of traffic will be permitted across provider networks to the open internet from consumer on-premise (or mobile) devices, then we need to fundamentally re-think how devices communicate and store data on the Internet.

We need to ensure that we can continue to use the types of services we currently enjoy -- as well as those developed in the future -- without interruption or degradation of quality, regardless of what sort of policy implementation we see regarding the use of the internet going forward.

Ultimately, broadband services providers like Comcast, AT&T, and Verizon never want you to leave their networks. They want you to consume the balance of your traffic within their intranets because it's much easier for them to control costs within their own network infrastructure. Once traffic transits from their network onto the open internet, there are ingress and egress costs when that traffic is handed off to another provider.

The point at which your broadband traffic leaves the provider network and transits the public internet (and is potentially handed off to another provider) is referred to as the "edge". And if Net Neutrality is revoked, our understanding of and how we interact with that edge is going to become much more important.

Today, when we consume content from the internet, the endpoint device -- whether it resides on your home network or used on a provider mobile network -- streams data originating from a cloud datacenter that stores and serves up the content.

This is how Netflix, YouTube, Hulu, HBO Go, and CBS All Access and any number of content-streaming apps work.

To accelerate the consumption of that content and to minimize the number of network "hops" it takes to get to that content, many broadband providers have direct network connections to public cloud providers.

For services hosted on AWS, Azure and Google -- what the industry refers to as "hyperscale" cloud providers -- it's usually fewer than three hops away since they already have peering agreements with the big telcos.

Content providers like Netflix also have the ability to put compute and storage caching devices on the edge of each of these telco networks. This is called a Content Delivery Network, or CDN.

Companies like Akamai and Limelight specialize in this -- when you download Windows Updates, Android and iOS apps from the respective stores, or pull down large, multi-gigabyte ISO files for things like Windows 10, Linux distributions or Mac OS X, a CDN will often come into play.

They are also frequently used for caching image hosting for social networks like Facebook and Instagram.

So all your cat memes, animated GIFs and fake news that you like to share? That's CDN.

So we can expect that the more profitable and more influential content delivery companies -- Netflix, Amazon, Google, Facebook, and Apple -- will probably continue to deliver to your home without interruption. It's absolutely in their interest to maintain good relationships with each of the major telcos acting as home broadband providers -- and vice versa.

But what about everything else? And what if a particular telco, for whatever reason, has a deteriorating relationship or a break with a major content provider due to conflicting or overlapping interests?

That possibility becomes much more realistic as we see telcos continuing to buy content providers and who would prefer you actually consume the content from their own services and not someone else's.

For that eventuality, we will need mesh/distributed content networks and blockchain technology (for secure transactions and rights management) incorporated into the Internet of Things.

Peer to Peer distributed node/mesh/swarm technology has been around for a while -- particularly for things like BitTorrent. However, it hasn't really been considered for mass adoption and embedded into consumer technology until now.

We need to reconsider the use of localized storage on our streaming devices in order to fully optimize the use of telco networks, and to form pods/swarms of compute and storage in large residential areas.

Imagine, for example, that you have an Amazon Fire TV, Roku, or Apple TV box in your living room. Instead of it being a strictly streaming device, it or your internet gateway device (router, etc) could have a large flash drive.

That flash drive might not be able to store all of Amazon's, Apple's or Netflix's content, but it could easily cache the top 5 or 10 percent of it. And if you were to group a hundred or a thousand households together, virtually all of the content could be distributed across the nodes.

You would simply need a cloud service to actually keep track of where the bits are located -- just as the way BitTorrent works.

Data could be transmitted within the provider network without jumping onto the internet at all, and it could be done during off-hours, such as late at night when network utilization is low to make the use of network resources most effectively.

All of this can be done via a DVR-like interface where subscribed content is pulled from the internet CDN to algorithmically-chosen primary distribution nodes in representative households and then disseminated to the rest of the local pod as required by the coordinating cloud service.

In essence, we would be moving from a streaming-based model to a distributed download model.

Regardless of the actual legislative impact of Net Neutrality, I actually prefer this method for content distribution because you no longer have to worry about buffer interruptions, competing for bandwidth with your next door neighbor's kids during prime hours or degradation of content quality because the speed and QoS of your broadband connection are no longer as important for content consumption.

Even with relatively slow asymmetric broadband uplink speeds of 5Mbps to 20Mbps, it's still very workable if the objective is to transfer data on a coordinated schedule between nodes, and not to download content on demand. Uplink speeds also become less of a concern on a provider's intranet if the switch infrastructure is sufficiently robust, so allowing synchronous speeds for pod members when not transiting the internet is possible.

All of this would work fine as long as the provider itself did not shut down the firewall ports or apply highly restrictive QoS that wouldn't allow such a theoretical device and pod to communicate.

However, data could also be transmitted using a Wi-Fi mesh network which the provider would have no control over at all.

There are other mechanisms that could be used to work around this issue such as VPNs, and also a return to sneakernets (which are extremely popular in oppressive regimes like North Korea). But this could be a much smarter sneakernet, because we now have smartphones with the ability to communicate with the cloud and to supernodes in local pods.

So for example, a Redbox-like company could provide an app that allows your mobile device -- many of which are now equipped with hundreds of gigabytes of underutilized local storage -- to pull subscribed content from kiosks or access points distributed in high-traffic areas using gigabit Wi-Fi speeds.

It could do this all in the background without significantly affecting regular mobile device tasks and app responsiveness.

When you get home, your mobile device could automatically update your distribution node with the content you've downloaded. The cloud service would do all the work of telling all the other nodes where to get it.

There is obviously a great deal of complexity in a mesh-based distributed content approach and a lot of additional software engineering work is going to be required in order to pull this off. But I believe that out of necessity, our industry will have to rise to the occasion.