A look into Usenet, the original wild west of the Internet

If you’ve hung around the internet for any length of time, you’ve probably come across the word “Usenet” a couple of times. Depending on the context, you may have heard it described as a near-mythical hacker’s proto-internet, a good place to find free files or a forum with complicated names like alt.binaries.boneless. If you took the time to read the Wikipedia article, you were faced with a meandering history that covers everything from network diagrams to “The Eternal September” and “The Great Renaming.”

There are gigabytes of interesting information about it out there on the Internet, but summing up what Usenet is in a single sentence is a challenging proposition. It’s a decentralized forum designed for textual communication within categories called “newsgroups.” It’s also a popular place to store and share files that users may or may not have the rights to. It’s the original internet community, existing before P2P file sharing apps, Reddit, 4chan, and vBulletin fan forums. Usenet was the internet before the internet was the internet; it’s a piece of digital history that still accessible and useful today.

Using Usenet

Here’s the thing about getting started on Usenet: you have to pay for it. In the ’90s many ISPs provided access directly to Usenet alongside internet services but technological changes and legal battles have led almost all of them to cut access entirely. If you want to log on today, you’ll have to purchase access from one of the dozens of Usenet providers available. Major providers like Newshosting, EasyNews, Usenet Storm, and Astraweb offer packages that range from less than $5 to around $30 per month. Each service and package will have advantages and disadvantages.

Key traits to look for in a Usenet provider include file retention, data caps, security, and parallel connections. Users looking to download files should pay particular attention to data caps and security. Data caps will set a monthly limit on how much you can download, but unlimited packages are also available on almost all providers. At a minimum, security features should include SSL encryption and a no-logging policy. Encryption protects your connection during access and a no-logging policy protects you by not keeping track of what you download.

Some providers will bundle VPN access with their basic service, while others charge separately for the service. If you’re interested in how your provider uses your information and what stance they take on providing data to law enforcement you should read their privacy policy, Terms of Service, and DMCA policies. For the truly paranoid, some providers even accept Bitcoin for true anonymity. Heavy downloaders will also want to prioritize providers that allow lots of parallel connections. This determines the number of downloads you can run concurrently, which is a big deal if you want to get the most bang for your Usenet buck.

File retention refers to the length of time a Usenet provider will keep files on their servers. You should look for at least one year of retention, but if you’re interested in older files some providers offer retention rates of up to four years. Many providers distinguish between their text and binary (software/media) retention. Text retention tends to be higher due to the small file size and lack of copyright issues, so if you’re shopping for a high-retention provider be sure the number they advertise includes all groups or that they specialize in alt.binaries like UsenetStorm.com.

Once you’ve signed up with a Usenet provider you’ll need a newsreader (or newsgroup readers). Think of a newsreader as a web browser for Usenet; you’ll use it to search, read content and handle file downloads. Due to the way Usenet file storage is structured (something we’ll talk about below), large files are broken into many small files. We download these files using something called the NZB standard. An NZB points to the location of file parts across Usenet; your newsreader will read this file and use it to download the pieces and assemble them into your finished file. Not all newsreaders will support this kind of downloading. Those looking for a free option can check out SABnzbd or platform-specific alternatives like Hellanzb on Mac and Alt.Binz on Windows.

Hierarchies

If you step away from the NZB search engines and choose to download and browse Usenet directly, you’ll notice that the “threads” are organized under newsgroup names like alt.binaries.pictures. These names contain the hierarchy of the newsgroup and are the backbone of how Usenet categorizes content. Every single upload to Usenet belongs to a newsgroup and is categorized under one of these names. The names contain all newsgroups above it in the hierarchy starting with the highest; a hypothetical racing fan newsgroup might exist at alt.binaries.multimedia.racing. There are 9 main hierarchies you need to be aware of today: The Big 8 and the infamous alt.* hierarchy.

The Big 8 hierarchies cover, you guessed it, eight topics:

comp.* (computing), humanities.*, misc.* (miscellaneous topics including education and classifieds), news.*, rec.* (entertainment), sci.* (science), soc.* (society) and talk.* (controversial discussions).

Groups are added to or removed from the Big 8 based on a system of public discussion, nomination, and voting. The Big 8 Management Board works to “create well-named, well-used newsgroups,” make adjustments to and remove existing groups, and assist in the creation of a canonical list of Big 8 groups. The Big 8 represents a fulfillment of the original vision of Usenet: a system of community-moderated boards dedicated to discussion.

If you’re looking for the mythical hacker file-sharing Internet, head over to the alt.* (alternative) hierarchy. Alt.* includes everything from the discussion of adoption and poetry to file sharing (alt.binaries) and famed hacker newsletters. There is no centralized control of the hierarchy and anyone capable of creating a group can do so. However, there is public discussion of new groups at alt.config that helps newsgroup creators create successful newgroups and integrate them into the hierarchy. The alt.* hierarchy is responsible in large part for Usenet’s legal troubles and its somewhat shady reputation.

Hierarchies exist outside of alt.* and the Big 8, but this outline covers mainstream Usenet. Explaining why the hierarchies are arranged this way requires discussing the history and creation of the platform itself.

Usenet History 101

Duke University students began experimenting with what they called “netnews” in 1979. This software utilized Unix-to-Unix Copy Protocol (UUCP) to connect Duke computer science department members to each other and to other universities. When this experiment succeeded in replacing their current collaboration tools (long-distance phone calls and postal newsletters), they continued work on the project and released it to the public as “A News.” A News organized content into three hierarchies: fa.* for content From ARPANET, mod.* for curated discussions and net.* for unmoderated groups. Developments in the A News Software (and successors B News and C News) made some changes to the structure but Usenet continued on with roughly this structure until The Great Renaming of 1987.

Originally designed solely for textual conversations, the face of Usenet changed thanks to the development of Uuencode by Mark Horton in 1980. Uuencode took a binary file and encoded it as a series of text files. This made binary files compatible with Usenet’s upload requirements and transformed Usenet from a discussion board into a filesharing hub.

The Great Renaming of 1987 was organized by B News maintainer Rick Adams. Adams worked with administrators from the “backbone” of Usenet providers to create seven hierarchies: comp.*, misc.*, news.*, rec.*, sci.*, soc.* and talk.*. The alt.* hierarchy came into existence soon afterward to avoid the moderation of the mainstream groups. Later, humanities.* would turn the Big 7 into the Big 8 during an academic traffic surge in 1995.

“Usenet Culture” would come to an end under a wave of new users in what’s referred to as the “Eternal September”, which was the September 1993. Before the Eternal September, Usenet users largely came from academic networks. Freshmen would arrive in droves every September and would either acclimate to the Usenet culture (also known as “netiquette”) or slowly fade from the service. In September of 1993, America Online began offering Usenet access to all subscribers. This endless influx of new users buried Usenet culture under mass usage and helped coin the term AOL’er as an insult.

Since the Eternal September, Usenet has faced a host of obstacles: the ever-growing bandwidth and storage weight of piracy, legal pressure from copyright lawsuits and law enforcement investigations, blacklisting by most ISPs and the retirement of the original Duke University server in 2010. Still, Usenet lives on today; the providers seek legal cover behind their status as a “telecommunications provider” that isn’t responsible for user-generated content and the users continue anonymously swapping stories and software in the far corners of the internet. With a little money and a little work, you too can experience The Internet That Was. Whether you’re looking for files to download or want to experience the Internet community that gave us terms like “flame war”, “thread”, and the “Meow Wars”, it’s worth putting in the time to dig into Usenet.

Thanks for the writeup! Any advantages in using usenet over private trackers?

Originally Posted by hellogoodbye

Thanks for the writeup! Any advantages in using usenet over private trackers?

Comparing Usenet vs torrents:

1. Server link vs. Peer-to-peer

With Usenet, you directly connect to the server to access the available files for download. Plus, you download the files directly from the server. However, when using torrents you download multiple pieces from multiple peers. The Usenet server has an almost 100% uptime. Whereas if you use torrents, you are limited by the number of seeds and their availability. If the seeds are not active, you will not be able to download the files.

2. Encrypted connection vs. Unencrypted connection

The connection to a Usenet server is encrypted by using SSL. That’s a cryptographic protocol that provides communications security over a computer network. Therefore, your Internet service provider will not be able to detect the files you are downloading, thus protecting you from getting notices. The torrent connection is not encrypted. So your ISP can easily detect the type of download as being a peer-to-peer connection. As such, you’re more prone to receive notices from them. But you can protect yourself from prying eyes while torrenting by using a VPN service like IPVanish. The VPN service will encrypt the connection, offering a higher degree of protection when downloading files while using torrents. You can check our article on how to protect yourself using a VPN service to learn more about VPNs.

3. Paid subscription vs. free access

In order to connect to a Usenet server, you need to subscribe to a Usenet service. This usually requires a monthly/yearly fee. Although free alternatives exist, I would not recommend even trying them out because the amount of available files is limited. My favorite Usenet provider is Newshosting which offers unlimited data transfers for less than 16$/month. Torrent access is free for the most part. But when using a free torrent tracker the transfer speeds are low. Besides that, if you manage to get access on a private tracker, you need to seed your files and build up a ratio, which can be time and resource consuming.

4. Constant download speeds vs. limited speeds

Usenet servers usually offer speeds ranging from 1Gbps to 10Gbps, so you will probably be limited by the speeds offered by your internet service provider. When using torrents, you can saturate your internet speed, but only if you use a private tracker with good seeders. When using a public tracker, the download speeds are usually mediocre because of fewer seeders. This means that if there are not enough seeders, the speeds will be slow, while on Usenet, the speeds are usually constant and fast. If you have a good Internet connection speed, you should direct your attention to Usenet.

5. Automated file processing vs. raw files

When downloading files from Usenet, you may encounter content that has been previously compressed into an archive. For example, a video file is usually compressed into multiple small archives to improve download speeds. The Usenet client will automatically unrar the video file, preparing it for your consumption. On the torrent side, you will usually get the video file directly, without it being into an archive.

There are a lot of times when content from Usenet gets to the torrent trackers and the content is untouched, with the video file split into multiple archives without it being unarchived in advance. This can be a problem for automated download scenarios because most of the torrent clients are not automatically unarchiving the content. There are some ways to automate the unrar process, so if you want to learn more about this, make sure you check our article on how to automatically extract RAR files after download.

6. Limited availability vs Longer availability for files

On Usenet, all the files available for download are stored on Usenet servers, in a central location while on torrents, the files are distributed on multiple servers and computers. While having all the files on a single location offer great speeds, it also means that the Usenet servers have a limited availability because of limited storage space. In the Usenet world, this is called retention, and when looking for a good Usenet provider you should look for one with a long retention time if you need access to older files. On the other hand, files available as torrents are limited only by the space available to the users, and their willingness to seed those files. This means that if you are looking for some rare file, you will have a better chance to find it on torrents rather than Usenet.

And some Extra Info:

What are Usenet Providers? How do I choose one?

In order to have a complete Usenet guide and answer the question “What is Usenet” while doing a good comparison for Usenet vs torrents, we need to learn about Usenet providers. Maintaining the comparison for Usenet vs torrens, we can say that the Usenet providers are really similar to Torrent trackers. The Usenet provider gives you access to a server with content ready to download, which is similar to torrent trackers, but there are some differences between them.
When using torrents, you are downloading a .torrent file that points to the file you want to download. The same happens on Usenet, where you download a .nzb file. While the .nzb file points directly to the Usenet Server, the .torrent file only points to the tracker, then the client looks for seeds from where to download the files.

Here are the most important aspects to have in mind when choosing a Usenet Provider:

1. Allowed traffic per month

When choosing a Usenet provider, probably the most important aspect is the traffic included in the subscription. This is the amount of data you are allowed to download from the Usenet Server. Some Usenet providers have data caps (ex: you are allowed to download 10GB/month). Other providers offer unlimited data transfers which means that you can download as much as you want without having to limit your consumption.

2. Retention

As previously mentioned, Usenet servers usually have limited storage capabilities, meaning that files need to be periodically deleted in order to add new files. Retention is the number of days for the files to be available on the server. Newshosting is offering 3248 days of retention which should be more than enough for most users.

3. Simultaneous connections

Another important aspect in our Usenet guide on choosing a Usenet provider is simultaneous connections. This is important for the speed you are downloading the files from the server. Usually, files are split into multiple smaller files which are downloading at the same time to improve download speed. Each part of this file uses one of your available connections, so for example, if your Usenet Provider allows 30 simultaneous connections, you can download a file split into 30 parts at maximum speed.
Please note that if you don’t have a good Internet connection, this aspect may not be that important in your case. Also, if you have a fast internet connection, you will need a fast drive that is able to keep up with the amount of data downloaded at the same time (ex: an SSD).

4. Security

While most of the Usenet providers offer secured connection with SSL encryption, there are other cheap providers that are not securing your connection to the Usenet server. Because of this, you will be required to use a VPN connection to secure your traffic from your internet service provider which will add additional costs. To avoid this, make sure you choose a Usenet Provider with SSL encryption to save yourself from spending more money and to keep your privacy.

5. Server location

For users with good Internet speeds, the server location is another important aspect to consider when choosing a Usenet provider. It is important to choose a Usenet provider that has server farms in your region. For example, it’s not the best idea to choose a provider that has servers in the US while you are located in Europe because the speeds will be lower.

6. Other benefits

Providers like Newshosting offer a free VPN service included in the monthly subscription for Usenet access. If you want to protect yourself from traffic interception, you will want to use a VPN even if you are not downloading files, and only browse the internet. This aspect can help you make a better decision when comparing different plans offering different benefits. You may pay a bit more for a more expensive plan for Usenet, but if you are already planning to also subscribe to a VPN service, then it’s probably cheaper to only buy the top plan from the Usenet Provider