Binary content

Usenet was originally created to distribute text content encoded in the 7-bit ASCII character set. With the help of programs that encode 8-bit values into ASCII, it became practical to distribute binary files content. Binary posts, due to their size and dubious copyright status, were in time restricted to specific newsgroups, making it easier for administrators to allow or disallow the traffic.

The oldest widely used encoding method is uuencode, from the Unix uucp package. In the late 1980s Usenet articles were often limited to 60,000 characters, and larger hard limits exist today. Files are therefore commonly split into sections that require reassembly by the reader.

With the header extensions and the Base64 and Quoted-Printable MIME encodings, there was a new generation of binary transport. In practice, MIME has seen increased adoption in text messages, but it is avoided for most binary attachments. Some operating systems with metadata attached to files use specialized encoding formats. For Mac OS, both Binhex and special MIME types are used.

Other lesser known encoding systems that may have been used at one time were BTOA, XX encoding, BOO, and USR encoding.

In an attempt to reduce file transfer times, an informal file encoding known as yEnc was introduced in 2001. It achieves about a 30% reduction in data transferred by assuming that most 8-bit characters can safely be transferred across the network without first encoding into the 7-bit ASCII space.

The standard method of uploading binary content to Usenet is to first archive the files into RARParchive files. Parity files are used to recreate missing data. This is needed often, as not every part of the files reach a server. These are all then encoded into yEnc and uploaded to the selected binary groups. archives (for large files usually in 20 MB or 50 MB parts) then create

Technical details

Usenet is a set of protocols for generating, storing and retrieving news "articles" (which resemble Internet mail messages) and for exchanging them among a readership which is potentially widely distributed. These protocols most commonly use a flooding algorithm which propagates copies throughout a network of participating servers. Whenever a message reaches a server, that server forwards the message to all its network neighbors that haven't yet seen the article. Only one copy of a message is stored per server, and each server makes it available on demand to the (typically local) readers able to access that server. Usenet was thus one of the first peer-to-peer applications, although in this case the "peers" are themselves servers that the users then access, rather than the users themselves being peers on the network. However, the end users connects to the server of the service provider and are therefore not interacting with each other, opposed the common meaning of a peer-to-peer network.

RFC 850 was the first formal specification of the messages exchanged by Usenet servers. It was superseded by RFC 1036.

In cases where unsuitable content has been posted, Usenet has support for automated removal of a posting from the whole network by creating a cancel message, although due to a lack of authentication and resultant abuse, this capability is frequently disabled. Copyright holders may still request the manual deletion of infringing material using the provisions of World Intellectual Property Organization treaty implementations, such as the U.S. Online Copyright Infringement Liability Limitation Act.

On the Internet, Usenet is typically on TCP Port 119.

Moderated and unmoderated newsgroups

A minority of newsgroups are moderated. That means that messages submitted by readers are not distributed to Usenet, but instead are emailed to the moderators of the newsgroup, for approval. Moderated newsgroups have rules called charters. Moderators are persons whose job is to ensure that messages that the readers see in newsgroups conform to the charter of the newsgroup. Typically, moderators are appointed in the proposal for the newsgroup, and changes of moderators follow a succession plan.

The job of the moderator is to receive submitted articles, review them, and inject approved articles so that they can be properly propagated worldwide. Such articles must bear the Approved: header line.

Unmoderated newsgroups form the majority of Usenet newsgroups, and messages submitted by readers for unmoderated newsgroups are immediately propagated for everyone to see.

Creation of moderated newsgroups often becomes a hot subject of controversy, raising issues regarding censorship and the desire of a subset of users to form an intentional community.

ISPs, news servers, and newsfeeds

Internet service providers, and many other Internet sites, operate news servers for their users to access. ISPs that do not operate their own servers directly will often offer their users an account from another provider that specifically operates newsfeeds. Most commonly, these accounts are through UsenetServer.com, Supernews, Giganews and Usenet.com. Usually the ISP will get a kickback for referring the customer to the Usenet provider. In early news implementations, the server and newsreader were a single program suite, running on the same system. Today, one uses separate newsreader client software, a program that resembles an email client but accesses Usenet servers instead.

Not all ISPs run news servers. A news server is one of the most difficult Internet services to administer well because of the large amount of data involved, small customer base (compared to mainstream Internet services such as email and web access), and a disproportionately high volume of customer support incidents (frequently complaining of missing news articles that are not the ISP's fault). Some ISPs outsource news operation to specialist sites, which will usually appear to a user as though the ISP ran the server itself. Many sites carry a restricted newsfeed, with a limited number of newsgroups. Commonly omitted from such a newsfeed are foreign-language newsgroups and the alt.binaries hierarchy which largely carries software, music, videos and images, and accounts for over 99 percent of article data.

For those who have access to the Internet, but do not have access to a news server, Google Groups allows reading and posting of text news groups via the World Wide Web. Though this or other "news-to-Web gateways" are not always as easy to use as specialized newsreader software, especially when threads get long, they are often much easier to search. Users who lack access to an ISP news server can use Google Groups to access the alt.free.newsservers newsgroup, which has information about open news servers.

There are also Usenet providers that specialize in offering service to users whose ISPs do not carry news, or that carry a restricted feed. One list of such providers is available at UsenetProviders' list of Usenet providers (Germany) or Jeremy Nixon's list of (paid) Usenet providers.

Introduction

Usenet is one of the oldest computer network communications systems still in widespread use. It was established in 1980, following experiments from the previous year, over a decade before the World Wide Web was introduced and the general public got access to the Internet. It was originally conceived as a "poor man's ARPANET," employing UUCP to offer mail and file transfers, as well as announcements through the newly developed news software. This system, developed at University of North Carolina at Chapel Hill and Duke University, was called USENET to emphasize its creators' hope that the USENIX organization would take an active role in its operation .

The articles that users post to Usenet are organized into topical categories called newsgroups, which are themselves logically organized into hierarchies of subjects. For instance, sci.math and sci.physics are within the sci hierarchy, for science. When a user subscribes to a newsgroup, the news client software keeps track of which articles that user has read.

In most newsgroups, the majority of the articles are responses to some other article. The set of articles which can be traced to one single non-reply article is called a thread. Most modern newsreaders display the articles arranged into threads and subthreads, making it easy to follow a single discussion in a high-volume newsgroup.

When a user posts an article, it is initially only available on that user's news server. Each news server, however, talks to one or more other servers (its "newsfeeds") and exchanges articles with them. In this fashion, the article is copied from server to server and (if all goes well) eventually reaches every server in the network. The later peer-to-peer networks operate on a similar principle; but for Usenet it is normally the sender, rather than the receiver, who initiates transfers. Some have noted that this seems an inefficient protocol in the era of abundant high-speed network access. Usenet was designed for a time when networks were much slower, and not always available. Many sites on the original Usenet network would connect only once or twice a day to batch-transfer messages in and out.

Usenet has significant cultural importance in the networked world, having given rise to, or popularized, many widely recognized concepts and terms such as "FAQ" and "spam."

Today, almost all Usenet traffic is carried over the Internet. The current format and transmission of Usenet articles is very similar to that of Internet email messages. However, Usenet articles are posted for general consumption; any Usenet user has access to all newsgroups, unlike email, which requires a list of known recipients.

Today, Usenet has diminished in importance with respect to Internet forums, blogs and mailing lists. The difference, though, is that Usenet requires no personal registration with the group concerned, that information need not be stored on a remote server, that archives are always available, and that reading the messages requires not a mail or web client, but a news client (included in many modern e-mail clients).

Usenet

Usenet (a contraction of user network) is a global, decentralized, distributed Internet discussion system that evolved from a general purpose UUCP architecture of the same name. It was conceived by Duke University graduate students Tom Truscott and Jim Ellis in 1979. Users read and post public messages (called articles or posts, and collectively termed news) to one or more categories, known as newsgroups. Usenet resembles bulletin board systems (BBS) in most respects, and is the precursor to the various web forums which are widely used today. Discussions are threaded, with modern news reader software, as with web forums and BBSes, though posts are stored on the server sequentially.

One notable difference from a BBS or web forum is that there is no central server, nor central system owner. Usenet is distributed among a large, constantly changing conglomeration of servers which store and forward messages to one another. These servers are loosely connected in a variable mesh.^[clarify] Individual users usually read from and post messages to a local server operated by their ISP, university or employer. The servers then exchange the messages between one another, so that they are available to readers beyond the original server.