Content delivery and distributed file sharing networks

There are a variety of technologies for increasing the efficiency of files and other content to which many users want access. Some of these technologies have come into conflict with laws and business concerns about intellectual property rights, although it must be made clear that virtually all the conflicts involve the content being transferred, not the means of transfer.

The means of transfer differ in the degree of distribution, ranging from serverless peer-to-peer (P2P), to peer-to-peer services with servers playing the role of broker to find desired content, to content distribution networks (CDN) with distributed servers owned by the content provider or a CDN contracted to it. Distributed information sharing is yet another paradigm that addresses some of the same issues as traditional file transfer, and evolving distributed file systems such as the Network File System (NFS). In distributed information sharing, there is far less of an emphasis on client/server than in other protocols. It is more similar to the Network News Transfer Protocol (NTTP).

Legal, ethical, and business issues

There are distinct security problems with some of the distributed models, and, indeed, some were shut down under legal pressure. The legal pressure came mostly from the entertainment industry, not amused with technology that enabled easy, royalty-free transfer of copyrighted music.

Needless to say, there was a conflict between the copyright holders of entertainment content, and music users who believed music was overpriced. A portion of participants in P2P networks indeed were copyright holders, who believed that the ability to sample tracks would get new customers to buy albums or attend live shows. The legality and morality of these practices are beyond the scope of this article, other than to notice they created much network demand and, indeed, new technologies. It is worth noting that several P2P networks have closed, or had major sites refuse to download their software.

All open implementations that can distribute files can be used to distribute both legal and illegal information, just as a photocopier can copy unauthorized material. At the same time, some large entertainment corporations have been extremely aggressive in legal attack, to the extent of creating chilling effect on legitimate users, demanding expensive and possibly intrusive technologies to be implemented by Internet Service Providers, and even suggesting that digital rights enforcement be built into standard integrated circuits that might be used for a purpose totally different than entertainment.

College network administrators, in particular, found music downloads to take seemingly infinite amounts of bandwidth, interfering with their academic and research methods. Some administrators simply blocked ports used for file sharing, or content detected as multimedia (e.g., MP3). This was an issue of protecting resources, which handily countered arguments about academic freedom and censorship.

Ironically, the no longer extant Napster was used, at one point, as a test load for Internet2. Commercial BitTorrent works best over broadband dedicated access, not dialup, and has been as advanced in its relationships with entertainment content owners as with the P2P technology, as seen in the Torrent Entertainment Network.

Of course, academic institutions tend to have large numbers of computer-literate individual with time and resources, so the situation began to resemble the military minuet of electronic warfare: electronic attack, countermeasure, and response by counter-countermeasures. It is noteworthy that the military has generalized to the concept of information operations, which includes electronic warfare, but also attack, countermeasure, and counterattack on computers and computer networks.

P2P users, especially with questionable legal positions, might encrypt content, defeating MP3 detection, and send it to a nonstandard port, or just try nonstandard (or dynamically changing) ports until one was blocked.

Content delivery networks (CDN)

Akamai pioneered content distribuion, using a push technology with content providers as their customers. They pushed the content to local caches at ISPs, which could deliver content to end users with less upstream bandwidth than the ISPs had previously needed, as well as incentives to the ISPs.

They have evolved their service to provide various edge accelerators, some intended as products and some as Akamai-run services. Time-Warner, for example, took over content distribution for CNN after the acquisition.

Torrent offers a CDN with a different business model.

P2P with assistance in finding information=

There are a wide range of P2P services whose relationships are established by personal contact and social networks. Their content may be legal or illegal to transfer.

Still, finding content can be a major challenge without purpose-built assistance. While it has shut down, Napster was the first widely available "peer to peer" service. It was not true point-to-point as was the original NetNews, because it did have servers to help mediate content exchanges, although the actual exchange was P2P.

Kazaa

Kazaa also is a system with "matchmaking" servers and direct P2P transfer between user machines. As with other programs, it had its assortment of legal problems with copyright. Helping Kazaa was a ruling from a Dutch court that it was, essentially, a carrier not responsible for the acts of its users.

BitTorrent

BitTorrent is a hybrid between pure peer-to-peer, and varying levels of server involvement. The servers are more concerned with directory functions and managing content flows that actually holding the content. BitTorrent seems to have been much more successful than other P2P companies in establishing working relationships with the entertainment industry, avoiding the adversarial relationships that closed down other P2P systems. When the protocol first deployed, it was at end user quality levels. Evolved versions can meet enterprise quality needs. There is also a content distribution service, based on BitTorrent technology but as opposed to P2P, with contractual SLAs. CacheLogic runs this service. BitTorrent claims that its traffic is 35% of the total Internet traffic, although this has not been independently confirmed.

The original BitTorrent client, written in Python, had its source code released under the BitTorrent Open Source License, which is a modified version of the Jabber Open Source License. There are numerous compatible clients, written in a variety of programming languages, and running on a variety of computing platforms.

While BitTorrent implements are called clients, the protocol operations are not strongly client-server. NetNews is a conceptual predecessor, in that the clients exchange information about what they have and what they need. As opposed to news, however, the exchanges are not pairwise, but as parallel as possible. The more clients that have the file of interest, the smaller the workload of each. Each client in the “swarm” of cooperating computers knows which other pieces of the file have been transferred.

After downloading a file, the BitTorrent netiquette is to act as a seeder after it has the complete file. This, it offers the file to other clients. Being able to expand the number of sources of content led to the marketing-oriented but meaningful term "application multicasting".

As important as the share ratio may be the length of time the client stays open. The longer that time, the more new clients can find a peer that has that file.

How do would-be downloaders find the clients with the files? The first client to get the file and offer it, called the initial seeder, registers its existence and content with a server called a tracker. Trackers make the information about seeders and trackers available as web pages, but, unless they are coincidentally seeders/clients, do not themselves have the content. They are merely brokers.

In NNTP, the client requests specific messages, but here, they request random pieces of the content file. Randomness increases the probability that more than one client will have the file, such that the work is spread over more computers. The larger the file, the larger the piece. A hash for error detection covers the pieces.

As more peers join the swarm, they share pieces with one another, reducing the load on the initial seeder. Clients should prefer to send data to peers who upload data to them, a means of encouraging fair sharing of the work. If this policy is overly strict, however, a peer that just joined the swarm may not get any data because it is not offering any.

Authorized use of BitTorrent

Where several P2P vendors have gone out of business, typically due to huge legal problems with pirated information, BitTorrent seems to have managed to stay more mainstream. BitTorrent distributes BitTorrent's own updates, as well as OpenOffice.org and most LINUX distributions. Free and Open Source conferences have distributed video of their conferences via BitTorrent. NASA uses it for large image files.

Bram Cohen, the inventor of BitTorrent, and the movie industry executed an agreement in 2005. It required that the bittorrent.com website not have any linked to unlicensed movie content. Of course, this had no effect on other servers not under Cohen's control, but it appears there is a mutual attempt to work out compromises between this P2P operator and the entertainment industry than with several other services.

BitTorrent has received the usual legal attacks over copyright infringement, but its main counterargument is that its servers do not contain copyrighted information, but only metadata about it. Nevertheless, the matter is before several courts at the time of this writing. Law enforcement, in several countries, seized servers. BitTorrent websites, perhaps more than for other P2P services, tend to have prominent anti-piracy warnings. Other BitTorrent servers not subject to this agreement continue to be shut down through legal action. BitTorrent's design does give it some legal protections that other P2P services do not have. BitTorrent itself does not have a search engine, so clients are responsible for finding the material.