TweetFollow Us on Twitter

May 01 MacTech Online

Volume Number: 17 (2001)
Issue Number: 05
Column Tag: MacTech Online

by Jeff Clites online@mactech.com

The users of Napster couldn’t possibly have known what they were really doing. They thought they were just trading some music. The recording industry thought they were stealing. But what they were really doing, of course, was networking. Peer-to-peer networking, actually. In truth, peer-to-peer (or “p2p”) networking is nothing new, but it has taken a product like Napster, with its popularity and legal controversy, to bring it into the limelight. Now, not only are there new p2p applications, but there are articles, web sites, books, and conferences springing up. Like other “hot” technologies (Java and XML come to mind), there is a great deal of hype surrounding p2p — but there is also substance, if you dig deep enough.

As a side effect of the recent interest in p2p, there has also been a great deal of focus on defining what exactly constitutes a p2p technology. At its most basic, p2p networking is just client-server networking in which the machines involved act as clients sometimes and as servers other times. As a trivial example, two machines running FTP servers can act as peers. Some feel that “true” p2p application, in the modern sense, must explicitly allow for the involvement of computers which are not always connected to the internet and whose IP addresses change. But in truth, a precise definition is beside the point, as is the issue of whether a given application qualifies as p2p. What’s more important are the systems and technologies that are being developed, and the uses to which they are being put. At its core, the interesting part of p2p isn’t that it’s peer-based; the interesting aspect is content distribution, and putting this into the hands of end users.

Content Distribution

From its inception, the web has been about content distribution — it was originally designed as a mechanism for sharing research papers. But end users in today’s web are largely passive — most users don’t produce web content, but only view it. But p2p technologies are changing this, and turning end users into active participants, and they’re doing this as a matter of course — for instance, Napster users are automatically servers of MP3 content. This is a theme which recurs in many p2p applications: users play the role of content providers without needing to make an extra effort to do so, and while they are becoming more involved in the Internet they are not necessarily participating in the web, per se — users are more often sharing files than creating web pages.

This month we are going to take a look at Gnutella, one of the many interesting p2p applications springing up today, both for its technological features and for the insights it gives into the p2p “movement”. It’s a file-sharing-focussed p2p application, and a contender to be the successor to Napster. But first we are going to look at Napster itself, and find out why it has turned out to be such an important technology.

Napster as a Paradigm

The most important technological insight of Napster was its judicious mixing of centralization and decentralization. In a nutshell, what Napster adds to the “two FTP servers” scenario is integration with a search engine. (From a sociological point of view, Napster also adds ease of use, and a focus on a specific type of content.) Searches in Napster are preformed via a central server which indexes the music on each of the connected Napster clients. This allows for fast and efficient searching. But actual file transfers are conducted directly between two client machines, without passing through any centralized server. This is key, because it allows the Napster system as a whole to effectively take advantage of the aggregate bandwidth and disk space of all of its users — there is no need for a huge disk farm to store the content, and an unlimited number of transfers between clients can happen simultaneously without impacting the performance of the system as a whole. The decentralized aspects are in a sense the most inspired strength of Napster, and they are really what has sparked interest in peer-to-peer technologies in themselves. The centralized search facilities are in a sense the greatest weakness, but more from a political or legal point of view than from a technological one, because they necessitate a single authority which can be targeted with legal action, and an essential linchpin which can be “attacked” to shut down the entire system. Unsurprisingly, this has led to a trend in new p2p technologies to try to avoid any centralization at all, in order to provide both legal and technological robustness against attempts to shut down any systems developed. This backlash is a mixed blessing, because centralization in and of itself it not always a flaw — some operations are appropriate to centralize, such as searching, and in the absence of worries about “social attacks” this is often the best approach. At the same time, a community obsessed with maximizing the “p” in p2p is producing some interesting systems which are pushing the limits of what would have been thought to be possible in a purely distributed peer system, and is giving us greater insight into what can and cannot be done without centralization. After the legal controversy dies down I think that we will again be designing systems which are a sensible mix of p2p and traditional client-server, but they will be better than they would have been without the war stories from today’s p2p innovators.

Gnutella

Gnutella began at AOL with developers from Nullsoft, the company behind the popular Winamp MP3 player for Windows, and it quickly became on open source project being developed outside of the company. It has been described as the open-source Napster. At the surface this is true, but Gnutella differs from Napster in several important ways.

To begin with, Gnutella is actually a protocol, rather than an application, and there are several different applications for multiple platforms which speak the Gnutella protocol. Like Napster, Gnutella software allows users to perform searches and transfer files, and the same software plays the role of both client and server, both downloading and vending content. The key difference is that Gnutella is completely decentralized — there is no central entity to service queries. When a Gnutella application starts up, it joins the Gnutella network by connecting to another “node” — to any other machine running Gnutella software. In the beginning, users would find another node to connect to through word-of-mouth, or from web sites which posted lists of IP addresses of nodes, but today there are nodes called “host caches” which are set up to vend the locations of active nodes to new users joining the network (one point of slight, and optional, centralization in Gnutella). Once a node has connected to a handful of other nodes, it is part of the network, and equivalent to every other node. This has been described as a software-based network on top of the physical network, and it’s in a constant state of flux as new nodes join and others drop out — as new connections form and old ones are severed, from one minute to the next. When a user want to search the network for a piece of content, his node sends his query to each of its “adjacent” nodes (the nodes to which it is connected), which in turn forward the query to the nodes that they know about, and so on. Each node can then respond to the query by sending its response back along the network to the node which originated the query. In case you missed it, Gnutella queries happen via a broadcast across the network of users, rather than through a central server which indexes all of the content. More about this in a minute. As responses to a query trickle back to a node, if any of them contain the URL of a file that the user want to download, he can download it directly from the machine which sent that response. This is the same strategy employed by Napster, in that the (potentially large) file transfers bypass the greater Gnutella network completely.

In addition to the search strategy, Gnutella differs from Napster in the search content. A Gnutella query is free-form, and each node is independent and free to interpret a query however it likes. Napster treats queries as searches for MP3s, but a Gnutella query could be looking for music, some other type of file, or a different kind of information entirely. Most commonly, a node will treat a query as a request for a file, and will match the query against either file names or file content (the former would be appropriate for an MP3, and the latter for a news article). But to demonstrate the flexibility of this approach, Gnutella developers set up a small network with nodes which each handled queries in a different way — one interpreted them as requests for stock quotes, one matched them against news headlines, and another evaluated them as arithmetic expressions — and provided a web-based search interface to the network. This demonstrated the flexibility of the notion of a query along the network, and also pointed out that private Gnutella networks could easily be set up to serve novel functions. (This type of private Gnutella network is trivial to set up — it’s just a set of nodes which don’t happen to know about any nodes outside of a small set.)

As mentioned above, Gnutella uses a broadcast-based approach to searching. This strategy is a key feature of Gnutella and its decentralization, and it’s arguably its fatal flaw. It’s a flaw because queries take a significant amount of bandwidth across the network, and this poses a direct barrier to scalability. Gnutella’s approach is actually quite strange, once you begin to analyze it, and it’s difficult to decide whether it’s clever or pathological. Although logically a broadcast, queries are passed from node to node via direct TCP/IP connection, rather than via UDP broadcast. Also, despite being TCP-based, the queries are lossy — nodes will discard their responses if querying nodes are reacting too slowly. This saves things somewhat, as it prevents a few slow nodes from clogging the network. Also, queries have a time-to-live (TTL), just like IP packets — typically, Gnutella queries have a TTL of 7, meaning that queries are only transmitted to nodes 7 hops away. On average, a query reaches about 10,000 other nodes.

The Gnutella community is quick to point out that this “distributed query” approach models the way humans interact, transferring knowledge in an ever-changing network of interactions. On the other hand, as nice as the analogy may sound, it might not be the best design strategy, and the comparison breaks down in the details. If you do the math, even with TTLs and lossy responses, the performance of Gnutella networks will degrade and hit a bandwidth wall when the network grows above a certain number of nodes. Steps are being taken to address the “bandwidth issue”, and more sophisticated software is being developed to implement various strategies. Interestingly, this problem is no surprise to Gnutella developers — it was known at the very beginning, and is an inherent problem with broadcasts. Nevertheless, developers persisted in this approach, demonstrating an enthusiasm (or dementia) which has been attendant to p2p development, characterized as a willingness to go ahead and try things which seem “theoretically impossible but practically possible”. This sounds like a rather irrational philosophy for a community of developers, but despite this Gnutella has actually performed better than the naysayers predicted, and has given us a better idea of what is possible with this type of approach. And although the Gnutella community is now forced to do some technological clean up, they are able to address the real problems that have actually arisen, at a point where they know the strengths and weaknesses of their system better than they would have to start out with, and so are in a better position to solve them.

Further Reading

If you want to learn more about the ever-changing p2p landscape, and about Gnutella in particular, there are several great places to start. O’Reilly’s OpenP2P portal site contains original content as well as pointers to articles on other sites, and has a Gnutella section which can help you keep up to speed as the technology develops. O’Reilly also has a recently-published book, Peer-to-Peer: Harnessing the Power of Disruptive Technologies (ISBN: 0-596-00110-X), with articles on specific p2p technologies as well as articles covering general topics which are relevant across applications. In particular, check out “Gnutella: Alive, Well, and Changing Fast” and “Gnutella and the Transient Web” here, and “The Gnutella paradox” on Salon.com. Also, the Gnutella web site at wego.com is a portal for both Gnutella developers and users, and Gnutelliums maintains information on all of the Gnutella clients available for various platforms. If you are interested in developing p2p software, or just in figuring out how certain things are done, the source code to these clients (often licensed under the GPL) is a great resource. It’s often included with the client distributions, or you can find much of it at GnutellaDev. And finally, if you plan on doing any Gnutella development, you’ll need to know the details of the protocol, which you can find at Gnutelliums and at Clip2.

OpenP2P.com
http://www.openp2p.com/

OpenP2P.com: Gnutella
http://openp2p.com/topics/p2p/gnutella/

Gnutella: Alive, Well, and Changing Fast
http://www.openp2p.com/pub/a/p2p/2001/01/25/truelove0101.html

Gnutella and the Transient Web
http://www.openp2p.com/pub/a/p2p/2001/03/22/truelove.html

The Gnutella paradox
http://www.salon.com/tech/feature/2000/09/29/gnutella_paradox/

Gnutella portal page
http://gnutella.wego.com/

Gnutelliums
http://www.gnutelliums.com/

GnutellaDev
http://www.gnutelladev.com/

The GnutellaNet protocol at MROB
http://www.gnutelliums.com/linux_unix/gnut/doc/gnutella-prot.html

The Gnutella Protocol Specification v0.4
http://dss.clip2.com/GnutellaProtocol04.pdf

Lessons from Gnutella and Napster

The genesis and growth of Gnutella and Napster have a lot to tell us about both the technological and the sociological aspects of new technologies. In particular, Napster is popular not because it is p2p, but because people really like music — it's the content that's really important to the end user, not the means to get there, and it just happened that in Napster's case p2p was the best way to deliver this. I also find it telling that Napster isn't web browser based, but instead uses its own standalone client. This confounds the prevailing wisdom today that everything needs to live in a browser or the public won't use it. In Napster's case, it really wasn't technologically feasible to work inside of a browser, and I think it would have been a disaster if they had tried. The important lesson here is that if the content is compelling enough, and the technology delivers it well, then users are perfectly happy downloading a new piece of software to use. They really don't need to use Netscape or IE for everything, and I hope that developers realize this as they are planning for the next generation of applications.

In a way, Gnutella is a result of embracing the technology of Napster and inferring a set of ideals from it, rather than understanding what really has made it popular. On the other hand, there's really nothing wrong with that, and I don't think the Gnutella community would mind being accused of putting the cart before the horse. Not everything has to be the basis for a business plan, after all. Enthusiasm for the idea of p2p and a willingness to experiment are improving the underlying technology, and showing us what is possible, and in some cases how simply things can be done. And in the end, people are having fun with it — both developers and users. The "let's just try it and see what happens" approach is working out pretty well in fact. So when a business plan does come along, they'll be ready.

 
AAPL
$112.65
Apple Inc.
+3.24
MSFT
$47.52
Microsoft Corpora
+1.78
GOOG
$511.10
Google Inc.
+6.21

MacTech Search:
Community Search:

Software Updates via MacUpdate

BBEdit 11.0.2 - Powerful text and HTML e...
BBEdit is the leading professional HTML and text editor for the Mac. Specifically crafted in response to the needs of Web authors and software developers, this award-winning product provides a... Read more
ExpanDrive 4.2.1 - Access cloud storage...
ExpanDrive builds cloud storage in every application, acts just like a USB drive plugged into your Mac. With ExpanDrive, you can securely access any remote file server directly from the Finder or... Read more
Adobe After Effects CC 2014 13.2 - Creat...
After Effects CC 2014 is available as part of Adobe Creative Cloud for as little as $19.99/month (or $9.99/month if you're a previous After Effects customer). After Effects CS6 is still available... Read more
Command-C 1.1.7 - Clipboard sharing tool...
Command-C is a revolutionary app which makes easy to share your clipboard between iOS and OS X using your local WiFi network, even if the app is not currently opened. Copy anything (text, pictures,... Read more
Tidy Up 4.0.2 - Find duplicate files and...
Tidy Up is a complete duplicate finder and disk-tidiness utility. With Tidy Up you can search for duplicate files and packages by the owner application, content, type, creator, extension, time... Read more
Typinator 6.3 - Speedy and reliable text...
Typinator turbo-charges your typing productivity. Type a little. Typinator does the rest. We've all faced projects that require repetitive typing tasks. With Typinator, you can store commonly used... Read more
GraphicConverter 9.5 - Graphics editor w...
GraphicConverter is an all-purpose image-editing program that can import 200 different graphic-based formats, edit the image, and export it to any of 80 available file formats. The high-end editing... Read more
Toast Titanium 12.0.1 - The ultimate med...
Toast Titanium goes way beyond the very basic burning in the Mac OS and iLife software, and sets the standard for burning CDs, DVDs, and now Blu-ray discs on the Mac. Create superior sounding audio... Read more
QuickBooks 2015 16.0.2.1422 R3 - Financi...
Save 20% on QuickBooks Pro for Mac today through this special discount link QuickBooks Pro 2013 helps you manage your business easily and efficiently. Organize your finances all in one place, track... Read more
Remotix 3.0.6 - Access all your computer...
Remotix is a fast and powerful application to easily access multiple Macs (and PCs) from your own Mac. Features: Complete Apple Screen Sharing support - including Mac OS X login, clipboard... Read more

Latest Forum Discussions

See All

Mahjong Journey Review
Mahjong Journey Review By Jennifer Allen on December 18th, 2014 Our Rating: :: STEADY MATCHINGiPad Only App - Designed for the iPad Aimed at the more laid back gamer, Mahjong Journey isn’t for everyone, but those looking for some... | Read more »
Emoji Type - custom keyboard with predic...
Emoji Type - custom keyboard with predictive emojis 0.4.0 Device: iOS iPhone Category: Utilities Price: $.99, Version: 0.4.0 (iTunes) Description: Emoji Type is custom keyboard for iOS 8 that auto suggests emojis as you type. ABOUT... | Read more »
Game of the Year 2014 – 148Apps Staff Pi...
The end of 2014 is almost here, which can only mean one thing. Okay it can mean a lot of things, but in this specific context it means Game of the Year lists! Which is why the 148Apps staff have all picked their favorites from the past year. And why... | Read more »
UponPixels Review
UponPixels Review By Jennifer Allen on December 18th, 2014 Our Rating: :: CREATIVE TYPOGRAPHYUniversal App - Designed for iPhone and iPad Add cool typography and objects to your photos with the easy to use UponPixels.   | Read more »
The Vikings are Coming! CastleStorm’s Ne...
The Vikings are Coming! CastleStorm’s New Update Adds a Survival Mode Posted by Jessica Fisher on December 18th, 2014 [ permalink ] | Read more »
Duet Display (Productivity)
Duet Display 0.3.3 Device: iOS Universal Category: Productivity Price: $9.99, Version: 0.3.3 (iTunes) Description: Duet Display allows you to use your iPad or iPhone as an extra display. Developed by a team of ex-Apple engineers,... | Read more »
Dragon Quest III Review
Dragon Quest III Review By Jennifer Allen on December 18th, 2014 Our Rating: :: DATED BUT NOT WITHOUT MERITUniversal App - Designed for iPhone and iPad A walk down memory lane isn’t foolproof for Dragon Quest III, but it has its... | Read more »
8 KEMCO JRPGs Are Just $0.99 in Celebrat...
8 KEMCO JRPGs Are Just $0.99 in Celebration of the Holidays Posted by Jessica Fisher on December 18th, 2014 [ permalink ] KEMCO has announc | Read more »
Brothers in Arms 3: Sons of War Review
Brothers in Arms 3: Sons of War Review By Jennifer Allen on December 18th, 2014 Our Rating: :: FUN BUT PUSHYUniversal App - Designed for iPhone and iPad Brothers in Arms 3: Sons of War could be great fun, but its plethora of... | Read more »
The Snow is Falling on the French Countr...
The Snow is Falling on the French Countryside of Carcassonne’s Newest Update Posted by Jessica Fisher on December 18th, 2014 [ permalink ] < | Read more »

Price Scanner via MacPrices.net

Save up to $400 on MacBooks with Apple Certif...
The Apple Store has Apple Certified Refurbished 2014 MacBook Pros and MacBook Airs available for up to $400 off the cost of new models. An Apple one-year warranty is included with each model, and... Read more
Save up to $300 on Macs, $30 on iPads with Ap...
Purchase a new Mac or iPad at The Apple Store for Education and take up to $300 off MSRP. All teachers, students, and staff of any educational institution qualify for the discount. Shipping is free,... Read more
iOS and Android OS Targeted by Man-in-the-Mid...
Cloud services security provider Akamai Technologies, Inc. has released, through the company’s Prolexic Security Engineering & Research Team (PLXsert), a new cybersecurity threat advisory. The... Read more
KMI MIDI K-Board Great Gift for Amateur &...
The K-Board is a MIDI Nano keyboard for music creation for iPad, Android, And computers; the easiest way to make music with iPads & Android tablets, and Mac, Windows, or Linux computers. Ultra-... Read more
Amazon offers 15-inch 2.2GHz Retina MacBook P...
 Amazon.com has the 15″ 2.2GHz Retina MacBook Pro on sale for $1699 including free shipping. Their price is $300 off MSRP. Stock is limited, so act now if you’re interested. Read more
Holiday sales continue: MacBook Pros for up t...
 B&H Photo has new MacBook Pros on sale for up to $300 off MSRP as part of their Holiday pricing. Shipping is free, and B&H charges NY sales tax only: - 15″ 2.2GHz Retina MacBook Pro: $1699... Read more
Holiday sale: Mac minis available for up to $...
 B&H Photo has new 2014 Mac minis on sale for up to $80 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 1.4GHz Mac mini: $459 $40 off MSRP - 2.6GHz Mac mini: $629 $70 off... Read more
Google Search App For iOS Gets A Major Makeov...
Google has given iOS users an early Christmas present with a substantial update of it’s not-very-often-upgraded Google Search app. Google Search has been my go-to tool for Web searches since it was... Read more
ShopKeep Apple Pay And Chip Card Reader Avail...
ShopKeep, a cloud-based technology provider to more than 10,000 small business owners to manage retail shops and restaurants with iPads, has released its new Apple Pay and chip card reader. This... Read more
Holiday sale! 27-inch 5K iMac for $2299, save...
 B&H Photo has the 27″ 3.5GHz 5K iMac in stock today and on sale for $2299 including free shipping plus NY sales tax only. Their price is $200 off MSRP, and it’s the lowest price available for... Read more

Jobs Board

*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
*Apple* Retail - Multiple Positions (US) - A...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
Project Manager / Business Analyst, WW *Appl...
…a senior project manager / business analyst to work within our Worldwide Apple Fulfillment Operations and the Business Process Re-engineering team. This role will work Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.