Volume Number: 21 (2005)
Issue Number: 11
Column Tag: Programming

The Source Hound

Bonjour, Mon ami: Automatic Service Discovery in Tiger

by Dean Shavit

When Apple released Mac OS X 10.2 in August of 2002, it included a somewhat obscure networking technology dubbed Rendezvous. Nearly than three years later with the release of OS X 10.4 Tiger, Mac users have said au revoir to Rendezvous and hello to Bonjour because of a trademark infringement lawsuit against Apple by Tibco Software. The new name for Rendezvous was to be OpenTalk, which had a familiar, warm Classic Mac OS ring, but instead, we walk around with a frog in our throat, lips shaped like the Texas border, trying to emulate an uncomfortable accent. Every time I say the B-word I have visions of Pepe LePew sticking his head out of a doorway and exclaiming, "Bonjour, mon amour, embrasse-moi." Freaky.

Yet the renewed focus on this taken-for-granted technology, no matter how odd the name or the freely associated cartoon character, reveals that it has become such a useful and ubiquitous helpmate that it's not difficult to make the argument that without Bonjour, OS X would be a lesser experience for everyone from the casual iPod user to the Education Systems Administrator with dozens of XServes and XServe RAIDs to manage. For example, the requisite RS-232 serial connection required to configure nearly every storage subsystem on the market from a SCSI array, NAS or high-end switch, is easily dispensed with in favor of Bonjour. Just plug a PowerBook into the Ethernet port on the back of the XServe RAID, and in seconds they're talking, all with no manual IP configuration, and no serial port. While web-based configuration is the norm for such devices, a trip to the serial port is almost always required to configure the IP address of the web interface first.

Zero, My Hero, How Wonderful You Are...

By the subhead above, I'm probably giving away my age. I grew up with the Schoolhouse Rock cartoons on Saturday morning television. I was especially a fan of Multiplication Rock, where the multiplication tables were brought to life as cartoon characters. Zero, of course, was the ultra-powerful multiplier, portrayed as a superhero, complete with mask and cape. If you've been reading this column for a while, you'll know that while I might appear to have cartoons on the brain, there's usually a point, at least an intersection of Bullwinkleish meaning with something Open-Source. At the core of Bonjour is an Open Source (surprised?) technology known as Zeroconf (http://www.zeroconf.org).

One of the Classic Mac OS's claims to fame (and undoing) was the ease of use and automatic discovery of network shares and printers via the AppleTalk networking protocol. While AppleTalk was excellent for small groups of Macs, running it on larger networks and over WAN connections required special planning, hardware, and "seed routers." Contemporaries of AppleTalk, NetBIOS and Novell's IPX (Internetwork Packet Exchange) also provided facilities for discovery of network resources via broadcast. In the end, it was that broadcasting and lack of compatibility with TCP/IP, which became the darling standard of corporate networks in the mid-1990s, not just the Internet, that doomed AppleTalk to a deprecated protocol. Now, with Tiger, file sharing over AppleTalk isn't just deprecated; it doesn't work at all. When Leopard's released in late 2006 or early 2007, I wouldn't be surprised if AppleTalk wasn't supported for printing as well as file sharing.

The Zeroconf project outlines the following requirements in achieving what it calls the "AppleTalk ease-of-use in IP":

• Allocate addresses without a DHCP server (IPv4 Link-Local Addressing)
• Translate between names and IP addresses without a DNS server (Multicast DNS)
• Find services, like printers, without a directory server (DNS Service Discovery)
• Allocate IP Multicast addresses without a MADCAP server (future work)

A final requirement is that the solutions in the four areas must coexist gracefully with larger configured networks. Zeroconf protocols MUST NOT cause harm to the network when a machine is plugged into a large network.

It is important to understand that the purpose of Zero Configuration Networking is not solely to make current personal computer networking easier to use, though this is certainly a useful benefit. The long-term goal of Zero Configuration Networking is to enable the creation of entirely new kinds of networked products, products that today would simply not be commercially viable because of the inconvenience and support costs involved in setting up, configuring, and maintaining a network to allow them to operate.

The idea here is to allow machines of disparate operating system to easily "find" each other without needing to configure a network interface. The example given at the Zeroconf site is of two people wanting to play a networked computer game. If both are using PowerBooks, it ought to be a cinch to use AppleTalk for the two opponents to find each other, but if one's using a Windows laptop, then TCP/IP would be the only common network protocol (At least prior to OS X), and in the absence of a DHCP server for the players' home network, manual IP configuration would be a requirement for game play, and that's something many casual computer users still find somewhat difficult, and a process that game publishers really can't afford to support with their in-house staff. Part of the solution has been to use "tracker" servers to register game players, but that doesn't really address the issue of an ad-hoc network of two gamers who might not have an Internet connection at their disposal.

It's pretty much taken for granted now that physically connected computers (this includes those on wireless--Airport--networks as well) can discover each other's services. The Sony PSP (PlayStation Portable) features ad-hoc 802.11b support, the same as Apple's original Airport implementation that allows players running the same game to "find" each other when in range.

Life Without a Serial Port

"Entirely new kinds of networked products. . ." is exactly the promise that the XServe RAID fulfills in allowing itself to communicate without an assigned IP address over an Ethernet connection, as well as an XServe in headless setup mode. Airport base stations, iTunes music libraries, iPhoto libraries and more just show up to those looking for such services. Not surprisingly, the very first devices to take advantage of Zeroconf capabilities were network printers, which are often a network admin's worst nightmare when it comes to assigning static IP addresses. There's nothing more frustrating than standing over a hard-to-read LCD screen tapping on little buttons that require you to cycle from zero to 255 just because you can't go down as well as up. While printers have gotten better in that department (HP, that means you), it's not unusual to run into printers left pulling IP addresses via DHCP because someone doesn't want to navigate the counter-intuitive menu tree on a tiny LCD screen.

Much of Bonjour's zero configuration magic lies in what's called "IPv4 Link-Local Addressing," which consists of IP addresses in the 169.254.x.x network range. Regardless of whether a Mac or another device on a network has either a manually assigned or DHCP assigned IP address, all OS X computers maintain a 169.254.x.x Link-Local address in their routing table in the event that a device with Link-Local addressing only shows up on the network, first polling other machines to make sure it gets a unique address. Using the nestat -r command to dump the routing table to standard output reveals the Link-Local network destination:

Internet:
Destination       Gateway              Flags   Refs         Use      Netif      Expire
default           192.168.0.1          UGSc      13           6        en0
127               localhost            UCfS       0           0        lo0
localhost         localhost            UH        41      340679        lo0
169.254           link#4               UCS        1           0        en0
169.254.113.141   link#4               UHLW       1         889        en0
192.168.0         link#4               UCS        6           0        en0
192.168.0.1       0:0:94:83:34:68      UHLW      13           0        en0        1112
Dual2Ghz          localhost            UHS        0          19        lo0
192.168.0.255     link#4               UHLWb      1        6089        en0

Should the DHCP server on the network fail, it's likely that Mac OS X clients could continue printing and even mounting file server volumes via Bonjour. To see if a computer is using a Link-Local address instead of an assigned IP address, check the interface in the network preferences.

Even with this self-assigned Link-Local IP address (which is usually taken as a sign of trouble by those entrusted to maintain networks), it's still possible to do business as usual, provided that clients connect to the Bonjour name of the server, which is set to "Computername.local" by default.

Many Windows 2000 and 2003 Server networks are mistakenly set up with an internal DNS domain ending in .local, which of course interferes with Bonjour service discovery. While it makes some sense to some (not to me) to use an internal domain name that's "not real," (mostly because people are of afraid of, or don't understand how DNS works) it might be a better idea to use .lan or .internal instead of .local. If it becomes apparent that the Bonjour .local extension is interfering with Windows Active Directory services. it's easy to permanently (but not irreversibly) disable Bonjour via the following launchd command:

launchctl unload -w /System/Library/LaunchDaemons/com.apple.mDNSResponder.plist

mDNSResponder

While Bonjour might exist on most TCP/IP networks in the shadows of the routing table, or as a stepping-stone to help facilitate connections between Macs with DHCP or manually assigned IP addresses, it's also very useful in other situations, such as mass deployments, in conjunction with Apple's new multicast ASR (Apple Software Restore) capability. For example, when rolling out many new computers, it's sometimes desirable to simply plug them into an isolated switch, boot them from a custom (normally hidden) boot partition, install CD or DVD, and start a multicast restore process with Bonjour enabled. All that's needed is the Bonjour name of the multicast server Mac.

One of the reasons I wrote Mac HelpMate was to make the multicast ASR process easier to configure, tweak, and manage. Note that the ASR URL (asr://mostsvr.local) uses Bonjour to locate the IP address of the server hosting the multicast data stream. Each client simply needs to know that URL, and that's all. With a little extra work, it might be possible to add the ability to browse for multicast streams on the ASR client as well. The mechanism that allows OS X applications and services to use Bonjour is called the mDNSResponder, and it's Open-Source software available for download at http://developer.apple.com/darwin/projects/bonjour. Apple's not only busy building Bonjour into nearly all of its signature applications on OS X and network services on OS X Server, it's also providing an easy way for developers to include Bonjour functionality in their software and hardware products. Network-enabled products can easily use the Bonjour source code to locate servers and peers. Somewhat logically, the mDNSResponder uses UDP (User Datagram Protocol) port 5353, while traditional DNS uses TCP port 53. A quick line in the Terminal shows it, ears pricked, listening for connections:

Dual2ghz:~ dean$ netstat -a|grep -i mdn
udp4       0      0  *.mdns                 *.*

And a quick grep of /etc/services reveals:

Dual2ghz:~ dean$ cat /etc/services|grep 5353   
mdns            5353/udp    # Multicast DNS
mdns            5353/tcp    # Multicast DNS

One of the first large Open-Source projects to use the Apple mDNSResponder source code is KDE (K Desktop Environment, http://www.kde.org/) 3.4 for Linux. KDE is a very popular Desktop, Office and Application bundle for Linux workstations. Although there are plans for integrating Apple's Bonjour source code more tightly into KDE, for now the best benefit is the discovery of shared X Windows desktops (something that OS X has yet to offer beyond simple screen sharing).

Interestingly enough, the KDE developers actually had a choice of which mDNSResponder to use. Besides Apple's own Bonjour, there's also the parallel Open-Source Howl project, which implements almost exactly the same functionality for OS X, Windows, and many types of Unix, but with a GPL (GNU Public License) and its own code base. Interestingly enough, Howl (http://www.porchdogsoft.com/products/howl) also offers Zeroconf for OS X, which is available in the form of a Fink installer package (http://fink.sourceforge.net).

The Howl browser window is strangely reminiscent of the Network Browser application that shipped with Mac OS 8.5 and 9. However, the Network Browser used SLP (Service Location Protocol), an older standard mostly used by Novell. While it's not completely obvious, such a browser exists in OS X, hidden in the Dock Menu of the Terminal application. To access the browser, hold down the control key and click on the Terminal icon in the Dock or use the right button of a two-button mouse and wait for the "Connect to server" dialog to appear.

It's quite interesting to note that the only appearance of a "browserish" interface is here, for services that many would never truly consider candidates for Bonjour-ization, but for those network admins that might have kept a sheet of manually assigned IP addresses at their desk to ssh into the Macs they manage, it's most welcome! For those on networks with DHCP, a ping scan of the subnet, followed by a reading of the arp cache and a script to pin machine names onto the Ethernet addresses was necessary:

Dual2ghz:~ dean$ ping -c 1 192.168.0.255 ; arp -a
PING 192.168.0.255 (192.168.0.255): 56 data bytes
64 bytes from 192.168.0.87: icmp_seq=0 ttl=64 time=0.149 ms

--- 192.168.0.255 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.149/0.149/0.149 ms
? (192.168.0.1) 	at 0:0:94:83:34:68 on en0 [ethernet]
? (192.168.0.84) 	at 0:3:93:be:42:ba on en0 [ethernet]
? (192.168.0.86) 	at 0:5:2:71:0:d5 on en0 [ethernet]
? (192.168.0.255)	at ff:ff:ff:ff:ff:ff on en0 [ethernet]

Also interesting is the ability to ping the mDNSResponder on each Mac OS X computer by using the multicast broadcast address:

Dual2ghz:~/Desktop dean$ ping 224.0.0.251
PING 224.0.0.251 (224.0.0.251): 56 data bytes
64 bytes from 192.168.0.87:   icmp_seq=0 ttl=64 time=0.735  ms
64 bytes from 192.168.0.128:  icmp_seq=0 ttl=64 time=107.39 ms (DUP!)
64 bytes from 192.168.0.86:   icmp_seq=1 ttl=64 time=0.435  ms (DUP!)
64 bytes from 192.168.0.85:   icmp_seq=1 ttl=64 time=0.505  ms (DUP!)

Note that the response is from "normal" internal IP addresses, not from the multicast or the Link-Local address. In this respect, mDNSResponder acts as a "helper" in locating the IPv4 address in the routing tables of those computers running Bonjour on the local subnet. And, if that were all Bonjour were capable of, that might be good enough.

DNS-SD

While the mDNSResponder could be considered to be the engine that connects the multicast, Link-Local, and IPv4 addresses, making it easy for unconfigured computers to talk to each other as well as the configured computers on their local subnet, it's the DNS-SD (Service Discovery) part of Bonjour (and Rendezvous, its predecessor in name) that really brings Bonjour technology into the spotlight (symbol crash) of an OS X user's daily interaction with their network.

When OS X 10.0 was released, SLP (Service Location Protocol) and AppleTalk were used to advertise file sharing resources on the local network. While SLP was a decent method of browsing for servers, there were some significant problems. Configuring the SLP DA (Directory Agent) for the local network wasn't very straightforward, not to mention the infamous issue with laptops broadcasting multiple instances of their presence on the local network for each IP address they'd every had for any location they'd ever visited. The solution was to delete the /var/slp.regfile which contained the SLP registrations for each of those IP addresses.

OS X 10.3 marked a turning point in the fact that SLP was deprecated in favor of mDNSResponder and Rendezvous. Today, Tiger Servers configured as Open Directory Masters, it's even possible to manipulate the Bonjour browsing views of managed clients. This capability, though not very well documented at this point as far as how the backend tech works, is significant in that it uses a managed DNS-SD capability to control what OS X clients see when browsing. Not only can the Managed Network Views be carved up into virtual and logical segments, it's even possible to control which services the clients can browse for. DNS-SD browses for and announces registered service types.

Some IT managers with a semi-paranoid security focus are of the opinion that the DNS-SD services advertised by Bonjour can constitute a security risk. Although OS X doesn't provide a global browser for all computers on the network, there is a freeware utility called "Bonjour Browser" available at http://www.tildesoft.com/Programs.html that scans the network for registered Bonjour services and displays them in a window. Although it's not of much practical use, it does give away which machines are listening for different types of connections (file services, ssh, etc.) allowing for potential evildoers to get quick down-and-dirty looks at who's offering what network services. Security concerns aside, it's absolutely amazing to see just how deeply integrated Bonjour is in OS X and now diverse the DNS-SD services it supports have become.

It's pretty safe to say that Apple has a registered DNS-SD protocol for each major service that runs on OS X and OS X Server, as well as at least one to advertise the presence of any Apple hardware product with a network port, such as XServe RAIDs and Airport base stations. DNS-SD registrations are in the following format: _servicename._tcp or _servicename._udp. A complete list of DNS-SD registrations, including instructions for developers on how to register a service for their application or hardware devices, is available at the DNS-SD project website: http://www.dns-sd.org/ServiceTypes.html.

It's always interesting when there's a mistake in the man page of a command in OS X. Sometimes, those errors are trivial, other times they are not so trivial. For example, the man page for the asr command lists the suggested default multicast address as 224.0.0.123. However, as we learned from experience, and this article, Bonjour reserves that network range for its own use, and using that same network range for multicast asr is a very bad idea (think network crash). In a similar, but less critical vein, the man page for the dns-sd command line tool suggests that the command first appeared in OS X 10.3, when in reality, it's the dns_sd.h header library that's being referenced, as the command line tool is nowhere to be found on an OS X 10.3 box. However, copying the /usr/bin/dns-sd tool from and OS X 10.4 installation will work just fine. The dns-sd command line tool is intended as a helper utility for developers wanting to test Bonjour services, but it can also register a service as well as browse for one.

Remember the Mac HelpMate asr multicast window in Figure 3? Well, it's actually possible to browse for the multicast server using the dns-sd command line tool, even if the Bonjour name of the server is unknown. Here's how it works. First, it's necessary to know the registration name of the dns-sd service. For multicast ASR, it's simply asr:

Dual2ghz:~/Desktop dean$ ./dns-sd -B _asr._tcp
Browsing for _asr._tcp
Timestamp    A/R  Flags   if   Domain   Service Type   Instance Name
6:49:28.981  Add      2    4   local.     _asr._tcp.   mostsvr

Or, for Apple File Servers, it's afpovertcp:

mostsvr:~ mostadmin$ dns-sd -B _afpovertcp._tcp
Browsing for _afpovertcp._tcp
Timestamp     A/R   Flags  if  Domain        Service Type             Instance Name
 6:54:10.451  Add       3   4  local.        _afpovertcp._tcp.        mostsvr
 6:54:10.454  Add       3   4  local.        _afpovertcp._tcp.        Dual2ghz
 6:54:10.454  Add       3   4  local.        _afpovertcp._tcp.        host2
 6:54:10.454  Add       2   4  local.        _afpovertcp._tcp.        Panther

It's also possible to register an instance of a service on the network with DNS-SD as well:

mostsvr:~ mostadmin$ dns-sd -R "My Test" _http._tcp . 80 path=/path-to-page.htm
Registering Service My Test._http._tcp port 80 path=/path-to-page.htm
Got a reply for My Test._http._tcp.local.: Name now registered and active

I'm Bad, I'm Nationwide

Beyond the gradual adoption of mDNSResponder and DNS-SD technology into other Operating Systems such as Linux, Apple has primed the pump, so to speak, by offering an "official" Bonjour implementation for Windows. I use Apple's Bonjour for Windows whenever setting up Bonjour-capable printers with Windows XP and 2000 computers. It's far simpler than creating a network printer attached to a server or a local TCP/IP port on the Windows box; it's even easy enough for an end user! It's a little known fact, however, that Bonjour for Windows enables the full suite of Bonjour capabilities, not just printers.

Perhaps the biggest (and most unheralded) change in Bonjour, besides the name from Panther to Tiger was the addition of wide-area (and multiple subnet) DNS-SD advertising via Dynamic DNS Updates and unicast DNS queries (multicasts aren't allowed on the Internet with a few exceptions). Last month, (MT 21.10) I wrote about emerging capabilities for more complex Open Directory deployments via LDAP OUs (Organizational Units) and DACs (Directory Access Controls). As Directory Service deployments get more complex and spread out over wide-area networks, it's also necessary that the discovery protocols for browsing those networks keep developing. As Open Directory matures, Bonjour must keep pace.

Even before Bonjour officially supported multiple subnets or wide-area service discovery, there were attempts to bridge Rendezvous over two network segments. The Rendezvous Proxy project (http://ileech.sourceforge.net/) originally sought a way to allow sharing of iTunes music libraries between subnets, but people found out pretty quickly it could be used to advertise printers, servers, or just about any other Rendezvous service. The developer even released a Rendezvous proxy for Windows. Configuring wide-area support for Bonjour at this time is still somewhat experimental, however, Apple has provided a few white papers detailing the theory behind running a Bonjour DNS Domain over the Internet, as well as specific instructions at www.dns-ds.org.

In Next Month's Source Hound

In next month's column, I'll actually set up a wide-area Bonjour DNS Server, see how well it works, then look to see if it's possible to integrate our new DNS capabilities into Apple's Open Directory LDAP server, instead of using BIND. I'll either be singing like the ZZtopsters, or maybe singing soprano, depending on how it goes.

Dean Shavit is an ACSA (Apple Certified System Administrator) who loves to use a Mac, but hates paying for software. So each month he's on the hunt for the best Open-Source and freeware solutions for OS X. Besides surfing for hours, following the scent of great source code, he's a partner at MOST Training & Consulting in Chicago, where he trains system administrators in OS X and OS X Server, facilitates Mac upgrade projects for customers, and writes for his own website, www.themachelpdesk.com. Recently, he became the surprised father of an application: Mac HelpMate, available at www.machelpmate.com. If you have questions or comments you can contact him: dean@macworkshops.com.