TweetFollow Us on Twitter

Debugging with EtherPeek

Volume Number: 13 (1997)
Issue Number: 8
Column Tag: Tools Of The Trade

Debugging with EtherPeek

by Sean Gilligan, Open Systems Development

Use a Macintosh-based network analyzer to circumvent the Heisenberg Uncertainty Principle and debug your network code

Introduction

If you are developing network software and you're not using a network analyzer to help you debug your product, you'll be amazed at how much time you can save. If you are using EtherPeek or a similar product to debug your network code, you probably wonder how you could do your job without it. After reading this article, you will have a better understanding of the capabilities of EtherPeek and how you can use them to get your job done faster. We'll start with an overview of network analyzers and then move on to the basic features of EtherPeek. Next, we'll look at the methodology of debugging with a network analyzer. Finally, we'll look at a simple HTTP client that contains a trivial bug and then show how to use EtherPeek to help fix it. (This example is based on a true story.)

What is a Network Analyzer Anyway?

In 1986, when I began developing network software on Ethernet at 3Com, our Macintosh team was given an internal, DOS-based, utility program called "EtherSpy". After reading the specifications for 3Com's DOS networking client, we used EtherSpy to understand how the DOS client really connected to the server and to design, develop, and debug the Macintosh version. I have used a variety of network analyzer programs over the years and I can't imagine trying to develop any substantial networking software without one.

A software-based network analyzer is a network utility program that runs on a standard desktop or laptop computer using a standard Ethernet adapter. A hardware-based network analyzer is sold as a complete package of a laptop or portable computer, Ethernet adapter, and software. (Network analyzers are also available for Token-Ring and other networks.) A network analyzer is able to receive all packets on its network (LAN segment) by using a feature of Ethernet called promiscuous mode. Normally an Ethernet adapter and its driver software are programmed to only receive packets that are directed to that specific adapter or are broadcast to all adapters on that network. (Packets destined for adapters on another network on an internet or on the Internet must be directed to a router.) When an Ethernet adapter is operating in promiscuous mode, it will receive all packets on its network.

Once triggered, the analyzer software captures all packets on the network using promiscuous mode, discards packets not matching user-specified filters, and then displays the captured packets in a human-readable format. This human readable format is achieved through the use of protocol decoders. An IP protocol decoder will know to display the 32-bit IP address in dotted decimal notation, based on the location of the IP address within the packet (e.g., 17.254.3.21). Good decoders will take this a step further and do a reverse-name lookup and display the machine name, such as "www.apple.com," in addition to the dotted decimal notation.

For most developers, a software-based analyzer is the only affordable option. A small company may be able to afford only a software-based analyzer, and getting access to a hardware-based analyzer in a large company often requires waiting your turn. A software-based analyzer can be installed on a machine in your office or in your engineering test lab -- you can set it up and leave it monitoring your software for extended periods of time. Hardware-based analyzers do offer a few advantages, such as protocol decoders for some of the more obscure protocols, expert analysis capabilities that can find and recommend a solution to a network problem, and the best performance and hardware error reporting capabilities. In general, these features tend to be more useful for network managers than for software developers. You should verify that any protocols you need are supported before purchasing an analyzer.

There are several software-based network analyzers available for both the MacOS and for Windows. In addition, there are hardware-based network analyzers such as the Sniffer from Network General Corporation. EtherPeek, from the AG Group, Inc., with a list price of $995 (developer discounts are available), is a fraction of the cost of the Sniffer, easier to use, and equally capable for the vast majority of applications. (Network General does offer optional modules to decode client/server messages for Oracle, Sybase, and Microsoft SQL Server. This capability could be very valuable to some developers.) EtherPeek is available for both Windows and the MacOS.

Basic EtherPeek Operation

Before using EtherPeek you must install it on a system with a supported Ethernet adapter. Built-In Ethernet on PCI and NuBus-based PowerMacs is supported. Most other common adapters are supported, but you should check the AG Group's web site before assuming a particular adapter is supported. EtherPeek is drag-installable and does not require any special files in the System Folder.

When you first run EtherPeek you'll be asked to select an Ethernet adapter, then the main window will open. Click "Start Capture" to begin capturing packets in the capture buffer. The "Start Capture" button will toggle to "Stop Capture". Click "Stop Capture" to end your trace. Figure 1 shows the main window with the results of a trace displayed in it. Each line of text in the window displays summary information for a single packet.

Figure 1. EtherPeek Main Window.

Double-click a packet summary line in the main window and you can view the details of a packet (as shown below).

Figure 2. EtherPeek Packet Contents Window.

As you can see, capturing and viewing packets is very easy. By using filters you can tell EtherPeek to capture only packets to or from certain addresses or capture packets containing particular protocols. Triggers are used to tell EtherPeek when to begin or end a trace. Filters are typically used to look at a dialog between two specific machines and ignore the remaining traffic on your network. Later, in the HTTP client debugging example, we will setup a filter to catch only the packets from the Macintosh on which we are debugging. Triggers are useful if you don't know exactly when an error is going to occur, but you know that the presence of a certain packet indicates the error has just occurred or is about to occur.

EtherPeek 3.1 allows you to automatically resolve AppleTalk and IP addresses into their corresponding human-readable names (using the NBP and DNS protocols, respectively). You can also manually enter names for the 6-byte Ethernet addresses of any adapter cards that you want to monitor.

Looking at Figure 2, you can see that EtherPeek has knowledge of the TCP/IP protocols and uses this knowledge to display various fields of the packet in a human-readable form. EtherPeek uses a set of rules called packet decoders to display these fields. EtherPeek comes with packet decoders for a wide variety of protocols and even provides a method for you to write your own decoders (using resource files) if you are working with custom protocols.

Another extensible feature of EtherPeek is support for Plug-Ins. Plug-Ins can extract information from packets and display it on the screen, log it to a file, send it to a pager, pass it to an AppleScript, or even "speak" the message. In the debugging example that follows, we will use a Plug-In that displays Web URLs in the packet list. As with decoders, you can write you own Plug-Ins if necessary.

Methodology: Using a Network Analyzer to Narrow the Field

A network analyzer is mandatory if you are developing low-level networking software such as device drivers and protocol stacks, and is essential in higher-level networking software or even network-based applications. In this section, we'll present a general approach, or methodology, for using EtherPeek to debug network software. For purposes of discussion we'll refer to the software you are developing as an application even though it could be system software or a device driver.

A typical type of error in networking software is the "host not responding" category. This is where the machine running your application sends a message to another machine that should respond to it with another message. For discussion purposes, we will define a message as the smallest chunk of data sent by your application to another machine in a single transaction. Typically, the target application on the other machine sends a message back to your machine as a response. The message could be as simple as an IP (ICMP) ping message that fits in a single packet or it could be a more complicated message that requires multiple Ethernet packets.

Imagine you're debugging your "pre-alpha" application and it reports a "host not responding" error. At this point you are probably smiling because a few minutes before your application was crashing. You step through your code in the debugger and everything looks fine: as far as you can tell the message was built correctly; the send() routine returned with no error; if your code is asynchronous, the completion routine was called with no indication of error; and now your application is waiting around patiently for a response, but the receive() routine never returns any data or your receive handler never gets called. You tell the debugger to "go" and after the timeout period you get "host not responding".

This is where you pull EtherPeek out of your bag of tools. There are six basic explanations for what may be going wrong at this point.

  1. Bad Message: You really aren't building the message right. It looks good to you, but you misread the specification -- or perhaps it contained an error. If you are communicating cross-platform, maybe there is an "endian" difference you haven't accounted for.
  2. Message Corrupted: The software below the send() routine that you are calling is corrupting your data. If you are writing an Ethernet driver and the send() routine you are calling is really a register in a prototype controller chip, maybe the chip has a bug.
  3. Message Not Sent: The software or hardware below the send() routine that you are calling is not working correctly or was not configured correctly and is failing to send your message.
  4. Message Sent, Remote Host Didn't Respond: A valid message was sent to the remote host, but either the host system failed to deliver it to the target application on the other machine, the target application failed to respond, or the send() routine and supporting software or hardware on the other machine failed to deliver the response to the network.
  5. Invalid Response Sent: The remote host is sending an invalid response. Perhaps the target application gave its send() routine a bad message or a bad address to send the data to. Maybe the send() routine and its supporting hardware or software corrupted the response data or sent it to the wrong address.
  6. Response Dropped: A correct response was sent back to your machine, but the software or hardware supporting the receive() routine failed to deliver the message. For example, the Ethernet driver or protocol stack could have run out of buffers. If you are writing an Ethernet driver, the adapter hardware could have run out of buffers. You might want to double check your machine's configuration, as well.

By looking at an EtherPeek trace of the session you can view the packets (if any) that were actually sent as part of your message and the response message (if any). With a basic understanding of your message format and the format of the underlying protocols (if any), you can usually narrow it down to one of the following six explanations.

  1. Bad Message: You will be able to see your message in the EtherPeek trace. For any normal protocol stack, such as TCP/IP, EtherPeek will have decoded the protocol information and provided you with a separate display of "user data" that will contain your message. If you see a message and no response, look closely at your message. Was some padding introduced that didn't show up in the debugger? Double check the specification. If you can trace a known working application that talks to the same target, compare the two traces. If you still don't see anything wrong, investigate case #4.
  2. Message Corrupted: Similar to case #1, but the bad data you see in the message doesn't look like anything you could have done (but don't be too quick to point the finger). If parts of the packet other than your "user data" are corrupted, that is a sign that it may not be your fault after all (unless your application is scribbling on memory).
  3. Message Not Sent: Easy to spot -- you have an empty trace. (Make sure you have EtherPeek installed correctly and your filters are correctly configured, though.)
  4. Message Sent, Remote Host Didn't Respond: You won't see a response message in the trace. If you're sure that the message you sent was valid, then the remote host is the culprit.
  5. Invalid Response Sent: You see that the remote host has sent an invalid response. Case closed.
  6. Response Dropped: You see a correct response coming back to your machine. If you're sure your debugger breakpoint was set correctly and no response was delivered, then you can safely blame the receive() routine or the configuration of the software or hardware responsible for receiving the message.

We'll demonstrate setting up a trace later, but for an easily reproducible failure, setting up, running, and analyzing a trace can be done in as little as fifteen minutes. With this much information instantly available from EtherPeek, I don't understand why more network software developers don't use it.

Beating Heisenberg

One of the nicest things about using EtherPeek as a debugging tool is that is has no effect on your program's timing. In physics, the Heisenberg Uncertainty Principle says that you can't observe a system without affecting it. In software development the presence of a debugger, let alone actually using one, can cause the bug's symptoms to change or go away altogether. Network software, particularly at the lower layers, is real-time software: packets are received from other systems asynchronously, timers are used to compute time-outs and to retransmit packets, and interrupt handlers contend for critical sections.

You can't step through this type of code in the debugger when you have a problem. With EtherPeek you can collect a significant amount of information about your bug without setting any breakpoints or stepping through your code. Sometimes the information you glean from a trace will tell you exactly what you must do to fix the bug. If not, you can set your initial breakpoints much more intelligently.

Example: Debugging a Simple HTTP Client

Now, it's time to solve a real problem with EtherPeek. This example is based on a real problem that OSD solved for a client. (The names have been changed and then rewritten to protect the innocent.) To follow this example step-by-step, you'll need two Macintosh systems connected to the same Ethernet network with access to any web server. On one machine you will run EtherPeek and on the test system you will run the buggy HTTP client. We'll add some dramatic context, to make things interesting, and get on with the debugging...

Imagine you arrive at work one morning to find out that one of the engineers working on your project has abruptly left the company to join a monastery overseas. His part of the project was a set of utility programs to download files from an HTTP server. He was working on a test application called "WebGet" that would use the HTTP protocol to download a single file from a web server. You are asked to finish the test application, so you copy the CodeWarrior project file to your hard disk and are able to successfully build WebGet. When you run WebGet, it prompts you for a URL to download. You enter a URL and press return and the application hangs. The hard disk doesn't sound like it downloaded anything, so you assume that the download was unsuccessful and that the program lacks some error checking.

Rather than dive into the source code, you decide to apply the methodology we examined above and take a look at the WebGet's network traffic using EtherPeek. The first thing to do is setup a filter to capture only IP traffic to or from the machine on which WebGet is running. Figure 3 shows the dialog used to create an EtherPeek filter named WebGet from Duo that will capture only IP packets from the Ethernet-attached PowerBook Duo 2300 on which you are running WebGet.

Figure 3. Filter Settings for WebGet.

For my network, I check the Address Filter box and enter the name SeanDuo IP as the only address captured packets should be sent to or from. This assumes that the name SeanDuo IP exists in EtherPeek's name table, otherwise I would have to enter a numeric IP address. I then check the Protocol Filter box and select Internet Protocol as the only protocol to capture -- this way I can still use AppleTalk on the test system.

After confirming these filter settings, I use the Plug-Ins dialog box and turn off IP Details and turn on Web URL for display in the packet list. This will show me the URLs of any web pages requested on the packet summary line in EtherPeek's main window. Finally, I click Start Capture in EtherPeek's main window and run WebGet.

Looking at the trace (Figure 4), our attention is drawn to packet #3, because EtherPeek's Web URL plug-in is showing a URL of "/" at the end of the packet summary line. Let's double click on packet #3 and take a closer look.

Figure 4. A Trace of the Problem Session.

In the TCP Data Area field of the Packet Contents Windows (Figure 5), we see what must be the HTTP GET Request. Unless you know HTTP fairly well, there is nothing obviously wrong with this packet. It says GET / using HTTP/1.0. Notice that the packet is terminated with a carriage-return/line-feed combination (0d 0a), which we'll refer to as CRLF. Looking back at the trace in the main window we don't see any packets large enough to contain any data coming back from the server. Packets #4, #5, and #6 are all 64 bytes long and therefore must be TCP overhead packets.

Figure 5. The Problem Packet.

Let's examine this evidence with our methodological framework from above. The basic situation looks like a "host not responding" error. We can rule out explanations #5 (invalid response) and #6 (response dropped), because the web server didn't send any response at all. Since we are talking to a production web server, we can probably rule out explanation #4 (valid message sent, remote host didn't respond). It can't be #3 (message not sent), because we've seen the request. It's probably too soon to blame it on Open Transport, so let's rule out explanation #2 (message corrupted) for now. That leaves explanation #1 (bad message) as the logical place to start.

Since this problem was just "dropped in our lap" and we haven't had time to read the HTTP specification, let's try running Netscape 3.0.1 and see what it sends for a request. Then we'll compare our request with Netscape's and see how ours is different. Start a new trace and run Netscape on the test system. You should get something that looks similar to Figure 1 (all traces in this example were generated using IP addresses so DNS lookup packets are not shown). Packet #4 is the GET request from NetScape 3.0.1 and is shown in Figure 2. One significant difference between Netscape's GET request and ours is that Netscape seems to be passing a handful of parameters of the form:

parameter-name : parameter-value CRLF

such as:

Connection: Keep-Alive CRLF

and:

User-Agent: Mozilla/3.0 (Macintosh; I; PPC) CRLF

Do we have to add some of these parameters to our request? Let's keep looking -- there is one more difference: each of the parameters is terminated with a CRLF, but there is an extra CRLF at the end of the entire request! If we assume the parameters are optional and remove them all, we still seem to be missing this extra CRLF at the end of the packet.

The Problem Function

Now that we suspect we are building our HTTP GET Request incorrectly, let's look at some code. It seems likely that the problem might be in a routine called BuildGetRequest() (Listing 1). This routine builds the GET Request that we saw in Figure 5. It takes as input a C string containing a Uniform Resource Identifier (URI). A URI is a more general form of a Uniform Resource Locator (URL) that allows for a relative path. The relative path in this case is "/" which tells the web server to return the default document from its root directory. BuildGetRequest() takes the URI that it was passed and inserts it into a string that conforms to the syntax for an HTTP Get Request and puts the resulting string into the specified buffer. A valid HTTP Get Request is defined in Section 5 of RFC 1945 "Hypertext Transfer Protocol -- HTTP/1.0". From looking at the RFC (or from looking at the correct line of code that is commented out), we can see that the request requires a CRLF. Now, we just make a one-line code change and it's time to go home.

Listing 1: BuildGetRequest()

BuildGetRequest
Builds an HTTP Get Request to retrieve the specified URI.

//=======================================================
// BuildGetRequest()
//
// Builds an HTTP Get Request to retrieve the specified URI.
//
//  Inputs:
//    theURI      C String with URI to retrieve
//    requestBuffer  Pointer to buffer to return built 
//                   request message in.
//
//  Outputs:
//    *requestBuffer  Buffer will contain the request 
//                    message
//
//=======================================================
int BuildGetRequest(
  char *theURI,            // URI to retrieve
  char *requestBuffer)    // Where to put request message
{

  // Invalid HTTP 1.0 (missing a CRLF) ...
  char request[] = "GET %s HTTP/1.0\015\012";

  // Correct HTTP 1.0 (has two CRLF's)...
  // char request[] = "GET %s HTTP/1.0\015\012\015\012";
  
  // Build the request message
  sprintf(requestBuffer, request, theURI);
  
  return noErr;
}

Conclusion

We have just used EtherPeek to fix a bug in unfamiliar code that uses a protocol with unfamiliar internals. Although it was a relatively simple example, I'm sure you see the full potential in this method of debugging. By using a network analyzer to quickly eliminate possibilities, you can greatly reduce or even eliminate much of the time you would spend stepping through your code in a debugger. If you are writing or debugging network software, you should definitely add EtherPeek to your toolbox!


Sean Gilligan, Sean_Gilligan@opsystems.com, has been developing cross-platform networking software on the Macintosh and other platforms since 1985. In 1988, he founded Open Systems Development (OSD) to provide software development services to hardware and software manufacturers. OSD develops device drivers, protocol stacks, and networking applications for Macintosh, Windows, and Web-based platforms.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Apple GarageBand 10.1 - Complete recordi...
The new GarageBand is a whole music creation studio right inside your Mac -- complete with keyboard, synths, orchestral and percussion instruments, presets for guitar and voice, an entirely... Read more
Duplicate Annihilator 5.7.7 - Find and d...
Duplicate Annihilator takes on the time-consuming task of comparing the images in your iPhoto library using effective algorithms to make sure that no duplicate escapes. Duplicate Annihilator... Read more
OS X Server 4.1.3 - For OS X 10.10 Yosem...
Designed for OS X and iOS devices, OS X Server makes it easy to share files, schedule meetings, synchronize contacts, develop software, host your own website, publish wikis, configure Mac, iPhone,... Read more
Firefox 39.0 - Fast, safe Web browser. (...
Firefox offers a fast, safe Web browsing experience. Browse quickly, securely, and effortlessly. With its industry-leading features, Firefox is the choice of Web development professionals and casual... Read more
pwSafe 4.1 - Secure password management...
pwSafe provides simple and secure password management across devices and computers. pwSafe uses iCloud to keep your password databases backed-up and synced between Macs and iOS devices. It is... Read more
Kodi 15.0.rc1 - Powerful media center to...
Kodi (was XBMC) is an award-winning free and open-source (GPL) software media player and entertainment hub that can be installed on Linux, OS X, Windows, iOS, and Android, featuring a 10-foot user... Read more
Coda 2.5.11 - One-window Web development...
Coda is a powerful Web editor that puts everything in one place. An editor. Terminal. CSS. Files. With Coda 2, we went beyond expectations. With loads of new, much-requested features, a few surprises... Read more
Bookends 12.5.7 - Reference management a...
Bookends is a full-featured bibliography/reference and information-management system for students and professionals. Access the power of Bookends directly from Mellel, Nisus Writer Pro, or MS Word (... Read more
Maya 2016 - Professional 3D modeling and...
Maya is an award-winning software and powerful, integrated 3D modeling, animation, visual effects, and rendering solution. Because Maya is based on an open architecture, all your work can be scripted... Read more
RapidWeaver 6.2.3 - Create template-base...
RapidWeaver is a next-generation Web design application to help you easily create professional-looking Web sites in minutes. No knowledge of complex code is required, RapidWeaver will take care of... Read more

Rage of Bahamut is Giving Almost All of...
The App Store isn't what it used to be back in 2012, so it's not unexpected to see some games changing their structures with the times. Now we can add Rage of Bahamut to that list with the recent announcement that the game is severely cutting back... | Read more »
Adventures of Pip (Games)
Adventures of Pip 1.0 Device: iOS iPhone Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: ** ONE WEEK ONLY — 66% OFF! *** “Adventures of Pip is a delightful little platformer full of charm, challenge and impeccable... | Read more »
Divide By Sheep - Tips, Tricks, and Stre...
Who would have thought splitting up sheep could be so involved? Anyone who’s played Divide by Sheep, that’s who! While we’re not about to give you complete solutions to everything (because that’s just cheating), we will happily give you some... | Read more »
NaturalMotion and Zynga Have Started Tea...
An official sequel to 2012's CSR Racing is officially on the way, with Zynga and NaturalMotion releasing a short teaser trailer to get everyone excited. Well, as excited as one can get from a trailer with no gameplay footage, anyway. [Read more] | Read more »
Grab a Friend and Pick up Overkill 3, Be...
Overkill 3 is a pretty enjoyable third-person shooter that was sort of begging for some online multiplayer. Fortunately the begging can stop, because its newest update has added an online co-op mode. [Read more] | Read more »
Scanner Pro's Newest Update Adds Au...
Scanner Pro is one of the most popular document scanning apps on iOS, thanks in no small part to its near-constant updates, I'm sure. Now we're up to update number six, and it adds some pretty handy new features. [Read more] | Read more »
Heroki (Games)
Heroki 1.0 Device: iOS Universal Category: Games Price: $7.99, Version: 1.0 (iTunes) Description: CLEAR THE SKIES FOR A NEW HERO!The peaceful sky village of Levantia is in danger! The dastardly Dr. N. Forchin and his accomplice,... | Read more »
Wars of the Roses (Games)
Wars of the Roses 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: | Read more »
TapMon Battle (Games)
TapMon Battle 1.0 Device: iOS Universal Category: Games Price: $.99, Version: 1.0 (iTunes) Description: It's time to battle!Tap! Tap! Tap! Try tap a egg to hatch a Tapmon!Do a battle with another tapmons using your hatched tapmons! *... | Read more »
Alchemic Dungeons (Games)
Alchemic Dungeons 1.0 Device: iOS Universal Category: Games Price: $.99, Version: 1.0 (iTunes) Description: ### Release Event! ### 2.99$->0.99$ for limited time! ### Roguelike Role Playing Game! ### Alchemic Dungeons is roguelike... | Read more »

Price Scanner via MacPrices.net

RamDisk4Mac App Helps Run Your Mac Faster And...
Ever use a RAM disk? If you’ve come to the Mac in the OS X era, likely not. The Classic Mac OS had a RAM disk function built-in, but that was dropped in the conversion to OS X. What is a RAM disk?... Read more
13-inch 1.6GHz MacBook Air on sale for $849,...
Best Buy has the 2015 13″ 1.6GHz/128GB MacBook Air on sale for $849.99 on their online store this weekend. Choose free shipping or free local store pickup (if available). Sale price for online orders... Read more
Apple Refurbished iMacs available for up to $...
The Apple Store has Apple Certified Refurbished iMacs available for up to $380 off the cost of new models. Apple’s one-year warranty is standard, and shipping is free: - 27″ 3.5GHz 5K iMac – $1949 $... Read more
Apple refurbished 2014 13-inch Retina MacBook...
The Apple Store has Apple Certified Refurbished 2014 13″ Retina MacBook Pros available for up to $400 off original MSRP, starting at $979. An Apple one-year warranty is included with each model, and... Read more
Seagate Backup Plus Drives Feature 200GB of C...
Seagate Technology plc has announced that its Backup Plus family of external storage offerings will now include 200GB of OneDrive cloud storage, a major added value, and the addition of Lyve’s photo... Read more
Canon PIXMA MG3620 Wireless Inkjet All-in-One...
Canon U.S.A., Inc. has announced the PIXMA MG3620 Wireless (1) Inkjet All-in-One (AIO) printer for high-quality photo and document printing. Built with convenience in mind for the everyday home user... Read more
July 4th Holiday Weekend 13-inch MacBook Pro...
Save up to $150 on the purchase of a new 2015 13″ Retina MacBook Pro at the following resellers this weekend. Shipping is free with each model: 2.7GHz/128GB MSRP $1299 2.7GHz/... Read more
27-inch 3.5GHz 5K iMac on sale for $2149, sav...
Best Buy has the 27″ 3.5GHz 5K iMac on sale for $2149.99. Choose free shipping or free local store pickup (if available). Sale price for online orders only, in-store prices may vary. Their price is $... Read more
Apple now offering refurbished 2015 11-inch...
The Apple Store is now offering Apple Certified Refurbished 2015 11″ MacBook Airs as well as 13″ MacBook Airs (the latest models), available for up to $180 off the cost of new models. An Apple one-... Read more
15-inch 2.5GHz Retina MacBook Pro on sale for...
Amazon.com has the 15″ 2.5GHz Retina MacBook Pro on sale for $2274 including free shipping. Their price is $225 off MSRP, and it’s the lowest price available for this model. Read more

Jobs Board

*Apple* Music Producer - Apple (United State...
**Job Summary** Apple Music seeks a Producer to help shepherd some of the most important content and editorial initiatives within the music app, with a particular focus Read more
Editor, *Apple* News - Apple (United States...
**Job Summary** Editor, Apple News The Apple News team is looking for passionate, knowledgeable editors to help identify and deliver the best in breaking national, Read more
*Apple* Watch SW Application Project Manager...
**Job Summary** The Apple Watch software team is looking for an Application Engineering Project Manager to work on new projects for Apple . The successful candidate Read more
Engineering Project Manager - *Apple* Searc...
**Job Summary** Apple 's new Spotlight Suggestions service provides fast, relevant search results from the Inte et in Spotlight and Safari on iOS and OS X. We are looking Read more
Business Development Manager - *Apple* Pay...
**Job Summary** Apple Pay is seeking an experienced relationship manager to support the ongoing management of partners for the Apple Pay platform. This position will Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.