TweetFollow Us on Twitter

The Web from Cocoa

Volume Number: 19 (2003)
Issue Number: 2
Column Tag: Cocoa Development

The Web from Cocoa

Integrating web content with desktop applications, Part 1 in a series

by Fritz Anderson

Introduction

More and more information is available through the World-Wide Web, and more and more computers are constantly connected to the Web. The public Internet aside, private intranets host HTTP servers to make corporate information available in-house. The problem with information presented on a web page is that once on the screen, it has reached a dead end: It has been formatted and presented for human consumption, and unless a human intervenes to copy it, the information will go no further. Often, you want to bring web content into an application so it can do further work.

One solution to this problem is the use of remote-procedure-call (RPC) protocols like SOAP and XML-RPC, whereby information providers supply computed or stored answers to queries submitted in a simple language. This is the ideal solution, because both provider and client have agreed on what service is on offer, on how to ask for it, and on how the result is formatted. It works especially well when both provider and client are "on the same team"--for instance, working on the client and server ends of a single system.

But there are always going to be information providers that don't offer RPC interfaces. A provider, for instance, might have a budget and a mandate to make its information public, but not enough of a budget to do the extra development needed to take its system beyond web service. There will always be a need for programs that retrieve web pages and process the results.

Road Map

This series will take you through two simple projects that explore the two ways a web client requests pages from a server, GET and POST. GET queries are simple, and built into many of the classes in the Cocoa framework; in this article we'll see how it's possible to do a rough-and-ready fetch in just a few lines. We can take advantage of the ease of making GET queries to put effort into making the user experience better.

POST queries are a little trickier, as there is no support for them built into Cocoa. Apple has, however, provided powerful tools to make up for it, in the Core Foundation framework, the procedural layer that underlies both Cocoa and Carbon. Many Cocoa programmers are chary of getting into Core Foundation, and that's a shame, because while there is a slight learning hump to overcome at first, the rewards are great. The remaining two articles will focus on getting over that hump, and packaging a simple POST facility in a Cocoa wrapper.

Thomas: Our Example

My introduction to page-scraping came in my project Jefferson, an application to put a Cocoa face on the Library of Congress's legislative-history database Thomas. Located at http://thomas.loc.gov/, Thomas is a comprehensive, well-indexed compendium of summaries of every bill considered in Congress over the past three decades. A variety of queries, including simple bill numbers and keyword searches, are offered, and produce a great wealth of information. The Library's staff appears to be working continuously to improve the content and appearance of the site, which fills me with as much pride (as a taxpayer) as it does chagrin (as a programmer trying to parse the pages).

We'll be using the Thomas site as the target for our examples. These worked at the time this article was written, but the Library may have improved their site out from under me by the time you read this. The examples, however, apply to any site, not just Thomas.

The Easy Part

Web browsing is a conversation between a browser on the user's machine, and a program that serves the Hypertext Transfer Protocol (HTTP) on the server machine. Most web browsing gets done through GET requests: At its simplest, the browser sends a message with the command "GET" and the URL for the desired web page. The server decodes the URL into the name of a specific file, and sends the contents of the file, wrapped in a suitable HTTP envelope, in the reply. The browser then does what it needs to with the file--usually displaying it on-screen.

The simplest kind of web-server computing comes from this insight: If the server has to do some computing anyway to decode the URL, why not encode requests to do more sophisticated computing tasks in the URL? Such URLs are still presented by GET requests, but include parameters for the desired computation after a question mark.

Apple has made it very easy to perform GET requests in Cocoa. The steps are: Form an NSURL for the request URL; ask for the result; wait for either the result or an error; handle the result or the error.

Basic Etiquette

But first, we pause for courtesy.

The user's computer won't necessarily be connected to the Internet when our code runs. It would be nice to know whether it's connected before we try to place the query, for two reasons. First, it's easier and cleaner to program that way. Second, if the user's computer has dial-up access to the Internet, making the query will cause the modem to dial out. Dialing the user's telephone without permission is rude; it would be a good idea to ask permission. Our first task is to check for connectivity, and ask permission if necessary.

Apple provides the necessary information through the SystemConfiguration framework. SystemConfiguration is not normally linked into a Cocoa project, so we add it to the project by selecting Add Frameworks... in Project Builder's Project menu, and selecting SystemConfiguration.framework in the /System/Library/Frameworks directory.

Listing 1 shows how to use the call SCNetworkCheckReachabilityByName in SystemConfiguration to preflight the net connection. All our main function needs to do is call ConnectivityApproved() to know whether to proceed with the query.

Listing 1a: Connectivity.h

Function declaration
#import <Cocoa/Cocoa.h>
BOOL      ConnectivityApproved(void);

Listing 1b: Connectivity.m

HowAmIReachingHost 
At the lowest level, ask the SystemConfiguration framework whether, and how, we can reach a named 
host. Render the result as an instance of HowConnected.

#import "Connectivity.h"
#import <SystemConfiguration/SCNetwork.h>
typedef enum {
   kCantReach,
   kWillConnect,
   kAmConnected
}   HowConnected;
HowConnected
HowAmIReachingHost(char *   hostName)
{
   SCNetworkConnectionFlags      theFlags;
   if (SCNetworkCheckReachabilityByName(hostName,
         &theFlags)) {
      if (theFlags & kSCNetworkFlagsReachable)
         return kAmConnected;
      if (theFlags & kSCNetworkFlagsConnectionAutomatic)
         return kWillConnect;
      else
         return kCantReach;
   }
   else
      return kCantReach;
   //   When in doubt, say it's unreachable.
}

ConnectivityApproved
For the particular case of thomas.loc.gov, see whether and how we can reach it. If we can't, inform 
the user and return NO. If we can without further action, return YES. If connecting requires dialing 
out, ask the user for permission, and return whether the user granted it.

BOOL
ConnectivityApproved(void)
{
   BOOL      retval = YES;   //   Assume connection is OK
   int      alertResult;
   switch (HowAmIReachingHost("thomas.loc.gov")) {
      case kCantReach:
         NSRunCriticalAlertPanel(
               @"Can't reach the Internet",
               @"Your computer could find no way to get a "
                  @"connection to the Library of Congress "
                  @"server (thomas.loc.gov).", 
               @"OK", nil, nil);
         //   Note that C compilers will join consecutive strings.
         //   GCC does this for @-strings as well as for C-strings.
         retval = NO;         //   Unreachable; return NO
         break;
      case kWillConnect:
         alertResult = NSRunAlertPanel(
               @"Will connect to the Internet",
               @"Your computer must connect to the Internet "
                  @"to answer your query. Is this all right?",
               @"Connect", @"Cancel", nil);
         if (alertResult == NSCancelButton)
            retval = NO;      //   User didn't want to dial out
            break;
      case kAmConnected:
         break;
   }
   
return retval;
}

Performing the GET

Even with preflighting, a GET transaction is embarrassingly easy. We start with the simple Cocoa application template in Project Builder. In Interface Builder, we can throw together a simple display of two text fields, a large text scroller, and a button, and instantiate a simple controller class in the application's .nib file, as shown in Figure 1. We don't have to change the Interface Builder-generated header file (Listing 2a) at all, and all that remains is to fill in the action method that Interface Builder generated for the Fetch button (Listing 2b).

Listing 2a: GETController.h

Class GETController
Notice that this is unchanged from the way Interface Builder generated it.
/* GETController */
#import <Cocoa/Cocoa.h>
@interface GETController : NSObject
{
    IBOutlet NSTextField *billField;
    IBOutlet NSTextField *congressField;
    IBOutlet NSTextView *resultText;
}
- (IBAction)doFetch:(id)sender;
@end


Figure 1. A simple GET window and its controller

Listing 2b: GETController.m

#import "GETController.h"
#import "Connectivity.h"
@implementation GETController
                                                            doFetch:
- (IBAction)doFetch:(id)sender
{
   //   Do nothing if we can't connect:
   if (! ConnectivityApproved())
      return;
   
   //   [1] Harvest the query parameters
   int               congress = [congressField intValue];
   int               bill = [billField intValue];
   
   //   [2] Format the query URL
   NSString *         urlString =
               [NSString stringWithFormat:
               @"http://thomas.loc.gov/cgi-bin/bdquery/z?"
               @"d%d:hr%05d:@@@L&summ2=m&",
               congress, bill];
   //   [3] Make an NSURL out of the string...
   NSURL *         theURL = [NSURL URLWithString: urlString];
   //   [4] ... and fetch the data as a single chunk of bytes
   NSData *         theHTML = [NSData dataWithContentsOfURL:
                                    theURL];
   //   [5] Turn the web page into styled text...
   NSAttributedString *
                     styledText = [[NSAttributedString alloc]
                                             initWithHTML: theHTML
                                             documentAttributes: nil];
   //   [6] ... and display it.
   [[resultText textStorage] 
         setAttributedString: styledText];
}
@end

The actual work of connecting to the Thomas server and retrieving the results occurs at [4] in Listing 2b, where an NSData is initialized with the class message dataWithContentsOfURL:. Many Cocoa classes--NSString, NSData, NSImage, and NSSound among them--can be initialized directly from URLs. I chose an NSData in this case, because I wanted to render the resulting HTML as styled text in an NSAttributedString at line [5], and the relevant initializer accepts an NSData object. If I had wanted to parse the results instead of just displaying them, I might use an NSString instead.

We can do better

That's all you need to do a GET query in a Cocoa application, but Cocoa makes it easy for us to do better. In the example we just saw, our program waits for the server's response to finish before proceeding with any other work: If the server takes more than a few seconds, the Quartz window server will detect that our program isn't handling user events, and will put up the dreaded "spinning beach-ball" cursor.

NSURL and NSURLHandle provide another way to perform GETs, offering more control and flexibility. A second Cocoa application, BetterGET, illustrates their use. BetterGet is just like CocoaGET, except that

  • Our controller class, BetterController, adds an outlet for the Fetch button.

  • BetterController adds a method, pendingTimer: to handle the progress bar animation.

  • BetterController implements the Objective-C protocol NSURLHandleClient.

You may be familiar with protocols under their Java name of "interfaces." A protocol defines a set of Objective-C messages. When a class claims to implement a protocol (by listing the protocol's name in <angle brackets> after its superclass in its @interface declaration), it makes a promise, enforced by the Objective-C compiler, that the class implements methods for every message in that protocol.

Our doFetch: method proceeds more or less as before, but instead of loading an NSData object directly from the target URL, we ask the NSURL for its underlying NSURLHandle. We register the BetterController as the NSURLHandle's client, and then tell it to loadInBackground.

Because NSURLHandleClient is a formal protocol (though some versions of the Cocoa documentation mistakenly claim it is "informal"), you must implement all five methods of the protocol. This is actually a blessing, because they provide a framework that takes you through the whole life cycle of your query.

  • URLHandleResourceDidBeginLoading is sent at the start of the download process. BetterController uses this opportunity to change the Fetch button to a Cancel button.

  • URLHandle:resourceDataDidBecomeAvailable: is ignored. BetterController doesn't use incoming data on-the-fly.

  • URLHandleResourceDidCancelLoading: reports the case in which the user canceled the downoad. BetterController cleans itself up in preparation for the next query.

  • URLHandle:resourceDidFailLoadingWithReason: informs BetterController that the query failed on the network side. Again, BetterController cleans up, but also puts up an alert informing the user of the problem.

  • URLHandleResourceDidFinishLoading:, last of all, is the "normal" end of the process. All of the data has come back from our query, and it's here we see the same few lines that finished the doFetch: method in the simple, one-step CocoaGET.

With practically no effort, Cocoa gave us a way to load and passively display web content in a simple application. With very little more, it gave us an application that downloads web content in the background, permits cancellation, and reports errors. Listing 3 tells the whole story.

Listing 3a: BetterController.h

//   BetterController.h
//   BetterGET
//   Copyright (c) 2002 Frederic F. Anderson
#import <Cocoa/Cocoa.h>
class BetterController
@interface BetterController : NSObject <NSURLHandleClient>
{
  IBOutlet NSTextField *      billField;
  IBOutlet NSTextField *      congressField;
   IBOutlet NSTextView *      resultText;
   IBOutlet NSButton *         fetchButton;
   NSURLHandle *                  myURLHandle;
}
- (IBAction) doFetch: (id) sender;
@end

Listing 3b: BetterController.m

//   BetterController.m
//   BetterGET
//   Copyright (c) 2002 Frederic F. Anderson
#import "BetterController.h"
#import "Connectivity.h"
@implementation BetterController
doFetch:
The default action method for the "Fetch" button in the application window. It harvests the bill and 
Congress numbers from the respective fields, forms the query URL, and initiates the query.

- (IBAction) doFetch: (id) sender
{
   if (! ConnectivityApproved())
      return;
   //   Harvest the numbers...
   int            congress = [congressField intValue];
   int            bill = [billField intValue];
   //   .. convert them to a URL...
   NSString *      urlString =
      [NSString stringWithFormat:
         @"http://thomas.loc.gov/cgi-bin/bdquery/z?"
            @"d%d:hr%05d:@@@L&summ2=m&",
         congress, bill];
   NSURL *         theURL = [NSURL URLWithString: urlString];
   //   ... and start a background fetch.
   myURLHandle = [theURL URLHandleUsingCache: NO];
   [myURLHandle addClient: self];
   [myURLHandle loadInBackground];
}

cancelFetch:
The alternate action method for the "Fetch" button, for use while the query is in progress. It simply 
tells the query to stop processing.

- (IBAction) cancelFetch: (id) sender
{
   [myURLHandle cancelLoadInBackground];
}

resetDownload
A common utility for the various URLHandleClient methods that signal the end of the query. It tears 
down the query and resets the UI for another query.

- (void) resetDownload
{
   //   Unsubscribe from the NSURLHandle
   if (myURLHandle) {
      [myURLHandle removeClient: self];
      myURLHandle = nil;
   }
   //   Change the Cancel button back to a Fetch button
   [fetchButton setTitle: @"Fetch"];
   [fetchButton setAction: @selector(doFetch:)];
}

URLHandleResourceDidBeginLoading
The URLHandleClient method signaling the start of a download. Starts the progress-bar timer and 
changes the Fetch button to be a Cancel button.

- (void) URLHandleResourceDidBeginLoading:
                                                                (NSURLHandle *)sender
{
   //   Change the fetch button to a Cancel button
   [fetchButton setTitle: @"Cancel"];
   [fetchButton setAction: @selector(cancelFetch:)];
}
URLHandle:resourceDataDidBecomeAvailable:
The URLHandleClient method signaling the arrival of data. Ignored.
- (void) URLHandle: (NSURLHandle *) sender
       resourceDataDidBecomeAvailable: (NSData *) newBytes
{
   //   Ignore. We're interested only in the completed data.
}

URLHandleResourceDidCancelLoading:
The URLHandleClient method signaling a user cancellation. Reset the UI for a new download.

- (void) URLHandleResourceDidCancelLoading: 
                                                                (NSURLHandle *)sender 
{
   [self resetDownload];
}

URLHandle:resourceDidFailLoadingWithReason:
The URLHandleClient method signaling a communications failure. Inform the user of the failure, using 
the message provided, and reset the UI for a new download.

- (void) URLHandle: (NSURLHandle *) sender
       resourceDidFailLoadingWithReason: (NSString *) reason
{
   [self resetDownload];
   NSRunAlertPanel(@"Fetch Failed", reason, 
                              @"OK", nil, nil);
}

URLHandleResourceDidFinishLoading:
The URLHandleClient method signaling a successful download. Display the results and reset the UI for 
a new download.

- (void) URLHandleResourceDidFinishLoading: 
                                                                (NSURLHandle *)sender
{
   //   Clean up Fetch button, etc.
   [self resetDownload];
   //   Collect the data and display it.
   NSData *            theHTML = [sender resourceData];
   NSAttributedString *   styledText =
      [[NSAttributedString alloc] initWithHTML: theHTML
                       documentAttributes: nil];
   [[resultText textStorage] 
            setAttributedString: styledText];
}
@end

Now it gets tricky

The GET request consists of an empty HTTP "envelope"--headers identifying the URL sought and other information like the referring web page, the browser being used, the browser's preferred language--and nothing more: The URL itself specifies the whole query.

The other way web browsers interact with server applications is through the POST query. Whereas queries in GET transactions are typically requests for simple lookups, POSTs are generally meant for transactions that are more complex, involving more data than can conveniently be put into a single URL. Alas, Cocoa doesn't include any direct way to make POST requests. Adding POST queries to Cocoa will be the subject of the rest of this series.

See you next month!


Fritz Anderson has been programming and writing about the Macintosh since 1984. He works (and seeks work) as a consultant in Chicago. You can reach him at fritza@manoverboard.org.

 
AAPL
$102.50
Apple Inc.
+0.25
MSFT
$45.43
Microsoft Corpora
+0.55
GOOG
$571.60
Google Inc.
+2.40

MacTech Search:
Community Search:

Software Updates via MacUpdate

Skype 6.19.0.450 - Voice-over-internet p...
Skype allows you to talk to friends, family and co-workers across the Internet without the inconvenience of long distance telephone charges. Using peer-to-peer data transmission technology, Skype... Read more
VueScan 9.4.41 - Scanner software with a...
VueScan is a scanning program that works with most high-quality flatbed and film scanners to produce scans that have excellent color fidelity and color balance. VueScan is easy to use, and has... Read more
Cloud 3.0.0 - File sharing from your men...
Cloud is simple file sharing for the Mac. Drag a file from your Mac to the CloudApp icon in the menubar and we take care of the rest. A link to the file will automatically be copied to your clipboard... Read more
LibreOffice 4.3.1.2 - Free Open Source o...
LibreOffice is an office suite (word processor, spreadsheet, presentations, drawing tool) compatible with other major office suites. The Document Foundation is coordinating development and... Read more
SlingPlayer Plugin 3.3.20.505 - Browser...
SlingPlayer is the screen interface software that works hand-in-hand with the hardware inside the Slingbox to make your TV viewing experience just like that at home. It features an array of... Read more
Get Lyrical 3.8 - Auto-magically adds ly...
Get Lyrical auto-magically add lyrics to songs in iTunes. You can choose either a selection of tracks, or the current track. Or turn on "Active Tagging" to get lyrics for songs as you play them.... Read more
Viber 4.2.2 - Send messages and make cal...
Viber lets you send free messages and make free calls to other Viber users, on any device and network, in any country! Viber syncs your contacts, messages and call history with your mobile device,... Read more
Cocktail 7.6 - General maintenance and o...
Cocktail is a general purpose utility for OS X that lets you clean, repair and optimize your Mac. It is a powerful digital toolset that helps hundreds of thousands of Mac users around the world get... Read more
LaunchBar 6.1 - Powerful file/URL/email...
LaunchBar is an award-winning productivity utility that offers an amazingly intuitive and efficient way to search and access any kind of information stored on your computer or on the Web. It provides... Read more
Maya 2015 - Professional 3D modeling and...
Maya is an award-winning software and powerful, integrated 3D modeling, animation, visual effects, and rendering solution. Because Maya is based on an open architecture, all your work can be scripted... Read more

Latest Forum Discussions

See All

Rhonna Designs Magic (Photography)
Rhonna Designs Magic 1.0 Device: iOS Universal Category: Photography Price: $1.99, Version: 1.0 (iTunes) Description: Want to sprinkle *magic* on your photos? With RD Magic, you can add colors, filters, light leaks, bokeh, edges,... | Read more »
This Week at 148Apps: August 25-29, 2014
Shiny Happy App Reviews   | Read more »
Qube Kingdom – Tips, Tricks, Strategies,...
Qube Kingdom is a tower defense game from DeNA. You rally your troops – magicians, archers, knights, barbarians, and others – and fight against an evil menace looking to dominate your kingdom of tiny squares. Planning a war isn’t easy, so here are a... | Read more »
Qube Kingdom Review
Qube Kingdom Review By Nadia Oxford on August 29th, 2014 Our Rating: :: KIND OF A SQUARE KINGDOMUniversal App - Designed for iPhone and iPad Qube Kingdom has cute visuals, but it’s a pretty basic tower defense game at heart.   | Read more »
Fire in the Hole Review
Fire in the Hole Review By Rob Thomas on August 29th, 2014 Our Rating: :: WALK THE PLANKUniversal App - Designed for iPhone and iPad Seafoam’s Fire in the Hole looks like a bright, 8-bit throwback, but there’s not enough booty to... | Read more »
Alien Creeps TD is Now Available Worldwi...
Alien Creeps TD is Now Available Worldwide Posted by Ellis Spice on August 29th, 2014 [ permalink ] Universal App - Designed for iPhone and iPad | Read more »
Dodo Master Review
Dodo Master Review By Jordan Minor on August 29th, 2014 Our Rating: :: NEST EGGiPad Only App - Designed for the iPad Dodo Master is tough but fair, and that’s what makes it a joy to play.   | Read more »
Motorsport Manager Review
Motorsport Manager Review By Lee Hamlet on August 29th, 2014 Our Rating: :: MARVELOUS MANAGEMENTUniversal App - Designed for iPhone and iPad Despite its depth and sense of tactical freedom, Motorsport Manager is one of the most... | Read more »
Motorsport Manager – Beginner Tips, Tric...
The world of Motorsport management can be an unforgiving and merciless one, so to help with some of the stress that comes with running a successful race team, here are a few hints and tips to leave your opponents in the dust. | Read more »
CalPal Update Brings the App to 2.0, Add...
CalPal Update Brings the App to 2.0, Adds Lots of New Stuff Posted by Ellis Spice on August 29th, 2014 [ permalink ] | Read more »

Price Scanner via MacPrices.net

Apple now offering refurbished 21-inch 1.4GHz...
The Apple Store is now offering Apple Certified Refurbished 21″ 1.4GHz iMacs for $929 including free shipping plus Apple’s standard one-year warranty. Their price is $170 off the cost of new models,... Read more
Save $50 on the 2.5GHz Mac mini, on sale for...
B&H Photo has the 2.5GHz Mac mini on sale for $549.99 including free shipping. That’s $50 off MSRP, and B&H will also include a free copy of Parallels Desktop software. NY sales tax only. Read more
Save up to $300 on an iMac with Apple refurbi...
The Apple Store has Apple Certified Refurbished iMacs available for up to $300 off the cost of new models. Apple’s one-year warranty is standard, and shipping is free. These are the best prices on... Read more
The Rise of Phablets
Carlisle & Gallagher Consulting Group, a businesses and technology consulting firm focused solely on the financial services industry, has released an infographic depicting the convergence of... Read more
Bad Driver Database App Allows Good Drivers t...
Bad Driver Database 1.4 by Facile Group is a new iOS and Android app that lets users instantly input and see how many times a careless, reckless or just plain stupid driver has been added to the... Read more
Eddy – Cloud Music Player for iPhone/iPad Fre...
Ukraine based CapableBits announces the release of Eddy, its tiny, but smart and powerful cloud music player for iPhone and iPad that allows users to stream or download music directly from cloud... Read more
A&D Medical Launches Its WellnessConnecte...
For consumers and the healthcare providers and loved ones who care for them, A&D Medical, a leader in connected health and biometric measurement devices and services, has launched its... Read more
Anand Lal Shimpi Retires From AnandTech
Anand Lal Shimpi, whose AnandTech Website is famous for its meticulously detailed and thoroughgoing reviews and analysis, is packing it in. Lal Shimpi, who founded the tech site at age 14 in 1997,... Read more
2.5GHz Mac mini, Apple refurbished, in stock...
The Apple Store has Apple Certified Refurbished 2.5GHz Mac minis available for $509, $90 off MSRP. Apple’s one-year warranty is included, and shipping is free. Read more
13-inch 2.5GHz MacBook Pro on sale for $999,...
B&H Photo has the 13″ 2.5GHz MacBook Pro on sale for $999.99 including free shipping plus NY sales tax only. Their price is $100 off MSRP. Read more

Jobs Board

*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
Senior Event Manager, *Apple* Retail Market...
…This senior level position is responsible for leading and imagining the Apple Retail Team's global event strategy. Delivering an overarching brand story; in-store, Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.