Volume Number: 14 (1998)
Issue Number: 7
Column Tag: Power Graphics

Introducing QuickTime 3

by by Tim Monroe, Apple Computer, Inc.
Edited by the MacTech Editorial Staff

What's new in the latest version of Apple's flagship multimedia software

On the 30th day of the 3rd month of this year, Apple released QuickTime version 3, the latest and probably the most eagerly awaited version of its award-winning multimedia software architecture. QuickTime 3 provides dozens of features that are new or improved over earlier versions. QuickTime now supports more than 30 industry-standard video and audio file formats, and it provides a host of new capabilities on both Macintosh and Windows computers. In this article, I'll describe what I take to be the major enhancements contained in QuickTime 3.

Of course, it would take far more space than is available here to cover all of what's new in QuickTime 3. Instead, to borrow some terminology from the game of baseball, I'll focus on the home runs and the extra-base hits; you can investigate the solid line-drive singles yourself at the QuickTime 3 web site and in the developer documentation. See the References section at the end of this article for pointers to those resources.

As you know, QuickTime comprises a number of media technologies, including synchronized sound and video, still graphics, sprite animation, text, music, virtual reality, and 3D imaging. QuickTime 3 has added some exciting new capabilities, such as wired sprites, vector drawing, and video effects, and has enhanced the interoperability of the existing technologies. Before discussing those innovations, let's begin by considering what's new in QuickTime 3 for users and content developers, and what's new in QuickTime 3 for the web. Then we can look at the goodies that QuickTime 3 provides for software developers.

The typical user's first experience with QuickTime 3 will probably involve one of three new pieces of software: a revised and expanded MoviePlayer application, an upgraded image-viewing application called PictureViewer, or the new QuickTime Plug-In for web browsers. As initially installed on either Macintosh or Windows computers, these three items are limited to "basic" operations. For example, the basic MoviePlayer application allows the user to open and view QuickTime movies, but does not allow editing or saving those movies. The basic PictureViewer application exhibits similar restrictions: the user can open images and apply various transformations to them (such as rotate and resize), but cannot save any modified images.

A user can upgrade to QuickTime 3 Pro by registering across the net. Upgrading involves obtaining a registration number and then entering that number into a panel of the QuickTime(tm) Settings control panel. Once the registration number is validated, the MoviePlayer and PictureViewer applications are automatically upgraded to their Pro versions, which allow editing, saving, and many other operations on movies and images. Keep in mind that the "basic/Pro" distinction applies only to MoviePlayer, PictureViewer, and the plug-in: all other software (your QuickTime-savvy application, for instance) is able to take full advantage of the QuickTime APIs, which are fully functional even if the user does not upgrade to the Pro version.

For content developers, QuickTime 3 provides a wealth of new capabilities. MoviePlayer Pro allows the content developer to perform sophisticated movie editing and to save edited movies using a wide variety of compression formats and special effects. Apple has partnered with a number of other companies to provide new compression technologies for both video and sound. On Windows computers, QuickTime 3 provides the Indeo codec from Intel. On both Macintosh and Windows computers, QuickTime 3 includes the Sorenson Video codec from Sorenson Vision, Inc. (which provides state-of-the-art compression technology giving extremely good image quality at a remarkably small data rate). Also included are cross-platform codecs from QDesign (which provides impressive music fidelity in as little as 1 percent of the original file size) and QualComm (whose Pure-Voice technology does very good compression on voice data). Finally, QuickTime 3 includes the Sound Canvas instrument set for playing MIDI sound files and the GS Format extensions developed by Roland Corporation.

QuickTime 3 includes a new, more powerful plug-in for the Netscape Navigator and Microsoft Internet Explorer web browsers, on both Macintosh and Windows. This plug-in supports the display and control of all of QuickTime's digital formats directly in the browser window. The browser plug-in provides several other important new features, including QuickTime Streaming and automatic selection of media based on the user's connection speed.

QuickTime Streaming is a feature of the QuickTime Plug-In that allows QuickTime movies to begin playing before the entire movie has been downloaded to the user's machine. The plug-in waits until it has enough of the movie downloaded that it is confident that the remaining movie data will be downloaded by the time it's needed. The result is that the user spends less time waiting and more time interacting with the digital content.

QuickTime Streaming also works for non-linear QuickTime media, such as QuickTime VR panoramas and objects. The only "gotcha" for VR content is that the movie must be specially flattened for streaming web playback. Content developers will need to install the QTVR Flattener extension (available at the QuickTime web site) and then export a flattened movie using MoviePlayer Pro. When exporting flattened VR content, the developer can include a lower-resolution preview track (or even an arbitrary image file) that downloads very quickly and provides the user with a passable preview of the VR movie. If no such preview track is included in a web-flattened VR movie, the QuickTime Plug-In substitutes a grey-on-black grid (rather like the Star Trek holodeck background) for any data not yet downloaded.

QuickTime 3 provides a second way to address the web bandwidth problem with its support for "multiple data rate" movies. Here the idea is to tailor the data to be downloaded to the user's connection speed (which can be set in the QuickTime(tm) Settings control panel). The content developer accomplishes this by creating a single QuickTime reference movie, which contains references to several other QuickTime movies of varying sizes and resolutions. When a user chooses to view a reference movie embedded in a web page, the QuickTime Plug-In automatically selects the target referenced movie best suited for the user's connection speed. For example, if the user has a LAN or T1 connection to the Internet, the plug-in selects the highest quality movie referenced in the reference movie. If the user has a slow modem connection, say 14.4KBaud, the plug-in instead chooses a smaller, lower-resolution movie, probably also with the audio and video compressed to a greater degree (with an attendant loss of quality). In this way, the QuickTime Plug-In delivers the digital content best suited for a specific user, automatically and completely transparently to the user.

Of course, the QuickTime Streaming feature works with multiple data rate movies too. Once the plug-in has decided which referenced QuickTime movie to download, the data is streamed as described above.

QuickTime 3 provides an extensible architecture for applying video effects to single images or video tracks ("filters"), or to pairs of images or video tracks ("transitions"). QuickTime 3 includes an implementation of the 133 standard transitions defined by the Society of Motion Picture and Television Engineers (SMPTE), as well as some additional effects defined by QuickTime. The SMPTE effects include transitions like a simple explode (where the first image is pushed outward by the second image) and a radial swipe (where the first image is covered over by the second image in a clockwise turn). The QuickTime effects include a very nice "cross fade" or "dissolve" (which uses a smooth alpha blending from the first image to the second) and a nifty "film noise" filter that makes a video track look like old, faded, dusty, and scratched film (see Figure 1).

Figure 1. The film noise filter applied to a movie frame.

QuickTime 3 even defines several video effects that operate on no source images or video tracks at all. For instance, you can use the fire effect to generate a real-looking fire, and you can use the cloud effect to render a wind-pushed, moving cloud. With zero-source effects, the idea is that you will want to composite the effect onto some other image or video track.

What's interesting about all these effects is that they use extremely little data to achieve the desired visual effect. The data describing an effect is stored in a video track, and the actual effect itself is generated in real time as the movie that incorporates the effect is played. For instance, a movie track that specifies the fire effect is only about 200 bytes in size, but when played generates a real time, non-repeating, dynamic effect.

Filters and transitions are implemented in the general QuickTime architecture as image decompressor components. One thing this means is that you can use filters and transitions anywhere you might use a decompressor, not only in connection with QuickTime movies. You can just as easily apply a transition between two arbitrary images (perhaps contained in two offscreen graphics worlds). I've seen this capability used to add QuickTime video effects as transitions between QuickTime VR nodes. The default behavior of QuickTime VR is simply to jump from one node to the target node. It's much nicer to render some video effect, say a nice smooth dissolve, when moving from node to node.

The fact that effects are implemented as decompressors also means that it's relatively easy to construct your own custom effects and to make them available to any applications that support QuickTime. Apple even provides source code for a sample effect that you can use as a starting point for your own effects development.

QuickTime 3 provides a pleasant surprise in its support for vector-based drawing. QuickTime vectors are mathematical descriptions of images, rather like PostScript images. An image that is described using QuickTime vectors can be resized with no quality loss in the image. More importantly, the description of an image using QuickTime vectors is almost always significantly smaller than a bitmap description of the same image (even when the bitmap is compressed). In addition, QuickTime's vector-based drawing supports a relatively fast form of antialiasing that determines the color of antialiased pixels based on the curve calculation itself rather than the standard technique of drawing the image at higher resolution and scaling it down. For these reasons, QuickTime vectors provide a welcome and powerful addition to the QuickTime family of digital media.

A QuickTime vector is roughly the same as a QuickDraw GX path object, which is a connected series of straight lines and curves. Using paths, you can define pretty much any geometric shape, from points and lines, to polygons, curves, and arbitrary concatenations of lines and curves. QuickTime vectors also support the standard QuickDraw GX drawing styles (for instance, pen widths and joins at corners) and inks (supporting many color spaces and image transfer modes). Indeed, the correspondence between GX paths and QuickDraw vectors is so close that you can easily convert existing GX data into QuickTime vectors using an image transcoder included with QuickTime 3.

A QuickTime vector image is described by a set of paths, which are ordered series of atoms in a QuickTime atom container. (This set of paths is called a vector data stream.) The atom container is stored in a movie file as a media sample in a video track. To render a path, the QuickTime vector codec traverses the atoms in the atom container, in the order they appear in the atom container. A particular atom may specify a path (in which case the vector codec draws the path using the current drawing attributes) or a drawing attribute (in which case the vector codec sets that attribute as the default drawing attribute of its type). QuickTime 3 defines a large number of vector atom types, most of which correspond to QuickDraw GX drawing styles. QuickTime 3 adds support for several atom types not supported by GX, including gradient atoms, antialiasing atoms, and minimum bit depth atoms.

QuickTime 2.5 introduced the Sprite Toolbox, a set of data types and functions that you can use to add sprite-based animation to an application. A sprite is simply one image from a finite set of images. You can animate the sprite by changing its spatial properties (its matrix, its layer, its visibility, its graphics mode, and so on) or by replacing it with another image from that set of images. Sprite animation is analogous to traditional cell animation (which is used to create many cartoons).

One of the coolest new features of QuickTime 3 is support for wired sprites, or sprites that react to mouse (and other) events and that have various actions attached (or "wired") to them. In effect, the user can now click on sprites and initiate certain actions with those clicks. By using wired sprite tracks in a movie, you can add a level of interactivity previously unavailable in QuickTime. For instance, you can use wired sprites to control various aspects of movie playback, such as the volume and balance of a sound track, or the graphics mode of a video track. With a modicum of cleverness, you could use wired sprite tracks as the entire graphical user interface for your application.

Figure 2 illustrates a simple wired sprite movie. This movie contains six sets of sprite images: two penguins and four buttons. Clicking on the rightmost penguin causes the other penguin to waddle down to the right and back again, in a loop. The rightmost penguin moves straight down until reaching the bottom of the movie, at which time he (she?) jumps back to the top and starts over. The buttons, which appear only if the mouse is over a button or within the waddling penguin, control the movements of the waddling penguin.

Figure 2. A simple wired sprite movie.

Wired sprites can respond to several kinds of events. Primarily they respond to mouse events (mouse downs, mouse ups, mouse enters, and mouse exits), but they can also receive and react to idle events and to events generated when the key frame containing the sprite data is first loaded. In response to any of these events, a wired sprite can effect changes in one or more targets (including itself). A target is always a sprite, a movie track, or the movie itself. QuickTime 3 defines a large number of actions that a wired sprite can initiate in its target. For instance, a wired sprite can cause a target sprite (possibly itself) to change the index of the displayed image. This is how you might make a sprite button appear to be pressed, by having a clicked-on sprite change its own image index.

QuickTime 3 includes an upgraded version of QuickTime VR, version 2.1. This new version provides several important enhancements to the previous major release, QuickTime VR version 2.0 (introduced for the Macintosh in January 1997). From a user's point of view, QuickTime VR 2.1 provides improved image quality and smoother panning, tilting, and zooming. In addition, QuickTime VR movies can be embedded in web pages and reap all the benefits provided by the QuickTime Plug-In, including QuickTime Streaming and URL linking from VR hot spots to other web pages or to multimedia files. Moreover, QuickTime VR movies can be compressed using most of the new codecs supported by QuickTime 3. Many VR content developers have discovered that the Sorenson codec, in particular, provides a very compact file with minimal loss to the image quality.

The application programming interfaces provided by QuickTime VR 2.1 are virtually identical to those previously supported in version 2.0. (See Monroe and Wolfson, 1997, for an introduction to the QuickTime VR application programming interfaces.) The main difference is that version 2.1 now provides support for both Macintosh and Windows platforms. A number of other minor changes were made, including renaming a handful of functions and data types, adding a few utility functions (for instance, QTVRGetHotSpotType), and expanding the user interface options for object nodes. Also, you can now request that the QuickTime VR movie controller display a custom button in the controller bar, and you can intercept clicks on that button and perform actions in response to those clicks.

Let's consider now what is undoubtedly the game-winning grand slam in QuickTime 3: with very few exceptions, the QuickTime application programming interfaces operate identically on both Macintosh and Windows computers. Before QuickTime 3, Windows support was limited primarily to movie playback using a movie controller. The application programming interfaces provided by the QuickTime for Windows product were different from the QuickTime APIs on the Mac; in particular, the Windows APIs did not support creating or editing movies, and they did not support playing movies without using a movie controller. With the advent of QuickTime 3, these limitations have been entirely removed, and the Macintosh and Windows versions of QuickTime 3 exhibit virtually complete feature parity. In short, if you can do it on the Mac, chances are you can do it, using the same source code, on Windows NT or Windows 95.

Let's look at a few source code examples to appreciate this fully. Listing 1 shows a simple routine that uses QuickTime functions to copy a video track from one movie into another movie.

Listing 1: Copying a video track from one movie to another

OSErr CopyVideoTrack (Movie theSrcMovie, Movie theDstMovie)
{
   Track         mySrcTrack, myDstTrack;
   Media         mySrcMedia, myDstMedia;
   Fixed         myWidth, myHeight;
   OSType        myType;
   
   ClearMoviesStickyError();
   
   // get the first video track in the source movie
   mySrcTrack = GetMovieIndTrackType(theSrcMovie, 1, 
                              VideoMediaType, movieTrackMediaType);
   if (mySrcTrack == NULL)
      return(paramErr);
   
   // get the track's media and dimensions
   mySrcMedia = GetTrackMedia(mySrcTrack);
   GetTrackDimensions(mySrcTrack, &myWidth, &myHeight);
   
   // create a destination track and media
   myDstTrack = NewMovieTrack(theDstMovie, myWidth, myHeight, 
                                 GetTrackVolume(mySrcTrack));
   GetMediaHandlerDescription(mySrcMedia, &myType, 0, 0);
   myDstMedia = NewTrackMedia(myDstTrack, myType, 
                                 GetMediaTimeScale(mySrcMedia), 0, 0);
      
   // copy the entire track
   InsertTrackSegment(mySrcTrack, myDstTrack, 0, 
                                 GetTrackDuration(mySrcTrack), 0);
   CopyTrackSettings(mySrcTrack, myDstTrack);
   SetTrackLayer(myDstTrack, GetTrackLayer(mySrcTrack));

   return(GetMoviesStickyError());
}

As you can see, all the functions called in Listing 1 are standard QuickTime functions. Because the QuickTime APIs are identical on both platforms, the CopyVideoTrack routine will compile and run on both Macintosh and Windows computers running QuickTime 3.

Consider next the function defined in Listing 2, which prompts the user to open an image file and draws the image into an offscreen graphics world. (Note that error-checking has been removed from this routine.)

Listing 2: Drawing an image file into an offscreen graphics world

void GetPictureAsGWorld (short theWidth, short theHeight, short theDepth, GWorldPtr *theGW)
{
   SFTypeList                   myTypeList;
   StandardFileReply            myReply;
   GraphicsImportComponent      myImporter = NULL;
   Rect                         myRect;
   PixMapHandle                 myPixMap = NULL;

   // have the user select an image file; kQTFileTypeQuickTimeImage means any image 
   // file readable by GetGraphicsImporterForFile
   myTypeList[0] = kQTFileTypeQuickTimeImage;
   StandardGetFilePreview(NULL, 1, myTypeList, &myReply);
   if (!myReply.sfGood)
      goto bail;
      
   // get a graphics importer for the image file
   GetGraphicsImporterForFile(&myReply.sfFile, &myImporter);

   // set the size of the GWorld and allocate a new GWorld
   MacSetRect(&myRect, 0, 0, theWidth, theHeight);
   NewGWorld(theGW, theDepth, &myRect, NULL, NULL, 0L);
   
   myPixMap = GetGWorldPixMap(*theGW);
   LockPixels(myPixMap);
   
   // draw the picture into the GWorld
   GraphicsImportSetGWorld(myImporter, *theGW, NULL);
   GraphicsImportSetBoundsRect(myImporter, &myRect);
   GraphicsImportDraw(myImporter);

bail:
   if (myImporter != NULL)
      CloseComponent(myImporter);
}

Notice that the GetPictureAsGWorld function defined in Listing 2 uses not only QuickTime functions, but also Standard File Package functions and QuickDraw's offscreen graphics world functions. Nonetheless, this routine compiles and runs fine, on both Windows and Macintosh computers. On Windows, the StandardGetFilePreview call is "translated" into a call to the Windows API function GetOpenFileName, which displays and manages the Windows equivalent of the standard file-opening dialog. In fact, the Windows implementation of StandardGetFilePreview enhances the stock Windows dialog box by adding the ability to view and create movie previews in the dialog box. Figure 3 shows the Macintosh version of the dialog box displayed by StandardGetFilePreview, and Figure 4 shows the Windows version of the same dialog box.

Figure 3. The dialog box displayed by StandardGetFilePreview on Macintosh computers.

Figure 4. The dialog box displayed by StandardGetFilePreview on Windows computers.

Consider finally the code snippet shown in Listing 3, which is typical Macintosh code for displaying and managing a modal dialog box with a pop-up menu in it. As you can see, this code uses the Dialog Manager, the Control Manager, and the Window Manager. Indirectly, through the GetNewDialog function, it also calls the Resource Manager. And yet, believe it or not, this code compiles and runs under Windows.

Listing 3: Displaying a dialog box with a pop-up menu

// get the dialog that lets the user select an effect component
myDialog = GetNewDialog(kDialogID, NULL, (WindowPtr)-1);
SetDialogDefaultItem(myDialog, kSelectButtonOK);

GetDialogItem(myDialog, kPopUpID, NULL, &myItemH, NULL);
SetControlValue((ControlHandle)myItemH, gCurrEffect);

// now show the dialog
MacShowWindow(myDialog);
MacSetPort((GrafPtr)myDialog);

do {
   ModalDialog(gModalFilterUPP, &myItem);
} while (myItem != kSelectButtonOK);

// get the current value of the pop-up menu
gCurrEffect = GetControlValue((ControlHandle)myItemH);
DisposeDialog(myDialog);

How far can you go using the Macintosh Toolbox and OS APIs on Windows? In other words, just how extensive is the cross-platform support provided by QuickTime 3? Suffice it to say, that I have successfully ported to Windows, with very little modification, code that calls the Macintosh File Manager, Memory Manager, Window Manager, Dialog Manager, Control Manager, Menu Manager, Sound Manager, Resource Manager, Gestalt Manager, and others, in addition to the core QuickTime Media Layer technologies (QuickTime, QuickTime VR, and QuickDraw 3D). Apple even provides Windows versions of the developer tools Rez and DeRez (plus a few Windows-specific resource tools).

To me, this is no less than amazing. Over the past few months, I have been continually impressed at how easy it is to get existing QuickTime code working in a Windows environment. Certainly not all of that code could be ported unchanged. There are important architectural differences between the Macintosh and Windows platforms (the event/message handling mechanisms and the endianness of multibyte data are good examples). Moreover, there are name-space collisions that need to be addressed (for instance, the Windows API includes a ShowWindow function, so the Macintosh function was renamed as MacShowWindow; see Listings 2 and 3). Nonetheless, the QuickTime 3 engineers have done a marvelous job of making it possible to write once and deploy cross-platform.

It's important to keep in mind, however, that QuickTime 3 for Windows is not intended primarily as a general-purpose porting library for getting any Macintosh code at all to run under Windows. Rather, it is intended as a means of making QuickTime and related APIs work the same on all platforms that QuickTime supports. In general, you're better off using standard Windows APIs to display and manage user-interface elements like windows, menus, and modeless dialog boxes, and to handle the basic flow of messages in your Windows applications.

Apple has spent considerable resources to provide high-quality developer documentation describing the new and revised features of QuickTime 3. Indeed, the documentation for the new and revised features of QuickTime alone (that is, not including QuickTime VR, QuickDraw 3D, or the sound technologies) runs to well over 1000 printed pages! This tome has been thoroughly reviewed by the key engineers and by the quality assurance teams to verify that the information is clear, correct, and useful.

Along with the main QuickTime 3 documentation, Apple provides several other documents targeted at specific audiences. It contracted Stephen Chernicoff (author of the well-known Macintosh Revealed series of books) to write an introduction to QuickTime 3 for Windows programmers, which is designed to help Windows developers understand the Macintosh concepts that underlie much of the QuickTime APIs (such as offscreen graphics worlds, handle-based memory management, and so forth). Apple has also collected the Inside Macintosh documentation for the principal cross-platform Toolbox and OS Managers into a useful book Mac OS for Windows. In all, the complete developer documentation posted on the QuickTime 3 web site occupies over 6,500 HTML pages.

The QuickTime 3 software development kit also includes extensive sample code that illustrates how to use many of the new and improved features of QuickTime 3. This source code is virtually all cross-platform, so you can compile and run the sample code for QuickTime video effects, wired sprites, and vector drawing (to name only a few of the samples) on both Macintosh and Windows computers.

Thanks to Sam Bushell, Deeje Cooley, Tom Dowdy, Brian Friedkin, Peter Hoddie, Matt Hoppa, Scott Kuechle, Jeff Mitchell, Ted Oliverio, and Guillermo Ortiz for reviewing this article.

The main source of QuickTime 3 software, documentation, sample code, and other information is the web site http://www.apple.com/quicktime.

Monroe, Tim, and Bryce Wolfson. "Programming With QuickTime VR". MacTech, 13:7 (July 1997), pp. 43-50.

Tim Monroe, monroe@apple.com is a software engineer on Apple's QuickTime team. He is currently developing sample code for the new QuickTime 3 programming interfaces. In his previous life at Apple, he worked on the Inside Macintosh team.