Volume Number: 13 (1997)
Issue Number: 7
Column Tag: Multimedia

Programming With QuickTime VR

by Tim Monroe and Bryce Wolfson, Apple Computer, Inc.

A look at the new API for managing QuickTime VR movies

QuickTime VR is the part of the QuickTime Media Layer that allows users to interactively explore and examine photorealistic, three-dimensional virtual worlds and objects. For several years, MacOS and Windows users have been able to play back QuickTime VR content using standard navigation controls. With version 2.0, QuickTime VR now supports a C API that allows developers to customize and extend the user's virtual experience. Here you'll find everything you need to know to get started supporting QuickTime VR in your application.

QuickTime VR is certainly one of the hottest Apple technologies today. Dozens and dozens of CD titles have appeared in the past months that depend on QuickTime VR. More significantly, the World Wide Web now serves up literally thousands of QuickTime VR movies. A large part of this popularity stems from the fact that the amount of data required to display a complex, photorealistic panorama using QuickTime VR is much smaller than would be required to model that panorama in detail using a standard 3D graphics application. This, together with the fact that no run-time rendering is occurring, drastically reduces the amount of CPU horsepower and RAM required to immerse the user in a realistic 3D environment. QuickTime VR can transport the user from the tightest passageway deep inside the pyramids to the wide-open expanse of the space shuttle payload bay.

But, to paraphrase Al Jolson, you ain't seen nothin' yet, folks. Apple has recently introduced a C language programming interface to QuickTime VR -- called the QuickTime VR Manager -- that provides a large set of tools for extending and customizing the user's virtual experience. The QuickTime VR Manager provides the necessary hooks for you to add your own custom processing to VR movie playback and to integrate QuickTime VR content playback with other technologies, particularly with other multimedia technologies. For instance, you can very easily integrate Apple's SoundSprocket with QuickTime VR to attach sounds to specific locations in a panorama. Or, you can play QuickTime movies on the screen of a television in a panorama. Or, you can embed QuickDraw 3D objects (even moving objects!) in a panorama or object node. Take your favorite technology and, chances are, there's a cool way to integrate it with a VR panorama or object.

In this article, we'll explain some basic ways of using the QuickTime VR Manager in your application. We start by reviewing some simple but important aspects of QuickTime movie support. Then we move into programmatic control of QuickTime VR movies by showing how to use standard Movie Toolbox and movie controller functions to operate on QuickTime VR movies. Finally, we'll describe the QuickTime VR Manager itself and present some code that demonstrates a few of its capabilities.

Before reading this article, you should be familiar with the basic capabilities and operations of QuickTime VR. At the very least, you should have opened some panoramas or object movies using a QuickTime player (for instance, MoviePlayer or QTVRPlayer) and have a good understanding of the VR user interface (panning and tilting, zooming in and out, triggering hot spots, and so forth). You can also get the necessary background information by looking at the first dozen pages of the book Virtual Reality Programming With QuickTime VR 2.0, the developer documentation for the QuickTime VR Manager.

The first thing to keep in mind is that QuickTime VR movies are just QuickTime movies, at least in the way the movie data is stored in files. The image data for panoramas and objects is stored in standard QuickTime video tracks in movie files. Other data (such as names of the nodes in a scene) is stored in data atoms in the movie resources. What's different about QuickTime VR is the way in which the image data is handled. In a QuickTime movie, frames of a movie are read from a video track and displayed in sequence to the user. In a QuickTime VR movie, the image handling is more complicated, because the image to be displayed depends more heavily on user interaction. In an object movie, for instance, the user can pan left or right, tilt up or down, or zoom in and out (among other things). These actions typically require a new frame from the movie's video track to be displayed. The new frame, however, may or may not follow the current frame in the movie's video track.

Displaying the appropriate panorama or object images in response to user actions is the responsibility of the QuickTime VR movie controller, which is a standard QuickTime movie controller. Every QuickTime VR movie contains a special piece of user data that identifies the movie controller for the movie. This user data is examined by the Movie Toolbox when an application calls the NewMovieController function to create a movie controller for the movie. If the QuickTime VR extension is installed and the QuickTime VR movie controller component is therefore available, the Movie Toolbox locates that controller and assigns it to that movie.

Now here's the payoff: if you've done things right, your QuickTime application already supports QuickTime VR! Go ahead and try it: launch your QuickTime-savvy application, select Open from the File menu, and then choose a QuickTime VR movie. (Easier still, just drag the VR movie icon onto your application's icon.) If everything goes according to plan, the VR movie will open correctly and you'll be able to navigate within the panorama or manipulate the object in all the right ways. This is because, as we've just said, the QuickTime VR file contains information that specifies the QuickTime VR movie controller instead of a QuickTime movie controller. It couldn't be any easier.

This payoff, however, has a catch: to get VR movie playback, you have to support QuickTime in the right way. Part of what that means is that you have to call NewMovieController to associate a movie controller with a movie. Years ago, Peter Hoddie voiced this warning in a develop column: "When you need a user interface for playing a movie, you should use NewMovieController to create a movie controller appropriate for use with that movie. A common mistake is to instead use the Component Manager routine FindNextComponent or OpenDefaultComponent to locate a movie controller. This finds the first movie controller in the system's list of registered components. QuickTime has always contained only one movie controller, so this worked fine. However, future versions of QuickTime will almost certainly include other movie controllers, so the first one isn't necessarily the most appropriate one." (Hoddie, p. 22)

These "future versions" of QuickTime are here now, and one of them is QuickTime VR. So, to support VR movies, you must call NewMovieController instead of the Component Manager. More generally, you should make sure that you've followed all the advice in Hoddie's important article. It won't take you long to do so, and it will make it easier to support QuickTime in all its manifestations.

Once you've implemented basic -- and correct -- QuickTime movie playback support in your application, there are still a few other things you should do to handle QuickTime VR movies. In particular, you'll want to enable and disable menu items correctly and to display the correct cursor when it's outside the movie window. Happily, you can do each of these things with just a few lines of code. We're supposing throughout that you want your application to handle both QuickTime and QuickTime VR movies; if instead you wanted to build just a QuickTime VR player, some of these steps (for instance, worrying about the Select All menu command) wouldn't be necessary.

The first thing you'll want to do is support the standard movie editing commands for QuickTime movies but also disable any Edit menu items that don't make sense for QuickTime VR movies. When you first open a QuickTime or QuickTime VR movie, you can call MCEnableEditing to turn on editing (which by default is off):

MCEnableEditing(myMC, true);      // enable movie controller editing

Here myMC is the movie controller. We'll keep track of this and other information by setting fields in a window object record, a data structure associated with each window that contains a QuickTime or QuickTime VR movie; see later on in this article for the exact structure of a window object record. As recommended in Hoddie's article, we use the MCSetUpEditMenu function in our menu-adjusting routine to enable and disable items in the Edit menu:

myMC = (**myWindowObject).fController;
if (myMC != NULL)
   MCSetUpEditMenu(myMC, 0L, GetMHandle(mEdit));

The QuickTime VR movie controller disables most of the Edit menu items, since they don't apply to VR movies. It does not however disable the Select All item, so you must do that yourself, like this:

myErr = MCGetControllerInfo(myMC, &myControllerFlags);
if (myErr != noErr)
   myControllerFlags = 0L;
isEditingEnabled = 
   (mcInfoEditingEnabled & myControllerFlags) != 0;
if (isEditingEnabled)
   EnableItem(GetMHandle(mEdit), iSelectAll);
else
   DisableItem(GetMHandle(mEdit), iSelectAll);

Next, the QuickTime VR movie controller manages the cursor whenever it's within a VR movie, changing its shape at appropriate times (such as when the cursor is over a hot spot). The QuickTime VR movie controller doesn't restore the cursor to the arrow shape when the cursor moves outside of the movie. To do that, you can put this code in your idle event processing code:

if (theWindow == FrontWindow()) {
   Rect   myRect;
   GetMouse(&myPoint);
   MCGetControllerBoundsRect(myMC, &myRect);
   if (!PtInRect(myPoint, myRect))
      InitCursor();
}

This cursor-adjusting code is currently needed only for QuickTime VR movies, but should probably be called by any QuickTime-savvy application.

You'll notice that we've used several movie controller component functions with a QuickTime VR movie. This underscores once again the fact that QuickTime VR movies are just special kinds of QuickTime movies. For instance, you can hide the controller bar by executing this code:

MCSetVisible(myMC, false);

In general, you can use any movie controller functions with QuickTime VR movies, except those that specifically relate to time or movie editing. It would make no sense, for example, to issue the movie controller action mcActionGoToTime on a QuickTime VR movie. When QuickTime VR movies do involve temporal aspects (for instance, an animated object movie has a play rate), the QuickTime VR Manager provides functions that you can use to manipulate those aspects.

The QuickTime VR movie controller also supports movie controller action filters. That is, you can intercept a VR movie's controller actions and react accordingly. So virtually the entire QuickTime programming interfaces are applicable also to QuickTime VR movies.

So far, we've shown how to support QuickTime VR movie playback, and even provide a significant level of customization of the movie playback and user interface, just using standard QuickTime movie controller functions. The sample application VRShell provides a concrete example of this capability. It's built on MovieShell, a QuickTime-savvy framework written by Apple DTS. All of our remaining sample applications are built in turn on VRShell.

The QuickTime VR Manager provides a large number of capabilities that you can use to customize and extend the user's virtual experience of panoramas and objects. Here we'll summarize the basic capabilities of the QuickTime VR Manager. Then, in the following sections, we'll illustrate how to use some of them. In this article, we'll keep things fairly simple; in the future, we hope to illustrate some of the more advanced capabilities of the QuickTime VR Manager.

The QuickTime VR Manager provides these main capabilities:

• Positioning. A VR movie file contains a scene, which is a collection of one or more nodes. Each node is either a panoramic node (a "panorama") or an object node (an "object") and is uniquely identified by its node ID. Within a panoramic node, the user's view is determined by three factors: the pan angle, the tilt angle, and the vertical field of view. For objects, the view is also determined by the view center. The QuickTime VR Manager provides functions to get and set any of these items. For instance, you can programmatically spin an object around by repeatedly incrementing the current pan angle.
• Hot spot handling. You can use the QuickTime VR Manager to manage any hot spots in a panorama or object. For instance, you can trigger a hot spot programmatically (that is, simulate a click on the hot spot), enable and disable hot spots, determine whether the cursor is over a hot spot, find all visible hot spots, and so forth. You can also install a callback routine that is called whenever the cursor is over an enabled hot spot.
• Custom node entering and leaving behaviors. The QuickTime VR Manager allows you to perform actions whenever the user enters a new node or leaves the current node. For instance, you might play a custom sound when the user enters a particular node. Current versions of QuickTime VR support scenes that contain both panoramic and object nodes. If you want to treat object nodes differently from panoramic nodes, you can use node-entering and node-leaving procedures to do any necessary processing.
• Getting information. You can use the QuickTime VR Manager to get information about a scene or about a specific node. For instance, you might want to determine the ID and type of the current node. Much of the information about scenes and nodes is stored in atoms in the movie file. To get information about a scene or node that isn't provided directly by the QuickTime VR Manager, you'll have to use the QuickTime atom container functions (introduced in QuickTime version 2.1) to extract information from those atoms.
• Intercepting QuickTime VR Manager functions. You can intercept calls to some QuickTime VR Manager functions to augment or modify their behavior. For example, to assign behaviors to custom hot spots, you can install an intercept routine to be called whenever a hot spot is triggered. Your intercept routine would check the type of the triggered hot spot and then perform the correct actions for that type. Another common use of intercept routines is to intercept positioning functions (changing the pan, tilt, and field of view) and adjust environmental factors accordingly.
• Accessing offscreen panorama buffers. QuickTime VR maintains two offscreen buffers for each panorama, a back buffer and a prescreen buffer. The back buffer contains some or all of the data in a panorama image track, which is a standard QuickTime video track whose frames contain slices of a cylindrically warped and rotated version of the original panorama. The prescreen buffer contains the unwarped and unrotated image that is about to be copied to the screen.

You can use QuickTime VR Manager functions to access the back buffer and the prescreen buffer. Which buffer you draw into is determined by the effect you want to achieve. To overlay a graphic (such as a head's-up display) that is unaffected by the user's panning, tilting, or zooming, you would draw into the prescreen buffer. To overlay a graphic that is affected by changes in the view, you would draw into the back buffer. Back buffer drawing is a bit tricky, however, because the images you draw there must be rotated and warped if they are to appear correctly on the screen. (Note: in future versions of QuickTime VR, the back buffer might not be rotated. The QuickTime VR Manager provides a way to check whether it is rotated or not.)

This list is not exhaustive. The QuickTime VR Manager provides many other capabilities as well. For a complete description, see the book Virtual Reality Programming With QuickTime VR 2.0.

Before you can do these neat things with the QuickTime VR Manager, you must do a little setting up (over and beyond what's required for using QuickTime). First, of course, you must ensure that the QuickTime VR Manager is available in the current operating environment. As you'd expect, there are several Gestalt selectors that you can use to see whether the QuickTime VR Manager is available and what features it has. Here we'll spare you the details on calling Gestalt; consult the sample code (or the reference book) if you absolutely can't live without seeing them.

The QuickTime VR Manager keeps track of QuickTime VR movies using an identifier called a QTVR instance (of data type QTVRInstance). Virtually all QuickTime VR Manager calls operate on QTVR instances. You can think of an instance as representing a scene -- that is, a collection of nodes, or sometimes just the current node. You get a QTVR instance by calling the QTVRGetQTVRInstance function, as shown in Listing 1. QTVRGetQTVRInstance takes a reference to a QTVR track, which you can get by calling QTVRGetQTVRTrack. (In general, you'll never do anything else to the QTVR track, so you can safely forget about it.)

Listing 1: InitApplicationWindowObject

void InitApplicationWindowObject 
                               (WindowObject theWindowObject)
{
   Track             myQTVRTrack = NULL;
   Movie             myMovie = NULL;
   MovieController   myMC = NULL;
   QTVRInstance      myInstance = NULL;
      
   if (theWindowObject == NULL)
      return;
   // make sure we can safely call the QTVR API
   if (!gQTVRMgrIsPresent)
      return;
   // find the QTVR track
   myMovie = (**theWindowObject).fMovie;
   myMC = (**theWindowObject).fController;
   myQTVRTrack = QTVRGetQTVRTrack(myMovie, 1);
   // get a QTVR instance and remember it
   QTVRGetQTVRInstance(&myInstance, myQTVRTrack, myMC);
   (**theWindowObject).fInstance = myInstance;
   // do any QTVR window configuration
   if (myInstance != NULL) {
      // set units to radians
      QTVRSetAngularUnits(myInstance, kQTVRRadians);
   }
}

The InitApplicationWindowObject function defined in Listing 1 takes as a parameter a window object, which is a handle to a data structure associated with each window containing a QuickTime (or QuickTime VR) movie:

typedef struct {
   WindowRef          fWindow;       // the window
   OSType             fObjectType;   // app-specific window tag
   Movie              fMovie;        // the main movie (QT or QTVR)
   MovieController    fController;   // the movie controller
   FSSpec             fFileFSSpec;
   short              fFileResID;
   short              fFileRefNum;
   QTVRInstance       fInstance;     // the QTVRInstance
   Handle             fAppData;      // a handle to app-specific data
} WindowObjectRecord, *WindowObjectPtr, **WindowObject;

The fields of this structure contain, among other things, references to the movie and movie controller, and the QTVR instance. The field fAppData is a handle to any other data that might have to be associated with the window. For the time being, we'll ignore that field.

Notice that Listing 1 calls the QTVRSetAngularUnits function. The QuickTime VR Manager can work with either degrees or radians when specifying angular measurements. The default angular unit type is degrees, but you can change the current unit type by calling QTVRSetAngularUnits. Internally, the QuickTime VR Manager always uses radians, and in some situations gives you measurements in radians no matter what the current angular unit. In general, therefore, we find it easier to work in radians most of the time, so we reset the angular unit type to radians. (Your preference may vary!) Of course, we can define some macros to convert degrees to radians and vice versa, should the need arise:

#define kVRPi                ((float)3.1415926535898)
#define kVR2Pi               ((float)(2.0 * 3.1415926535898))
#define QTVRUtils_DegreesToRadians(x)      \
                             ((float)((x) * kVRPi / 180.0))
#define QTVRUtils_RadiansToDegrees(x)      \
                             ((float)((x) * 180.0 / kVRPi))

Finally, we're ready to use the QuickTime VR Manager to do some real work. The most basic way to use the API is to control the view parameters of a node -- the pan, tilt, and field of view angles. Listing 2 defines a function that gradually increments the pan angle through 360 degrees. With panoramas, this has the effect of making the user seem to spin a full circle (as if the user is spinning on a rotating stool). With objects, this has the effect of making the object spin around a full circle (as if the object is spinning on a lazy Susan).

Listing 2: SpinAroundOnce

void SpinAroundOnce (QTVRInstance theInstance)
{
   float      myOrigPanAngle, myCurrPanAngle;
   
   myOrigPanAngle = QTVRGetPanAngle(theInstance);
   for (myCurrPanAngle = myOrigPanAngle; 
       myCurrPanAngle <= myOrigPanAngle + kVR2Pi; 
       myCurrPanAngle += QTVRUtils_DegreesToRadians(10.0)) {
     QTVRSetPanAngle(theInstance, myCurrPanAngle);
     QTVRUpdate(theInstance, kQTVRCurrentMode);
   }
}

The idea here is simple: get the starting pan angle (by calling QTVRGetPanAngle) and then repeatedly increment the pan angle by a certain amount (here, 10 degrees) until a full circle has been traversed. Note that we must call the QTVRUpdate function after we set a new pan angle to make sure the updated view is displayed on the screen.

Suppose you wanted to display a logo or other graphical element in the corner of a QuickTime VR panoramic movie (in the way that some TV channels often do to discourage pirating). Because the overlaid logo isn't affected by the view settings, we can just draw it into the panorama's prescreen buffer. We must keep track of the picture to be drawn, so we'll create an application data record and store a handle to that record in the fAppData field of the window object record. For present purposes, our data record can look like this:

typedef struct {
   PicHandle   fPicture;       // the picture to display
   Boolean   fDisplayPicture;  // is the picture to be displayed?
} ApplicationDataRecord, *ApplicationDataPtr, 
   **ApplicationDataHdl;

We'll get the overlay picture from a PICT resource, in a window data initialization routine (shown in Listing 3) called by the application's InitApplicationWindowObject function.

Listing 3: VRLogo_InitWindowData

ApplicationDataHdl VRLogo_InitWindowData 
                           (WindowObject theWindowObject)
{
   ApplicationDataHdl    myAppData;
   
   myAppData = (ApplicationDataHdl)
                           NewHandleClear(sizeof(myAppData));
   if (myAppData != NULL) {
      // get the picture to overlay
      (**myAppData).fPicture = GetPicture(kPictureResID);
      
      // set initial display state
      (**myAppData).fDisplayPicture = true;
   }
   return(myAppData);
}

Also, in the application's InitApplicationWindowObject function, we must install our prescreen buffer imaging completion procedure, which is called by the QuickTime VR Manager each time the prescreen buffer is about to be copied to the screen. We can install our procedure like this:

if (QTVRGetNodeType(myInstance, kQTVRCurrentNode) 
      == kQTVRPanoramaType) 
{
   ImagingCompleteUPP   myImagingProc;

   myImagingProc = 
      NewImagingCompleteProc(VRLogo_PrescreenRoutine);
   QTVRSetPrescreenImagingCompleteProc(myInstance, 
                myImagingProc, (SInt32)theWindowObject, 0);
}

The third parameter to QTVRSetPrescreenImagingCompleteProc is an application-specific reference constant value; here we pass the window object reference, so the prescreen buffer can access the data associated with the window.

Our prescreen buffer imaging completion procedure is called after QuickTime VR has finished drawing into the prescreen buffer. When it's called, the current graphics port is set to the prescreen buffer. All we need to do is draw the picture at the appropriate spot, as shown in Listing 4.

Listing 4: VRLogo_PrescreenRoutine

pascal OSErr VRLogo_PrescreenRoutine 
  (QTVRInstance theInstance, WindowObject theWindowObject)
{
#pragma unused(theInstance)

  ApplicationDataHdl      myAppData;
  Rect                    myMovieRect;
  Rect                    myPictRect;
   
  // get the application-specific data associated with the window
  myAppData = (ApplicationDataHdl)
                GetAppDataFromWindowObject(theWindowObject);
  if (myAppData == NULL)
     return(paramErr);

  // if there is no picture to display or displaying is toggled 
  // off, just return
  if ((**myAppData).fPicture == NULL)
     return(noErr);
            
  if (!(**myAppData).fDisplayPicture)
     return(noErr);
     
  // get the current size of the movie
  GetMovieBox((**theWindowObject).fMovie, &myMovieRect);
  
  // set the size and position of the overlay rectangle
  SetRect(&myPictRect, 0, 0, 32, 32);
  OffsetRect(&myPictRect, 
             myMovieRect.right - (myPictRect.right + 5), 
             myMovieRect.bottom - (myPictRect.bottom + 5));

  // draw the picture
  DrawPicture((**myAppData).fPicture, &myPictRect);
  
  return(noErr);
}

There's nothing very complicated in this prescreen buffer imaging completion procedure. Essentially, it just figures out where in the buffer to draw the picture and then draws it.

Note that the current release of the QuickTime VR Manager maintains prescreen buffers only for panoramic nodes. It's possible, however, with a little effort, to create your own prescreen buffer for object nodes and then perform the same kind of overlays that are possible with panoramic nodes.

Much of the real power provided by the QuickTime VR Manager derives from the ease with which it allows you to integrate VR movies with other media, such as video, sound, and 3D objects. In a future article, we'll show how to embed QuickDraw 3D objects in a panorama. In the meantime, we'll give you a good taste of what's possible by showing how to integrate QuickTime VR and SoundSprocket, the part of Apple Game Sprockets that supports localized sounds (that is, sounds emanating from a specific location in a panorama). We'll suppose that you're already familiar with SoundSprocket, but the ideas are so simple that you can probably follow along even if you aren't.

SoundSprocket provides a virtual audio environment consisting of a single listener and one or more sound sources. The listener and the sound sources all have independent positions in 3D space. In addition, the listener and sound sources all have independent orientations in 3D space. The basic idea behind our sample application is that the listener is situated at the nodal point of the panorama (the point around which the panorama turns) and is looking at the center of the movie window. As the user interactively changes the pan and tilt angles of the panorama, the fixed locations of the sound sources change relative to the listener. (SoundSprocket doesn't actually require that the sound sources have fixed locations, but we'll keep the locations of our sources fixed, for simplicity.)

The data we need to maintain for each VR movie has this structure:

typedef struct {
   SSpListenerReference    fListener;
   SSpSourceReference      fSources      [kMaxNumSourcesPerNode];
   SndChannelPtr           fChannels   [kMaxNumSourcesPerNode];
   SndListHandle           fResources   [kMaxNumSourcesPerNode];
   float                   fPrevPanAngle;
   float                   fPrevTiltAngle;
} ApplicationDataRecord,   *ApplicationDataPtr, 
   **ApplicationDataHdl;

We're keeping track of the listener and the sound sources used by SoundSprocket, as well as a sound channel and a sound resource for each sound in the panorama. Finally, we're keeping track of the previous pan and tilt angles, which we'll compare with the current pan and tilt angles to determine whether we need to update the listener's orientation.

We'll skip over the details of setting up the virtual audio environment. What's of interest here is the way in which we translate changes in the VR movie's pan and tilt angles into changes in the listener's orientation. We do this using a prescreen buffer imaging completion procedure, not because we want to actually draw anything into the prescreen buffer, but simply because we want to be called whenever the pan or tilt angles of a panorama have changed and (therefore) a new view is about to be displayed. Our prescreen routine is shown in Listing 5.

Listing 5: VR3DSound_PrescreenRoutine

pascal OSErr VR3DSound_PrescreenRoutine 
   (QTVRInstance theInstance, WindowObject theWindowObject)
{
   float                 myPan;
   float                 myTilt;
   TQ3Vector3D         myOrientation;
   ApplicationDataHdl    myAppData;

   myAppData = (ApplicationDataHdl)
                GetAppDataFromWindowObject(theWindowObject);
   if (myAppData == NULL)
      return(paramErr);
      
   // get the current pan and tilt angles (in radians)
   myPan = QTVRGetPanAngle(theInstance);
   myTilt = QTVRGetTiltAngle(theInstance);
   
   // determine whether the pan or tilt angle has changed
   if ((myPan == (**myAppData).fPrevPanAngle) && 
         (myTilt == (**myAppData).fPrevTiltAngle))
      return(noErr);

   (**myAppData).fPrevPanAngle = myPan;
   (**myAppData).fPrevTiltAngle = myTilt;
      
   // figure out the new orientation of the listener
   myOrientation.x = -sin(myPan) * cos(myTilt);
   myOrientation.y = sin(myTilt);
   myOrientation.z = -cos(myPan) * cos(myTilt);
      
   // set the new orientation of the listener
   SSpListener_SetOrientation
               ((**myAppData).fListener, &myOrientation);
   
   // update the virtual audio environment
   VR3DSound_Update3DSoundEnv(theWindowObject);
   
   return(noErr);
}

Once again, there isn't anything very complicated here. This prescreen routine gets the current pan and tilt angles and then (using a little elementary trigonometry) converts those angles into a point in three-dimensional space. For simplicity, we've assumed that the sound sources are all located one unit away from the listener, but it would be quite trivial to remove that restriction. Then the prescreen routine sets the new orientation for the listener and updates the virtual audio environment.

The source code for the VR3DSound sample application contains (in addition to all the necessary SoundSprocket processing) code for opening sound resources and stopping a node's sounds when the user moves to a new node. That code illustrates how to use node-entering and node-leaving procedures.

Suppose you want to play a sound every time the user clicks (that is, triggers) a hot spot. The easiest way to do this is to install an intercept procedure that is called each time a hot spot is triggered. The intercept procedure simply plays the sound and then returns, whereupon QuickTime VR processes the hot spot click as usual. Listing 6 shows a simple hot spot triggering intercept procedure.

Listing 6: VRSample_InterceptRoutine

pascal void VRSample_InterceptRoutine (
                      QTVRInstance theInstance, 
                      QTVRInterceptPtr theMsg, 
                      WindowObject theWindowObject, 
                      Boolean *cancel)
{
#pragma unused(theInstance, theWindowObject)

   Boolean          myCancelInterceptedProc = false;
   switch (theMsg->selector) {
      case kQTVRTriggerHotSpotSelector:
         MyPlaySound();
         break;
         
      default:
         break;
   }
   
   *cancel = myCancelInterceptedProc;
}

An intercept routine is executed whenever the intercepted routine is called, either programmatically or by a user action. (We'll show you shortly how to specify which routine or routines you want to intercept.) On entry, the QuickTime VR Manager provides three pieces of information: the relevant QTVR instance, a pointer to an intercept record, and an application-defined reference constant, which we use here to pass in the window object. The intercept record (pointed to by the theMsg parameter) has this structure:

typedef struct QTVRInterceptRecord {
   SInt32      reserved1;
   SInt32      selector;
   SInt32      reserved2;
   SInt32      reserved3;
   SInt32      paramCount;
   void        *parameter[6];
} QTVRInterceptRecord, *QTVRInterceptPtr;

For present purposes, we need inspect only the selector field, which contains a constant that indicates which intercepted routine is being called. As you can see in Listing 6, we simply look for any calls to QTVRTriggerHotSpot and call the application-defined function MyPlaySound when we get one.

You can install an intercept procedure by calling the QTVRInstallInterceptProc function, as shown in Listing 7.

Listing 7: VRSample_InstallInterceptRoutine

void VRSample_InstallInterceptRoutine (
                               QTVRInstance theInstance, 
                               WindowObject theWindowObject)
{
   QTVRInterceptUPP   myInterceptProc;
   
   myInterceptProc = 
               NewQTVRInterceptProc(VRSample_InterceptRoutine);
   
   QTVRInstallInterceptProc(theInstance, 
                             kQTVRTriggerHotSpotSelector, 
                             myInterceptProc, 
                             (SInt32)theWindowObject, 0);
}

It seems like we've barely scratched the surface of the QuickTime VR Manager, but even so we've illustrated some very powerful capabilities for managing QuickTime VR movies programmatically. We've shown how to perform basic positioning of the viewer, how to alter the displayed image by drawing into a panorama's prescreen buffer, how to link the orientation of a listener in SoundSprocket's virtual audio environment to the QuickTime VR view angles, and how to intercept some QuickTime VR Manager functions. Not bad for just over a hundred lines of code!

We mentioned at the outset that the QuickTime VR Manager allows you integrate QuickTime movies and QuickDraw 3D objects with QuickTime VR panoramas and objects. Now that you've seen how to use the VR API, and particularly how to support SoundSprocket, you can probably figure out how to do at least the QuickDraw 3D integration yourself. If not, don't despair. Just have some patience until we show you - in our next article - how to play movies and render 3D objects in a QuickTime VR panorama.

Apple Computer, Inc. Virtual Reality Programming With QuickTime VR 2.0 (1997). Cupertino, CA.

Hoddie, Peter. "Somewhere in QuickTime: Basic Movie Playback Support". develop, The Apple Technical Journal, issue 18 (June 1994), pp. 22-25. Apple Computer's Developer Press.

Tim Monroe, monroe@apple.com, is a software engineer on the QuickTime VR team, responsible for developing sample code for the new QuickTime VR C language API. In his previous life at Apple, he worked on the Inside Macintosh team.

Bryce Wolfson, bwolfson@apple.com, is also a software engineer with Apple's QuickTime VR team. He's responsible for parts of the QTVR runtime's architecture, human interface, and application interaction, and has been known to occasionally write bits of over-commented sample code.