March 96 - Generating QuickTime VR Movies From QuickDraw 3D
Generating QuickTime VR Movies From QuickDraw 3D
Philip McBride
QuickTime VR is a new technology from Apple that provides users with a virtual
reality experience through interactive panoramic and object movies. You can
generate images for QuickTime VR movies with either a real camera or a
three-dimensional rendering system such as QuickDraw 3D. Here you'll learn how
to create images from QuickDraw 3D models and generate movies from these images
with the QuickTime VR Authoring Tools Suite version 1.0.
QuickTime VR lets you create two kinds of interactive virtual reality movies:
panoramic movies and object movies. In a panoramic movie, users can
interactively view a scene at nearly all camera angles from a particular point
in space, which gives them the impression of being there and looking around. In
an object movie, users can interactively spin an object around and thereby see
it from all sides. Panoramic and object movies can be linked together or used
separately.
QuickTime VR has several advantages over three-dimensional modeling systems for
making interactive movies. Its movie files are much smaller than complex 3D
models in situations where complete interactivity with the scene isn't
necessary, or where the scene contains complex objects or large numbers of
textures. With QuickTime VR, the complexity of the scene and the number of
textures used are irrelevant to runtime performance, so even users with
lower-end machines can effectively interact with the scene. Finally, a
QuickTime VR scene needs only a few megabytes of free space in memory, much
less than the enormous amount of RAM usually taken up by complex 3D scenes.
You can create QuickTime VR movies using either digitized images captured from
a real camera or synthetic images generated by a 3D rendering system, such as
QuickDraw 3D. In this article, you'll learn how to generate images with
QuickDraw 3D and convert them to QuickTime VR movies. To make a panoramic
movie, you create a panoramic image from a 3D scene, generate a linear
QuickTime movie from the image, and convert the linear movie to an interactive
panoramic movie using the QuickTime VR Authoring Tools Suite (ATS). (The ATS is
a set of tools that you use from within MPW, the Macintosh Programmer's
Workshop.) To create an object movie, you generate a series of images from a 3D
model, add the images to a linear QuickTime movie, and then use the QuickTime
VR ATS to convert the linear movie to an interactive object movie.
Before we get into the specifics of making movies, we'll explore the basic
concepts of QuickTime VR. We assume you have a general understanding of
QuickDraw 3D, which you can get by reading "QuickDraw 3D: A New Dimension for
Macintosh Graphics" in develop Issue 22 and "The Basics of QuickDraw 3D
Geometries" in develop Issue 23. You can learn all about QuickDraw 3D in the
book 3D Graphics Programming With QuickDraw 3D.
This issue's CD contains all the code necessary to generate panoramic images
and linear object movies from QuickDraw 3D models. For brevity, the listings in
the article omit error handling; the code on the CD includes the complete
versions of these functions.
QUICKTIME VR BASICS
The basic components of QuickTime VR movies are panoramas, nodes, objects, and
scenes.
- A panorama is an image spanning 360deg. or less in a real or virtual
scene. The image is viewed from a particular location in the scene, called a
node. A single-node panoramic movie enables a user to look in all directions
from that location.
- An object is an interactive item that can be viewed from all angles.
Object movies can be linked to panoramic movies in a scriptable authoring
environment, enabling a user to pick up and turn the objects from within a
panorama. Object movies can also be used independently of panoramic movies.
- In our case, the object of the movie is generated from a QuickDraw 3D
model, which contains a single geometric object (or group of objects); we'll
use the term model in this article to refer to the object in a QuickTime VR
object movie.
- A scene is a collection of several panoramas or nodes, a panorama with
one or more objects, or several panoramas and objects all linked together by
interactive hot spots. In a multinode scene, a user can navigate from node to
node to move about the scene.
SHOOTING AN OBJECT
For object movies, you need to photograph the model (or the real object) from
all directions, as shown in Figure 1. All vertical camera positions above the
center of the model are considered positive, and all positions below it are
considered negative. The vertical position with the camera directly above the
model looking down at it is called
vertical pan 90deg.; the vertical position
directly below and looking up is called
vertical pan -90deg..
Vertical pan
0deg. is at the model's center (equator). Horizontal positions are measured in
degrees from
horizontal pan 0deg. to
360deg..
Horizontal pan 0deg. is typically
at the back of the model.
Figure 1. Shooting a model (or a real object)
Images must be stored as frames in row order from top to bottom in a linear
QuickTime movie. For best results, we (along with the QuickTime VR
documentation) recommend that you have a frame every 10deg. between positions
in both the horizontal and vertical direction. If you shoot at increments
greater than 10deg., the motion of the model in the QuickTime VR movie will be
choppier when the user turns it. If you shoot at increments of less than
10deg., the motion will be smoother, but you'll need more disk space to store
all the frames. Whatever increment you choose, it should be consistent between
all horizontal and vertical frames for the object and divide evenly into the
horizontal and vertical pan ranges.
Your first frame at each horizontal position should be of the back of the
model, so that the frame showing the front of the model is halfway through the
series at that horizontal position. This improves disk access time at run time
since the user will most likely be looking at the front of the model.
SHOOTING A PANORAMA
If you're using a real camera to shoot a panorama, you need to take the
appropriate number of equally spaced pictures in a circle, as shown in Figure 2.
Figure 2. Shooting a panorama
Although this sounds simple, there are a few things you must be aware of.
First, you need to make sure you're taking the right number of shots for the
lens you're using (see Chapter 6 in Volume 1 of the QuickTime VR ATS
documentation for a full explanation of this). The entire camera rig should be
level at all times, and the nodal point of the lens should be directly over the
point of rotation of the rig. For best results, you should also have a
consistent overlap between images; the more overlap, the better (30% to 50% is
recommended). Finally, you should maintain consistency between images in each
panorama by using similar exposure and a fixed focus.
Because this can get quite complicated, Apple strongly recommends the use of a
professional photographer for making any production-quality titles. However,
one way around this is to use rendered data, as we do in this article. The
programmatic control we have over the "virtual" camera in a 3D environment such
as QuickDraw 3D eliminates all of the problems just mentioned.
MAKING MOVIES WITH THE SAMPLE CODE
The sample code on this issue's CD enables you to make object and panoramic
movies from any 3DMF file (a file that conforms to the QuickDraw 3D Object
Metafile standard). For either type of movie, the code creates a new document
record structure, reads in the model from a 3DMF file, renders the images, and
writes out the images in a form that the QuickTime VR tools can work with.
Here we'll look at the first few steps, which are common to both types of
movies. The other steps for making QuickTime VR movies -- rendering and writing
out the images and converting linear movies to interactive movies -- are
different for object and panoramic moviemaking and are described later.
CREATING A NEW DOCUMENT
All of our scene information is stored in a document record structure, shown in
Listing 1.
Listing 1. The document record structure for a scene
typedef struct _DocumentRecord {
CWindowPtr theWindow; // Display window
FSSpec theFileSpec; // Model file specification
short fRefNum; // and reference
GWorldPtr drawContextOffScreen; // Offscreen buffer
TQ3GroupObject documentGroup; // Main group for the document
TQ3ViewObject theView; // The document's view object
TQ3Matrix4x4 modelRotation; // The model transform
TQ3Point3D documentGroupCenter; // Center of the model
... // Miscellaneous view, model, and QuickTime file details
} DocumentRecord, *DocumentPtr, **DocumentHandle;
The
MyNewDocument function (Listing 2) creates the document record structure and
sets up the view, camera, and other elements associated with the scene. It also
adds the background buffer and window used to display the rendered images of
the scene.
Listing 2. Creating a new document record structure
DocumentPtr MyNewDocument()
{
DocumentPtr theDocument;
CWindowPtr theWindow;
TQ3DrawContextObject theDrawContext;
Rect myBounds = kMyBoundsRect;
TQ3CameraObject camera = NULL;
RGBColor blackColor = kMyBlackColor;
...
theDocument = (DocumentPtr)NewPtrClear(sizeof(DocumentRecord));
// Create the window for the document record and add references to
// the document record.
theWindow = (CWindowPtr)NewCWindow(0L, &myBounds,
"\pRendering Window", true, documentProc, (WindowPtr)-1L,
true, NULL);
theDocument->theWindow = theWindow;
// Create and set up the offscreen GWorld/context.
// ** Notice that QuickDraw 3D prefers direct color. **
NewGWorld(&theDocument->drawContextOffScreen, 32,
&theWindow->portRect, nil, nil, 0L);
...
SetGWorld(theDocument->drawContextOffScreen, nil);
EraseRect(&theDocument->drawContextOffScreen->portRect);
...
// Create the new pixmap draw context.
theDrawContext = MyNewDrawContext(theDocument);
// Create the view and set up the view attributes.
...
// Initialize the model rotation and transitions used for object
// movie rotations.
Q3Matrix4x4_SetIdentity(&theDocument->modelRotation);
// Add more model and view properties to the document record.
...
// Create the camera and add it to the view.
camera = MyNewCamera(theDocument->theWindow);
Q3View_SetCamera(theDocument->theView, camera);
Q3Object_Dispose(camera);
// Add the renderer to the view. Set the window's GWorld.
Q3View_SetRendererByType(theDocument->theView,
kQ3RendererTypeInteractive);
SetGWorld(theWindow, nil);
return (theDocument);
}
CREATING THE CAMERA
The camera used to render the images for the movies is created by the
MyNewCamera function, shown in Listing 3.
Listing 3. Creating the rendering camera
TQ3CameraObject MyNewCamera(CWindowPtr theWindow)
{
TQ3ViewAngleAspectCameraData perspectiveData;
TQ3CameraObject camera;
// For object movies, we set the field of view to 30 degrees (or
// 30.0*kQ3Pi/180.0 radians). For panoramic movies, we set it to
// 74 degrees (or 74*kQ3Pi/180.0 radians). QuickDraw 3D requires
// angles to be in radians, while QuickTime VR requires them to
// be in degrees.
float fieldOfView = 30.0*kQ3Pi/180.0;
TQ3Status returnVal = kQ3Failure;
// Assign default placement.
perspectiveData.cameraData.placement.cameraLocation
= kMyDefaultFrom;
perspectiveData.cameraData.placement.pointOfInterest
= kMyDefaultTo;
perspectiveData.cameraData.placement.upVector = kMyDefaultUp;
perspectiveData.cameraData.range.hither = kMyDefaultHither;
perspectiveData.cameraData.range.yon = kMyDefaultYon;
// Assign standard view port.
perspectiveData.cameraData.viewPort.origin.x = -1.0;
perspectiveData.cameraData.viewPort.origin.y = 1.0;
perspectiveData.cameraData.viewPort.width = 2.0;
perspectiveData.cameraData.viewPort.height = 2.0;
perspectiveData.fov = fieldOfView;
perspectiveData.aspectRatioXToY =
(float) (theWindow->portRect.right
- theWindow->portRect.left) /
(float) (theWindow->portRect.bottom
- theWindow->portRect.top);
camera = Q3ViewAngleAspectCamera_New(&perspectiveData);
return (camera);
}
For
object movies, we set the field of view to approximately 30deg.. This is not a
fixed number; you can use any number that you see fit, based generally on the
aspect ratio of your viewing window and how much information you'd like to
display inside it.
For panoramic movies, we set the field of view to 74deg.. This matches the
horizontal field of view of a 15mm lens for our image. We specify the
horizontal rather than vertical field of view since our image is taller than it
is wide (768 x 512 pixels), and QuickDraw 3D requires the field of view to be
specified as that of the shorter side of the image (whether width or height).
We calculate the horizontal field of view based on the size of our image and
the known vertical field of view of a 15mm lens (97deg., as specified in
Chapter 9 of the QuickTime VR ATS documentation).
READING IN THE MODEL
For the model to be read from a 3DMF file, you must first create 3D file and
storage objects associated with that file. Once they've been created, you build
up the model by reading in all the drawable objects from the file and adding
them to a group, as shown in Listing 4.
Listing 4. Reading in the model
TQ3Status MyReadScene(TQ3FileObject file, DocumentPtr theDocument)
{
TQ3Object object;
TQ3Boolean isEOF;
TQ3ViewObject view;
TQ3Object model;
TQ3GroupObject lightGroup = NULL;
// Create the new model and get the view.
model = Q3DisplayGroup_New();
theDocument->documentGroup = model;
view = theDocument->theView;
// Collect all drawable objects (into the model) and collect any
// lights (into the lightGroup).
while ((isEOF = Q3File_IsEndOfFile(file)) == kQ3False) {
object = Q3File_ReadObject(file);
if (Q3Object_IsDrawable(object))
Q3Group_AddObject(model, object);
if (Q3Object_IsType(object, kQ3SharedTypeViewHints))
if (view)
Q3ViewHints_GetLightGroup((TQ3ViewHintsObject)object,
&lightGroup);
if (object != NULL)
Q3Object_Dispose(object);
}
// Add any lights found to the view. Otherwise create default
// lights.
if (lightGroup) {
Q3View_SetLightGroup(view, lightGroup);
Q3Object_Dispose(lightGroup);
}
else
MyNewLights(theDocument);
Q3File_Close(file);
return kQ3Success;
}
If
the model includes any lighting, we use those lights; otherwise we create our
own lighting for the model.
GETTING THE DIMENSIONS OF THE MODEL
We must know the dimensions of the entire model as well as its center in order
to place the camera in its initial position and to guide both camera and model
transformations. You obtain the dimensions and center of an already constructed
model by getting the model's
bounding sphere with the function
MyGetBoundingSphere (Listing 5). The bounding sphere is another 3D object that
fully surrounds the model and has as its center the exact center of the model.
Listing 5. Getting the model's bounding sphere
void MyGetBoundingSphere(TQ3ViewObject viewObject, TQ3GroupObject
mainGroup, TQ3BoundingSphere *viewBSphere)
{
TQ3Status status;
Q3View_StartBoundingSphere(viewObject,
kQ3ComputeBoundsApproximate);
do {
status = Q3DisplayGroup_Submit(mainGroup, viewObject);
} while (Q3View_EndBoundingSphere(viewObject, viewBSphere) ==
kQ3ViewStatusRetraverse);
}
For
object movies, the bounding sphere has an additional purpose. Although a 3D
model from a QuickDraw 3DMF file may contain more than one geometric object, a
QuickTime VR object movie has only one geometric object or one group of
objects. Thus, we use the bounding sphere to get the dimensions of the entire
group of objects.
MAKING A QUICKTIME VR OBJECT MOVIE
Now we'll get into the specifics of making a QuickTime VR object movie. The
MyConvert3DMFToObject function (shown in Listing 6) drives the entire process,
from creating the new document to generating the linear object movie. You use
the QuickTime VR ATS to generate an interactive object movie from this linear
movie.
Listing 6. Converting 3DMF files to linear object movies
void MyConvert3DMFToObject(FSSpec *myFSS)
{
DocumentPtr theDocument;
// Create the document record and make the view, camera, lights,
// window, and so on.
theDocument = MyNewDocument();
// Read in the model and add it to the document record's group.
MyOpenFile(myFSS);
// Set up the initial camera position.
MyInitObjCamera(theDocument);
// Draw initial view to the screen.
MyDrawOffScreen(theDocument);
MyDrawOnScreen(theDocument);
// Assign the codec type.
theDocument->theCodecType = kMyCodec;
// Generate all the images and add them to the movie.
MyGenerateObjImages(theDocument, 36, 19, 360, 0, 90, -90);
...
}
DOING THE MODEL AND CAMERA WORK
Photographing a real object involves using a spherical camera rig to rotate a
camera around the object. For 3D models, it's just as easy to rotate the model
in front of a stationary camera. Furthermore, since the camera doesn't move in
this case, the lighting is easier to manage because it doesn't need to be
rotated with the camera (unless you want the object to appear to be lighted
from a certain angle).
In our case, we'll render images of the model by rotating the model around two
of its axes while the camera views it from the third axis; thus the camera gets
a view of the model from every angle. In our case, we'll place the camera along
the z axis and rotate the model around the x and y axes. The initial positions
of the camera and the model can be seen in Figure 3.
Figure 3. Initial positions of the camera and the model for object movies
The initial placement of the camera for an object movie is performed by the
function MyInitObjCamera, shown in Listing 7. First we get the bounding sphere
of the drawable group of the model. From this we can get the center 3D point of
the drawable group (as the origin of the bounding sphere) and the radius. From
the center point we place the camera a distance of five times the radius down
the z axis from the object.
Listing 7. Setting the initial camera position for an object movie
TQ3Point3D MyInitObjCamera(DocumentPtr theDocument)
{
TQ3BoundingSphere viewBSphere;
float viewRadius;
// Get the bounding sphere of the drawable group
// (the entire model) in the view.
MyGetBoundingSphere(theDocument->theView,
theDocument->documentGroup, &viewBSphere);
// Get the bounding sphere's center and radius.
theDocument->documentGroupCenter = viewBSphere.origin;
viewRadius = viewBSphere.radius;
// Position the camera down the z axis from the bounding sphere
// based on the center and the radius.
MyPlaceCamera(theDocument, theDocument->documentGroupCenter.x,
theDocument->documentGroupCenter.y,
theDocument->documentGroupCenter.z +
kMyDistanceFactor*viewRadius + 1.0,
theDocument->documentGroupCenter.x,
theDocument->documentGroupCenter.y,
theDocument->documentGroupCenter.z +
kMyDistanceFactor*viewRadius);
return (theDocument->documentGroupCenter);
}
We
rotate the model by repeatedly modifying the model's transform object. The
function MyRotateObjectX rotates the object around its
x axis (see Listing 8).
An analogous function that's not shown here, MyRotateObjectY, rotates it around
the
y axis.
Listing 8. Rotating the model for object rendering
void MyRotateObjectX(DocumentPtr theDocument, float angle)
{
TQ3Matrix4x4 tempMatrix;
Q3Matrix4x4_SetTranslate(&tempMatrix,
-theDocument->documentGroupCenter.x,
-theDocument->documentGroupCenter.y,
-theDocument->documentGroupCenter.z);
Q3Matrix4x4_Multiply(&theDocument->modelRotation, &tempMatrix,
&theDocument->modelRotation);
Q3Matrix4x4_SetRotate_XYZ(&tempMatrix, angle, 0.0, 0.0);
Q3Matrix4x4_Multiply(&theDocument->modelRotation, &tempMatrix,
&theDocument->modelRotation);
Q3Matrix4x4_SetTranslate(&tempMatrix,
theDocument->documentGroupCenter.x,
theDocument->documentGroupCenter.y,
theDocument->documentGroupCenter.z);
Q3Matrix4x4_Multiply(&theDocument->modelRotation, &tempMatrix,
&theDocument->modelRotation);
}
We
step through the model rotations to create the images needed for the linear
object movie with the MyGenerateObjImages function, shown in Listing 9. In this
function, we iterate over
y angles (rotating around the
x axis) while iterating
over
x angles (rotating around the
y axis). This stepping takes us from the
position yAngle = maxVPan, xAngle = minHPan to the position yAngle = minVPan,
xAngle = maxHPan - minHPan. At each step in the
x and
y angles, the model is
rotated and then rendered. The resulting images are added to a previously
created movie, as described in the next section. Note that QuickTime VR likes
angles in degrees and QuickDraw 3D likes angles in radians, so we have to do
these conversions.
Listing 9. Generating images for the linear object movie
void MyGenerateObjImages(DocumentPtr theDocument, short rows, short
columns, long maxHPan, long minHPan, long maxVPan, long minVPan)
{
float xStart, yStart, xStep, yStep, xAngle, yAngle;
// Assign stepping angles.
yStep = ((float)(maxVPan-minVPan))/(float)(rows-1);
xStep = ((float)(maxHPan-minHPan))/(float)(columns-1);
MyPrepareDestMovie(theDocument);
for (yAngle = maxVPan; yAngle >= minVPan; yAngle -= yStep) {
for (xAngle = 0; xAngle <= maxHPan-minHPan; xAngle += xStep) {
// Rotate the object to the correct position.
xStart = (-kQ3Pi*(xAngle -
((float)(maxHPan-minHPan))/2.0))/180.0;
yStart = kQ3Pi*((float)yAngle)/180.0;
MyRotateObjectY(theDocument, xStart);
MyRotateObjectX(theDocument, yStart);
// Render the model (to get a PixMap image).
MyDrawOffScreen(theDocument);
MyDrawOnScreen(theDocument);
// Add the rendered PixMap image to the movie.
MyAddImageToMovie(theDocument);
// Undo the rotation.
MyRotateObjectX(theDocument, -yStart);
MyRotateObjectY(theDocument, -xStart);
}
}
MyCloseDestMovie(theDocument);
}
CONSTRUCTING THE LINEAR OBJECT MOVIE
The destination movie is an ordinary QuickTime movie. The movie file, track,
and track media need to be set up before rendered images can be added to the
movie. The images are added as frames to a track. (See
Inside Macintosh:
QuickTime for more details about this process.) The movie is constructed in the
function MyPrepareDestMovie, shown in Listing 10.
Listing 10. Creating the destination linear object movie
OSErr MyPrepareDestMovie(DocumentPtr theDocument)
{
long keyFrameRate, compressedFrameSize;
short frameRate, width, height, i;
CodecComponent theCodec;
FSSpec theFSSpec;
TimeScale dstTimeScale;
TimeValue duration;
CodecQ spatialQuality;
Str255 movieName = "\pObjectMovie";
keyFrameRate = 1; // Every frame must be a key frame. If
// not, we'll get garbage around the edges
// of our objects when we rotate them.
// QuickTime refreshes only key frames
// completely.
theCodec = anyCodec; // We'll use what's there
spatialQuality = codecHighQuality; // and make it look pretty.
frameRate = 10; // This can be any value.
dstTimeScale = 600; // This can be any value that's a multiple
// of frameRate.
duration = dstTimeScale/frameRate;
width = theDocument->theWindow->portRect.right -
theDocument->theWindow->portRect.left;
height = theDocument->theWindow->portRect.bottom -
theDocument->theWindow->portRect.top;
theFSSpec = theDocument->theFileSpec;
BlockMove(movieName, theFSSpec.name, movieName[0]+1);
CreateMovieFile(&theFSSpec, creatorType, 0,
createMovieFileDeleteCurFile, &theDocument->dstMovieRefNum,
&theDocument->dstMovie);
theDocument->dstTrack = NewMovieTrack(theDocument->dstMovie,
((long)width) << 16, ((long)height) << 16, 0);
theDocument->dstMedia = NewTrackMedia(theDocument->dstTrack,
VideoMediaType, dstTimeScale, 0, 0);
BeginMediaEdits(theDocument->dstMedia);
GetMaxCompressionSize(theDocument->drawContextOffScreen
->portPixMap,
&theDocument->drawContextOffScreen->portRect, 32,
spatialQuality, theDocument->theCodecType, theCodec,
&compressedFrameSize);
theDocument->compressedData = NewHandle(compressedFrameSize);
theDocument->compressedDataPtr =
StripAddress(*(theDocument->compressedData));
HLock(theDocument->compressedData);
theDocument->idh = (ImageDescriptionHandle)NewHandle(4);
...
}
We
add the rendered images to the movie with the QuickTime function
AddMediaSample, which is called from within the function MyAddImageToMovie
(Listing 11). MyAddImageToMovie is called after each model rendering, as seen
earlier in the MyGenerateObjImages function.
Listing 11. Adding images to the linear object movie
OSErr MyAddImageToMovie(DocumentPtr theDocument)
{
CodecQ spatialQuality;
TimeValue duration, currentTime;
spatialQuality = codecHighQuality;
duration = 60;
LockPixels(theDocument->drawContextOffScreen->portPixMap);
CompressImage(theDocument->drawContextOffScreen->portPixMap,
&theDocument->drawContextOffScreen->portRect,
spatialQuality, theDocument->theCodecType, theDocument->idh,
theDocument->compressedDataPtr);
UnlockPixels(theDocument->drawContextOffScreen->portPixMap);
AddMediaSample(theDocument->dstMedia, theDocument->compressedData,
0, (**(theDocument->idh)).dataSize, duration,
(SampleDescriptionHandle)theDocument->idh, 1, 0,
¤tTime);
...
}
GENERATING THE INTERACTIVE OBJECT MOVIE
To generate the interactive object movie, open the linear movie you just
created with the Navigable Movie Player application (in the QuickTime VR ATS)
and choose the Add Navigable Data menu item. This brings up the dialog shown in
Figure 4. Fill in the fields with the values shown and click OK to change the
linear movie to an interactive movie. Turn the model to the position you want
it to be in at the beginning of the interactive movie, choose the Set Poster
View menu item, and you're done!
Figure 4. The Add Navigable Data dialog
MAKING A QUICKTIME VR PANORAMIC MOVIE
There are two approaches to creating panoramic movies. One way is to simulate a
real camera, rotate the camera to generate a series of images, and then
"stitch" the images together into a single 360deg. panoramic PICT file with the
QuickTime VR stitching tool. This panoramic picture file can be converted first
to a linear movie and then to an interactive movie with the QuickTime VR ATS.
This is the technique we'll look at first. You can also render a single
panoramic image directly; this avoids the need for the stitching tool and
enables us to convert the image into an interactive panoramic movie with only
one line of script. The setup is similar for both approaches.
Throughout the QuickTime VR documentation, examples and references assume a
vertically oriented, 360deg. full panorama that's 768 pixels across by 2496
pixels high, captured with a 15mm lens using portrait orientation. This is
exactly the panorama we'll create.
Before you begin making the movie, you need to determine the number of shots
required to make your panorama. For a real 360deg. panorama, the number of
shots is a function of the length of your lens and the amount of overlap
between the shots. For a rendered panorama, however, the number of shots is a
function only of the horizontal field of view. Because the camera position and
lighting conditions are controlled in the code, overlap between the shots isn't
necessary. You can specify any amount of overlap, but theoretically a one-pixel
overlap is all that's required.
In our case, we'll simulate the camera that's recommended in the QuickTime VR
documentation -- that is, a camera with a 15mm lens. To be consistent with the
examples in the documentation, we'll shoot 12 pictures, each 30deg. apart. This
will give us an overlap of about 50%.
The function MyConvert3DMFToPano (Listing 12) drives the entire panoramic
moviemaking process. Much like MyConvert3DMFToObject, this function creates a
document, reads in the model, and renders the appropriate images.
Listing 12. Converting 3DMF files to panoramic images
void MyConvert3DMFToPano(FSSpec *myFSS)
{
DocumentPtr theDocument;
// Create the document record.
theDocument = MyNewDocument();
// Read the model into the document record.
MyOpenFile(myFSS);
// Set up the initial camera position.
MyInitPanoCamera(theDocument);
// Draw initial view to the screen.
MyDrawOffScreen(theDocument);
MyDrawOnScreen(theDocument);
...
// Create a series of images to stitch together into a panorama.
MyGeneratePanoFrames(theDocument);
// To create a single panoramic image, call
// MyGeneratePanoMovieDirect instead of MyGeneratePanoFrames.
...
}
DOING THE MODEL AND CAMERA WORK
The initial placement of the camera is performed by the function
MyInitPanoCamera (Listing 13). It first gets the bounding sphere of the model's
drawable group, and from the bounding sphere gets the center 3D point of the
group (as the origin of the bounding sphere). Of course, you can place your
camera anywhere you like in the scene. For simplicity, we placed our camera in
the center. From that position, the camera is rotated to create the images.
Listing 13. Setting the initial camera position for a panoramic movie
TQ3Point3D MyInitPanoCamera(DocumentPtr theDocument)
{
TQ3BoundingSphere viewBSphere;
float viewRadius;
// Get the bounding sphere of the drawable group
// (the entire model) in the view.
MyGetBoundingSphere(theDocument->theView,
theDocument->documentGroup, &viewBSphere);
// Get the bounding sphere's center and radius.
theDocument->documentGroupCenter = viewBSphere.origin;
viewRadius = viewBSphere.radius;
// Position the camera in the center of the bounding sphere.
MyPlaceCamera(theDocument, theDocument->documentGroupCenter.x,
theDocument->documentGroupCenter.y,
theDocument->documentGroupCenter.z,
theDocument->documentGroupCenter.x,
theDocument->documentGroupCenter.y,
theDocument->documentGroupCenter.z + 1.0);
return (theDocument->documentGroupCenter);
}
For
panoramic rendering, we rotate the camera around its y axis with the function
MyRotateCameraY, shown in Listing 14. To do the rotation, we do the equivalent
of translating to local coordinates, rotate, and then translate back to world
coordinates. We do the transformation by making a rotation matrix about the
local y axis (the up vector) at our location, getting our local z axis (by
subtracting the point of interest from the current location), and then rotating
the z vector around the y axis. We then apply this transformation to get a new
point of interest. The rectangles encircling the camera in Figure 2 represent
the images rendered after each camera rotation.
Listing 14. Rotating the camera for panoramic rendering
void MyRotateCameraY(DocumentPtr theDocument, float dY)
{
TQ3CameraObject camera;
TQ3CameraPlacement cameraPos;
TQ3Matrix4x4 myRotation;
TQ3Vector3D initialVector, rotatedVector;
Q3View_GetCamera(theDocument->theView, &camera);
Q3Camera_GetPlacement(camera, &cameraPos);
// Get the z vector.
Q3Point3D_Subtract(&cameraPos.pointOfInterest,
&cameraPos.cameraLocation, &initialVector);
// Rotate around the y axis.
Q3Matrix4x4_SetRotateAboutAxis(&myRotation,
&cameraPos.cameraLocation, &cameraPos.upVector, dY);
// Rotate the z vector around the y axis.
Q3Vector3D_Transform(&initialVector, &myRotation, &rotatedVector);
// Get the point of interest from the rotated vector. The upVector
// doesn't change.
Q3Point3D_Vector3D_Add(&cameraPos.cameraLocation, &rotatedVector,
&cameraPos.pointOfInterest);
Q3Camera_SetPlacement(camera, &cameraPos);
Q3View_SetCamera(theDocument->theView, camera);
Q3Object_Dispose(camera);
}
To
create a series of images, we step through the camera rotations with the
MyGeneratePanoFrames function (Listing 15). Here we rotate the camera 30deg. at
a time, render an image, and write the image to a PICT file. This gives us 12
PICT files named 01 through 12 that can be used in a very straightforward
manner by the QuickTime VR ATS. To create a single panoramic image, you use the
function MyGeneratePanoMovieDirect, as shown later.
Listing 15. Generating a series of images for a panoramic movie
void MyGeneratePanoFrames(DocumentPtr theDocument)
{
PicHandle thePict;
float zAngle;
long counter = 0;
Str255 fName;
GWorldPtr gw;
GDHandle gd;
FSSpec outSpec;
GetGWorld(&gw, &gd);
SetGWorld(theDocument->theWindow, nil);
outSpec = theDocument->theFileSpec;
for (zAngle = 360; zAngle >0; zAngle -= 30) {
short i;
MyRotateCameraY(theDocument, -30.0*kQ3Pi/180.0);
SetGWorld(theDocument->theWindow, nil);
MyDrawOffScreen(theDocument);
MyDrawOnScreen(theDocument);
SetGWorld(theDocument->drawContextOffScreen, nil);
thePict =
OpenPicture(
&theDocument->drawContextOffScreen->portRect);
LockPixels(theDocument->drawContextOffScreen->portPixMap);
CopyBits((BitMap*)&theDocument->drawContextOffScreen->portPixMap,
(BitMap*)&theDocument->drawContextOffScreen->portPixMap,
&theDocument->drawContextOffScreen->portRect,
&theDocument->drawContextOffScreen->portRect,
srcCopy, NULL);
UnlockPixels(theDocument->drawContextOffScreen->portPixMap);
ClosePicture();
... // Set up the outSpec for the next image.
MySavePICT(thePict, &outSpec);
KillPicture(thePict);
}
...
}
GENERATING THE INTERACTIVE PANORAMIC MOVIE
You use the MPW tools and scripts provided in the QuickTime VR ATS to generate
your interactive panoramic movie. (See the QuickTime VR ATS documentation for
more information.)
Stitching the images. The stitching tool, called by the Stitch768 script, joins
the series of computer-rendered images of your panorama into a single 360deg.
panoramic PICT file. It's also used to stitch digitized photographic images
together. The following example shows a portion of a stitch worksheet, with
appropriate values set for each of the input variables. The stitching tool will
use images numbered from 01 to 12 located on the drive named HappyMac.
set panInFolder "HappyMac:RenderedFrames:"
set panOutFolder "HappyMac:RenderedFrames:"
set panRotate 0
set panX 100
set panDX 20
set panY 0
set panDY 10
export panInFolder panOutFolder panRotate panX panDX panY panDY
Stitch768 01-12 MyPano.srcPict
This
script stitches your images into a vertically oriented, 360deg. full panorama
that's 768 pixels across by 2496 pixels high, named MyPano.srcPict.
Dicing the image into a linear movie. The SrcPictToMovie script calls the
dicing tool, which compresses your PICT file and divides it into equal-sized
sections called tiles. For example, a standard-size panorama is "diced" into 24
tiles, 1 across by 24 down. Since we haven't included hot spots in our movie,
our worksheet will include only one line, which calls the SrcPictToMovie
script:
SrcPictToMovie "HappyMac:RenderedFrames:MyPano.srcPict"
[[partialdiff]] "HappyMac:RenderedFrames:MyPano.srcMoov"
This
script dices your 360deg. panoramic PICT into a standard QuickTime linear movie
using 1-by-24 tiling and the Cinepak compressor.
Converting the linear movie to an interactive movie. The MakeSingleNodeMovie
script takes the linear movie we just created and generates an interactive
panoramic movie. Since we're creating a very standard type of interactive
movie, this script does everything we need.
This example creates a single-node interactive panoramic movie file named
My3DMovie:
MakeSingleNodeMovie "MyPano.srcMooV" "My3DMovie"
RENDERING YOUR PANORAMA DIRECTLY
You can avoid using the stitching tool by rendering your panorama directly.
However, since QuickDraw 3D supports rendering directly to a plane but not to a
cylinder, we have to approximate cylindrical rendering with a "slit" approach,
using the cameras available to us.
The slit approach is the equivalent of using a real panoramic slit camera,
which spins around, taking very thin pictures and laying them next to each
other on the film. When simulating cylindrical rendering, we do the camera work
described earlier, but instead of rotating 30deg. at a time and grabbing each
frame, we rotate a very small amount each time and just grab a slit out of the
middle of each frame, thus approximating a cylinder. The narrower the slit
width, the closer we get to a true cylinder. If you're curious about the
mathematics of slit sizes, see "Calculating the Optimal Slit Width."
CALCULATING THE OPTIMAL SLIT WIDTH
In the slit approach to simulating cylindrical rendering for QuickTime VR
panoramic movies, narrower slits approximate cylinders better than wider ones.
In our calculations, the size of the error shows us the effect of increasingly
wide slits.
Our error is defined as the vertical distance between the top of our projection
plane at the maximum vertical field of view and the top of the cylinder we're
trying to approximate. We consider an error of less than 0.5 pixels to be
acceptable. Since fractional pixels can't be drawn, errors greater than 0.5
will round up to be a full pixel error. Because this error is so small, we can
use the same field of view for generating both the slits and the entire
frame.
To determine the vertical error, we must first determine the maximum horizontal
distance between the plane and the cylinder. This distance, labeled y, can be
seen in the top view of our camera and cylinder, as shown in Figure 5. The two
triangles formed are identical (except for their orientation). The width of our
slit is 2x.
Given that 2x is the width of the slit, y is the distance between the plane and
the cylinder, r is the radius of the cylinder, and [[alpha]] is the angle:
Therefore, where r is the radius of the cylinder and c is the circumference:
Since we know the final panorama we end up with is 2496 pixels wide, we can use
this as our circumference, and
However,
this only gives us the slit width for a given distance y, so we must concern
ourselves next with the important error, the vertical error, labeled Ev. A side
view of the panorama showing this error appears in Figure 6.
Ev is the distance in pixels between the pixel we see on the plane and the
pixel we see on the cylinder for a given field of view. Since we already have
an equation for y in terms of our slit width (2x), and we know that the
vertical field of view (FOVv) of the lens we're using is 97deg., we can easily
determine this error using the tangent equation
Since we know that our FOVv is 97deg., we have
which leaves us with a final equation of
The slit widths for various vertical errors are as follows:
Vertical error (Ev) Slit width (2x)
0.14228682 20
0.20490732 24
0.27892462 28
0.36434438 32
0.4611731 36
0.56941818 40
For a slit width of about 38, we have an error of less than 0.5. Theoretically,
this should yield accurate pictures. Therefore, for panoramas that are 2496
pixels wide, like ours, the optimal slit width is 32 (the largest factor of
2496 that's still less than 38).
Figure 5. Determining y, the distance between the plane and the cylinder
Figure 6. Determining Ev, the vertical error
In our case, the largest slit size that gives us a very small amount of error
is 32, so we use this number to generate our panorama (Listing 16). To try
different sizes, simply put in a different number for the slit size constant,
which we've called factor.
Listing 16. Rendering the panorama directly
#define factor 32.0
void MyGeneratePanoMovieDirect(DocumentPtr theDocument)
{
PicHandle thePict;
float zAngle;
short i;
GWorldPtr gw, largeGW;
GDHandle gd;
FSSpec outSpec;
Rect sourceRect, destRect, largeRect = {0, 0, 768, 2496};
GetGWorld(&gw, &gd);
SetGWorld(theDocument->theWindow, nil);
outSpec = theDocument->theFileSpec;
NewGWorld(&largeGW, 32, &largeRect, nil, nil, useTempMem);
LockPixels(largeGW->portPixMap);
SetGWorld(largeGW, nil);
EraseRect(&largeRect);
sourceRect = destRect = largeRect;
sourceRect.left = 256 - factor/2.0;
sourceRect.right = sourceRect.left + (short)factor;
destRect.left = 0;
destRect.right = (short)factor;
for (zAngle = 360.0; zAngle > 0.0;
zAngle -= 360.0/(2496.0/factor)) {
MyRotateCameraY(theDocument, -2*kQ3Pi/(2496.0/factor));
SetGWorld(theDocument->theWindow, nil);
MyDrawOffScreen(theDocument);
MyDrawOnScreen(theDocument);
SetGWorld(largeGW, nil);
LockPixels(theDocument->drawContextOffScreen->portPixMap);
CopyBits(
(BitMap*)&theDocument->drawContextOffScreen->portPixMap,
(BitMap*)&largeGW->portPixMap, &sourceRect, &destRect,
srcCopy, NULL);
UnlockPixels(theDocument->drawContextOffScreen->portPixMap);
destRect.left = destRect.left + (short)factor;
destRect.right = destRect.right + (short)factor;
}
SetGWorld(largeGW, nil);
thePict = OpenPicture(&largeGW->portRect);
CopyBits(
(BitMap*)&largeGW->portPixMap, (BitMap*)&largeGW
->portPixMap,
&largeGW->portRect, &largeGW->portRect, srcCopy, NULL);
ClosePicture();
UnlockPixels(largeGW->portPixMap);
DisposeGWorld(largeGW);
MySavePICT(thePict, &outSpec);
...
}
The
PICT file you get from this operation is oriented horizontally. However, the
QuickTime VR tools expect the stitching tool output to be vertical, so you
first need to rotate your PICT clockwise 90deg. using a PICT editor such as
Photoshop or PhotoFlash. You then use the SrcPictToMovie and
MakeSingleNodeMovie scripts as described above to turn the PICT into an
interactive panoramic movie.
THE NEXT STEPS
So far we've made QuickTime linear object movies and panoramic PICT files that
can be converted to interactive movies with the QuickTime VR ATS. There are a
number of directions you can go from here. If you have your own panoramic
renderer, you may want to substitute it for our slit-based rendering. Or you
may want to build a full interface to allow the user to place the camera and
set up all the parameters involved in QuickTime VR moviemaking. We hope to
write a future article about the QuickTime VR movie file formats and how to
write out QuickTime VR movie files.
QuickTime VR movies already have several diverse uses. Developers with
extensive collections of 3D data sets can generate QuickTime VR movies from
their data sets to show to potential customers; the movies display the modeled
objects more effectively than a 2D representation and don't compromise the data
in the process. Archaeologists can use QuickTime VR movies to record site
information during digs, realtors can use them to give clients virtual tours
through the property they're offering, and cities can use them to provide
tourist information on kiosks. Museums can archive or display their collections
with QuickTime VR movies. For example, Apple and the Asian Art Museum of San
Francisco have put together a virtual walkthrough of one of the museum's
special exhibits; you can check it out on the World Wide Web at
http://sfasian.apple.com. Also, for the latest information on QuickTime VR, see
http://qtvr.quicktime.apple.com. Use your imagination -- the possibilities are
endless!
RECOMMENDED READING
- "QuickDraw 3D: A New Dimension for Macintosh Graphics" by Pablo Fernicola
and Nick Thompson, develop Issue 22.
- "The Basics of QuickDraw 3D Geometries" by Nick Thompson and Pablo
Fernicola, develop Issue 23.
- 3D Graphics Programming With QuickDraw 3D (Addison-Wesley, 1995).
- Inside Macintosh: QuickTime and Inside Macintosh: QuickTime Components
(Addison-Wesley, 1993).
- One Hundred Years of Solitude by Gabriel García Márquez
(Harper & Row, 1970). This won't directly help you with QuickTime VR or
QuickDraw 3D, but it's Philip's favorite novel.
PETE FALCO (AppleLink FALCOP) is a member of Apple's QuickTime VR team. Since
finishing school at Rensselaer Polytechnic Institute in upstate New York, where
he spent the last six years in the rainy, snowy weather of Troy, he's found the
sunny weather of California a welcome treat and vows he'll never leave this
area. His latest projects include working on the next release of QuickTime VR
as well as integrating all of Apple's multimedia technologies with QuickTime
VR.*
PHILIP MCBRIDE (mcbride@apple.com) has been working on multimedia tools and the
underlying media technologies since he's been at Apple. Most recently this
included helping to add QuickTime VR to Apple Media Tool 2.0. While not working
with digital multimedia, Philip likes to work with real multimedia by
sculpting. In fact, he's been developing a new product that will involve clay,
a mouse, and a bottle of cabernet. The details are sketchy, but he has a cool
T-shirt for it.*
Thanks to our technical reviewers Eric Chen, Michael Chen, Ken Doyle, Ian
Small, and Nick Thompson. Special thanks to Chris Flick and Pablo Fernicola.*
