VIDEO DIGITIZING UNDER QUICKTIME

CASEY KING AND GARY WOODCOCK

With the introduction of the 'vdig' component in QuickTime 1.0, Apple established an API that encompassed the critical features of video digitizing hardware. This article takes a closer look at the more complex aspects of the QuickTime 1.0 interface and at the new functionality provided by QuickTime 1.5. Thanks to the QuickTime framework, application writers and video digitizer developers have begun to deliver the kind of high-performance solutions we've all been waiting for. And the evolution of this technology has just begun.

Video digitizing hardware on the Macintosh is not new. Way back in 1990, dozens of hardware developers were offering video digitizing cards that they hoped would score big in that elusive but potentially lucrative market called multimedia. However, because these products targeted different niches and had different feature sets, there was a lot of chaos and redundancy in the marketplace. Each video digitizer card had its own API. A multimedia software developer who wanted to support more than one specific card had to either write a single application that included code for the API of every card on the market or else release multiple versions of an application, one version for each card. The inefficiency of this situation kept developers from introducing innovative products quickly. Users, for their part, were confused by the vast differences in the user interfaces of various products and in the capabilities of the hardware.

The introduction of QuickTime in December 1991 changed all this by providing a standard video digitizing interface. The component nature of QuickTime allowed video digitizer manufacturers to concentrate on making value-added hardware and software, secure in the knowledge that their products would work with whatever general-purpose video capture and editing applications were out there. Application writers could at last code to a standard interface and take advantage of improvements in the underlying hardware as they came along. Customers also benefited from a standard, and usually simplified, interaction with these devices.

So much for history! This article takes a look at the present and future of QuickTime video digitizing from two perspectives -- that of the digitizer developer and that of the video application developer (although the video neophyte will find valuable information here as well). The article focuses on the less understood areas of the 'vdig' interface and the new features in QuickTime 1.5. Because a discussion of video digitizing under QuickTime would be incomplete without a look at the video digitizer's main client, the sequence grabber, we also briefly examine this powerful component. We conclude with a wish list of features for the next generation of video digitizing products.

To get the most out of this article, you need to be familiar with QuickTime components in general, video digitizer components specifically, and some basic video terminology. For an overview of components, read our article "Techniques for Writing and Debugging Components" in Issue 12 ofdevelop. We also suggest that you read Chapter 7, "Video Digitizer Components," in Inside Macintosh: QuickTime Components(which is included in the QuickTime Developer's Kit v.1.5). Finally, while we've attempted to interject definitions of some basic video terms, the References box at the end of this article lists some of our favorite books on video. If you're really interested in dazzling your friends with your new-found video expertise, you'll want to investigate some of these books.

This issue's CD contains two pieces of sample code related to this article. The 'vdig' code is an example of a software-only digitizer. You can use it as a template to write your own video digitizer components or as a vehicle for testing grab applications when you don't have any video digitizing hardware. The sample application, HackTV, shows how to use the sequence grabber to preview and record movies. HackTV can use either the software 'vdig' provided or a hardware 'vdig' that you may already own. HackTV can also be used by 'vdig' makers to test their code.

THE VIDEO CAPTURE PROCESS

As Figure 1 shows, the process of making movies involves several components. The sequence grabber component (component type 'barg') plays an especially critical role. It's responsible for coordinating the activities of the lower-level components to achieve different results -- like displaying video in a window, grabbing a single picture, or grabbing a movie. By protecting application developers from having to deal with the low-level management of the video digitizer, the sequence grabber makes it much easier to incorporate video input capabilities in applications. Note that the sequence grabber also handles audio input devices and synchronization of picture and sound. For the sake of simplicity, these tasks aren't shown in Figure 1.

Figure 1 Grabbing Video -- The Big Picture

Data flow in the video digitizing pipeline begins with an analog video source like a video camera, but it could also be a VCR, a laser disc, or a direct broadcast feed. The video digitizer's primary purpose in life is to convert the analog signal into a digital form that can be either displayed in the frame buffer or processed further by an image compressor. The video digitizing hardware can optionally perform resizing, color conversion, or clipping. In the absence of hardware support for theseoperations, the sequence grabber will sometimes provide them, as we'll explain later. If the application's request is for video in a window, the process is complete. (From a video digitizing perspective, displaying live video in a window is calledplay through, and capturing video to a movie is called capturing or grabbing. The sequence grabber calls these operationspreviewingand recording, respectively.) When a movie is requested, an image compressor processes the data further, and the result is stored either in system memory or to disk.

An important restriction of the QuickTime 1.0 video capture process is that the digitizer is limited by the image compressor. Since the grabbed image has to be perfectly still while the compressor is working, the digitizer can't grab the next frame until the compressor is finished with the current one; otherwise, there will be frame tears. Thus, the speed of the compressor has a significant influence on the effective capture rate of the digitizer. (For a discussion of industry-standard frame rates and acceptable QuickTime capture rates, see "Frame Rates and Motion Quality.") One of the biggest challenges in making a movie is figuring out how to compress the data fast enough to maintain a good effective capture rate.

A CLOSER LOOK AT SOME 'VDIG' BASICS

In this section, we explore a few QuickTime 'vdig' topics that seem to give developers the most trouble when they first undertake the task of writing a video digitizer component. WhileInside Macintoshis the definitive reference source for all the information we present here and for all of QuickTime, this section will give you additional insight into the more difficult aspects of rolling your own 'vdig'.

To illustrate some specific points, this section presents code excerpts from our software implementation of a 'vdig'. In practice, how you write a video digitizer component depends heavily on your particular digitizer hardware implementation. For the sake of illustration, however, we'll simulate some hardware features with software that provides roughly equivalent functionality.

GETTING VIDEO COORDINATE SYSTEMS STRAIGHT
The coordinate system for video digitizers can seem confusing, but it's actually quite straightforward. The common mistake is to try to map the digitizer coordinate system onto the QuickDraw global coordinate system, even though the two aren't related. The critical point to keep in mind is that when referencing the video source, you're working in a coordinate system that's specific to your digitizing hardware. All cropping rectangles are relative to this coordinate system.

Figure 2 shows the four rectangles that define the video source. The MaxSrcRect defines the maximum source area that the digitizer is capable of grabbing. Typically this area includes all or portions of the vertical and horizontal blanking areas. Note that you don't have to define the top left point of MaxSrcRect as 0,0; this is an entirely arbitrary reference point that 'vdig' developers can define as they choose. The other three rectangles are defined in relation to MaxSrcRect. The ActiveSrcRect is the region of the maximum source rectangle that contains the actual video image. The first pixel of active video is the top left corner, and the last pixel is the bottom right.

Figure 2 A Video Digitizer Coordinate System

The DigitizerRect describes the area of the MaxSrcRect to be captured -- the image that the user will actually see, although it hasn't been scaled yet. The DigitizerRect defaults to the area of the ActiveSrcRect; to describe a cropped image, the DigitizerRect is usually defined as a portion of the ActiveSrcRect. It's not uncommon for part of the blanking signal to be displayed in the ActiveSrcRect. This is because different source devices -- like VCRs, laser discs, and broadcast signals -- send out slightly different analog signals. To align the image, a 'vdig' client can nudge the DigitizerRect a few pixels in the appropriate direction using VDSetDigitizerRect.

The last rectangle, VBlankRect, defines the area of vertical blanking. This region can contain vertical interval time code (VITC), closed captioning, and teletext. For those video dweebs out there, this corresponds to lines 10 to 19 of each field of the incoming video.

Remember, all rectangle coordinates are relative to MaxSrcRect. MaxSrcRect, ActiveSrcRect, and VBlankRect are always fixed and hardware dependent. The only control that a client has is in the definition of DigitizerRect. The following code shows how to implement the VDGetMaxSrcRect call for a 'vdig'. Implementing the VDGetActiveSrcRect and VDGetVBlankRect calls is very similar.

pascal VideoDigitizerError  GetMaxSrcRect
        (Handle storage, short inputStd, Rect *maxSrcRect)
{
    long        error = noErr;

    if (inputStd == ntscIn)     // Example supports only NTSC.
        SetRect(maxSrcRect, 0, 0, kMaxHorNTSCIn,
            kMaxVerNTSCIn+kVerBlank);
    else 
        error = paramErr;
    if (!error) 
        (**storage).maxSrcRect = *maxSrcRect;
    return (error); 
}

SetDigitizerRect, shown below, is also very straightforward. Notice that the DigitizerRect must fully intersect the MaxSrcRect.

pascal VideoDigitizerError vdigSetDigitizerRect
        (vdigGlobals storage, Rect *digiRect)
{
    Rect    tempR;

    // Can't be empty.
    if (!digiRect || EmptyRect(digiRect)) return (paramErr);
    // They must intersect . . .
    if (!SectRect(digiRect, &(**storage).maxSrcRect, &tempR))
        return (paramErr);
    // . . . completely.
    if (!EqualRect(digiRect, &tempR)) return (paramErr);
    (**storage).digiRect = *digiRect;
    . . .
    // Insert hardware-dependent code to crop.
    . . .
    return noErr;
}

One more very important point to understand about video coordinate systems is that scaling and translation can be specified either as a matrix or as a destination rectangle. Your digitizer must support both transformation methods. If the matrix is nil, use the destination rectangle; otherwise, ignore it and use the matrix. The 'vdig' must offset the top left of the DigitizerRect to 0,0 before applying the matrix, or the video won't be positioned correctly. This isn't necessary when using the destination rectangle. By the way, the sequence grabber uses the destination rectangle, and not the matrix, to specify scaling and translation, although this may change at any time.

ACCELERATING THE FRAME CAPTURE PROCESS
Video digitizer clients -- such as an application or, more likely, the sequence grabber -- that want to configure a 'vdig' for video play through can do so by specifying a video source rectangle with VDSetDigitizerRect and a destination with VDSetPlayThruDestination. The live video stream is then enabled via the VDSetPlayThruOnOff call. Clients that want to capture and store images to make a QuickTime movie, on the other hand, must deal with the tradeoff between compression time and capture rate described earlier. This section explains how a video digitizer client and a 'vdig' can cooperate to maximize the effective capture rate and thus the overall performance of a digitizing system.

At the most basic level, the frame capture process involves grabbing a frame, compressing it, appending the frame to a movie, grabbing another frame, compressing it, and so on. Unfortunately, while this synchronous frame grabbing (with the VDGrabOneFrame call) is fine for single grabs, it's too slow for capturing moving images. The real killer when using the synchronous call is that no parallel processing can occur. Once the call to the video digitizer is made, the entire system is brought to a stop waiting for the completion of the frame grab. Since the digitizer must synchronize to the incoming video's vertical blanking interval, this can mean waiting the duration of up to one or two video fields (33,333 to 66,666 microseconds for NTSC video). What's more, because the video stream must be stopped at the conclusion of the call and restarted to grab another frame, the actual wait between synchronous grabs tends to be even longer.

A video digitizer client can achieve parallel processing, and thus better performance, if it grabs asynchronously, sending the image either to a single buffer or to a series of buffers. If you use just one buffer, you get a slight performance improvement over a synchronous grab. The VDGrabOneFrameAsync call tells the digitizer to kick off a frame grab and to return immediately to the caller (this typically takes around 500 microseconds). The compressor can be turned loose on the frame being grabbed without waiting for the grab to be complete. (Suchbeam chasing assumes that the compressor is slower than the video digitizer and won't catch it. Most software compressors today are in fact slower than digitizers.) Once the compressor is finished with the frame, the client makes the VDDone call to determine whether the first frame grab is complete before requesting another one. This process continues until recording is terminated.

One problem with such single-buffer asynchronous grabs is that the client must still start and stop the video stream with each call, incurring the same overhead as in a synchronous grab. An even more serious problem is that the 'vdig' client must handle frame synchronization between the digitizer and compressor; otherwise, the resulting movie will contain annoying frame tears. Why does this happen? Suppose the VDGrabOneFrameAsync call is made while the digitizer is in the middle of a frame. The digitizer will honor the request and return immediately, but the video won't really start until the beginning of the next field. If the compressor is turned loose as soon as the digitizer returns, it will be compressing stale data. And depending on how slow the compressor is, new digitizer data may overwrite the stale data in midstream, all of which is bad news.

Clients of your 'vdig' can attain still better performance if the 'vdig' implements multiple buffers. By constantly ping-ponging between two or more buffers, the 'vdig' can increase the effective capture rate, potentially to the maximum rate. The process works roughly as follows: A 'vdig' client initializes the process by making the video digitizer call VDSetupBuffers (we'll look at this in more detail momentarily). The client then begins the capture process by calling VDGrabOneFrameAsync to fill buffer 1 with a frame and to start the next grab into buffer 2. The client calls VDDone to make sure that the frame grab in buffer 1 is complete. Next, the compressor compresses the frame in buffer 1. Then VDGrabOneFrameAsync is called to send the next digitized frame back into buffer 1, and the compressor starts compressing the contents of buffer 2. The whole sequence is repeated for successive frames.

If you think two buffers are a snap, that's great, because in practice there are often three. Figure 3 shows a snapshot of the asynchronous grab process when three buffers are implemented. The number of buffers your 'vdig' needs to implement, by the way, is typically equal to the number ofconcurrent operations a video digitizing system must perform on those buffers. The example shown in Figure 3 uses three buffers to supportinterframe compression. This technique, which is also known as frame differencing, makes it possible for a compressor to eliminate the redundant data in a sequence of frames -- essentially the parts of the image that stay the same from one frame to the next. At specified intervals, all the data in a frame is saved; this frame is known as akey frame. Depending on the content of the image sequence, frame differencing can provide a substantial increase in compression efficiency.

Figure 3 shows what's going on in three hypothetical buffers at a specific moment in time. The whole process of grabbing a frame from the digitizer and compressing it with frame differencing involves the following steps:

Figure 3 Multiple Buffers in Motion

1. The 'vdig' initially uses a buffer -- in this case, buffer 1 -- as a play-through destination.
2. The sequence grabber makes the VDGrabOneFrameAsync call with the next available buffer, buffer 2.
3. The 'vdig' receives VDGrabOneFrameAsync, returns to the sequence grabber, and begins the asynchronous grab -- in this case, completing a frame grab into buffer 1. Upon completion it's available to begin digitizing into buffer 2.
4. The sequence grabber uses VDDone to find out when the frame grab into buffer 1 is complete.
5. The sequence grabber makes another VDGrabOneFrameAsync call. The 'vdig' completes the frame grab into buffer 2 and then is available to begin digitizing into buffer 3.
6. The sequence grabber calls the Image Compression Manager to compress the completed frame, while the 'vdig' continues digitizing.
7. The sequence grabber writes the compressed frame to disk.
8. The sequence grabber repeats the basic process outlined in steps 4 through 7, this time using buffer 3 for the next buffer, compressing buffer 2, and using buffer 1 as the first key frame for frame differencing.
9. The sequence grabber repeats the process outlined in steps 4 through 7, using buffer 1 for the next buffer, compressing buffer 3, and using buffer 2 as the next key frame for frame differencing.
10. The 'vdig' is returned to play-through mode when the record operation is completed. A movie file is created.

A word of caution: A common mistake in using VDGrabOneFrameAsync is to pass in the current buffer to be worked on rather than the next destination buffer. By telling the digitizer in advance where the next frame should be placed, you give the digitizer all the information it needs to do a fast buffer change once the current frame grab is complete. The time required to reposition the video buffer is less than the time in a vertical blanking period, so the digitizer theoretically has enough time to grab every frame without the need for resynchronization. In this case, the obstacles to achieving the maximum capture rate are the other processes involved, such as compression and disk access speeds. Note that there's no need to turn the digitizer on and off with multiple buffers because the video frame in buffer 1 will in effect be frozen once the digitizer repositions its free-running output to buffer 2.

Now let's take a step back and look at the interaction between the video digitizer client and the 'vdig' during buffer initialization. Figure 4 shows two possible scenarios. In scenario A, the card has local memory available for multiple buffering but doesn't support generic hardware DMA. (Digitizers that support hardware DMA can send the video data to any available memory; they aren't restricted to local memory.) In scenario B, the digitizer supports hardware DMA but doesn't have local memory for multiple buffering.

Figure 4 Initializing Multiple Buffers -- Two Scenarios

For both scenarios in Figure 4, the basic sequence of events is similar.

1. The sequence grabber uses VDSetPlayThruDestination to set up the first destination, which happens to be visible to the user on a monitor.
2. The sequence grabber determines how many buffers to allocate based on the video characteristics, the compression settings, and the amount of memory available for multiple buffering.
3. The sequence grabber uses VDSetupBuffers to tell the video digitizer how to partition the buffers.

The differences between the scenarios depicted in Figure 4 are subtle but very important to understand. First, in scenario A, the sequence grabber uses the integrated frame buffer when setting up the play-through destination. In scenario B, where the digitizer is more flexible in its ability to redirect video data, the sequence grabber can make the play-through destination any frame buffer -- that is, any screen -- that the user chooses.

The second difference between the two scenarios in Figure 4 is in how the sequence grabber finds available memory and partitions that memory into multiple buffers. Because the digitizer in scenario A doesn't support hardware DMA, local memory on the card is the only memory available for multiple buffering. The sequence grabber will make the VDGetMaxAuxBuffer call to determine the maximum usable memory available. In our example, three buffers are allocated. If sufficient space for three buffers isn't available, the sequence grabber will make the buffer sizes smaller and use software to expand the frame to the desired size. In contrast, the video digitizer in scenario B supports hardware DMA and doesn't have any local buffering, so the sequence grabber will use system memory for buffer allocation. In this case the size of the buffers is limited by the amount of free memory available. Once the number and size of the buffers have been determined, VDSetupBuffers is used in both cases to communicate this information to the 'vdig'.

To sum up, a 'vdig' client looks at two video digitizer capabilities to determine how to initialize buffers -- the availability of local off-screen memory and the digitizer's ability to do generic hardware DMA. While there are actually four possible combinations of these two capabilities, the two scenarios shown in Figure 4 define the major options for initializing multiple buffers. The two combinations not shown in the figure are a 'vdig' that doesn't provide any local off-screen memory and doesn't support hardware DMA, and a 'vdig' that supports both capabilities. If a 'vdig' has neither capability, it's not a very interesting digitizer -- just consider yourself lucky that it can digitize video into a window at all. At the other end of the spectrum, the increasing numbers of digitizers that support both local memory and hardware DMA areveryinteresting. With these digitizers, buffers are allocated in the local off-screen memory first for performance reasons, while hardware DMA is normally used to display video in a window on any screen.

At this point you're probably asking yourself, "Do I really need to understand this stuff?" The answer for video digitizer developers is a resounding "Yes," especially if you want to squeeze out every last drop of performance. For those interested only in the general concepts, we've probably led you down a road you wish you hadn't embarked on. Sometimes the quest for new knowledge hurts!

If you're writing a digitizer, you'll be interested in the following code, which shows how to write the routines that support multiple-buffer asynchronous frame grabs. Notice that in the vdigSetupBuffers routine, pendingAsyncBuffer is used to keep track of the next buffer to be grabbed. Here it's initialized to -1, which is not a valid buffer number.

pascal VideoDigitizerError vdigSetupBuffers
        (vdigGlobals storage, VdigBufferRecListHandle bufferList)
{
    OSErr           err;
    MatrixRecord    matrix;
    short           i;
    RgnHandle       clipRgn;

    if (!bufferList) return paramErr;       // Can't have empty list.
    if (!(**bufferList).count) return paramErr;
                                            // Can't have 0 buffers.
    // Dispose of any buffers previously created.
    if ((**storage).bufferList) {       
        DisposeHandle((Handle)(**storage).bufferList);
        (**storage).bufferList = 0;
    }
    // Same with any clipRgn previously created.
    if ((**storage).clipRgn) {
        DisposeRgn((**storage).clipRgn);
        (**storage).clipRgn = 0;
    }
    // Don't accept a mask if the 'vdig' can't clip.
    if (!gCanClip && (**bufferList).mask) return paramErr;
    // Copy the matrix if it exists.
    if ((**bufferList).matrix)
        matrix = *(**bufferList).matrix;
    // Make a local copy. 
    HandToHand((Handle *)&bufferList);
    if (err = MemError()) return err;
    if (clipRgn = (**bufferList).mask) {
        HandToHand((Handle *)&clipRgn);
        if (err = MemError()) return err;
        (**storage).clipOrigin = (**bufferList).list[0].location;
    }
    // Save the important stuff in private storage for later
    // retrieval.
    (**storage).bufferList = bufferList;
    (**storage).clipRgn = clipRgn;
    (**storage).pendingAsyncBuffer = -1;
    (**storage).matrix = matrix;
    // Do the important error checking when it's not
    // performance-critical.
    for (i=0; i < (**bufferList).count; i++) {   
        if (!validatePixMap(storage, (**bufferList).list[i].dest))
            return paramErr;
    }
    return noErr;
}

The vdigGrabOneFrameAsync routine (below) is continually being called by the sequence grabber once movie making starts. After some simple error checking, the software digitizer draws the frame at the appropriate destination. The buffer used in drawVideoFrame is the pending buffer that was passed into this routine on the last call. While this isn't so important with the software-only video digitizer illustrated here, it can give hardware digitizers some additional time to get things switched over and started in anticipation of the next vdigGrabOneFrameAsync call.

pascal VideoDigitizerError vdigGrabOneFrameAsync
        (vdigGlobals storage, short buffer)
{
    VdigBufferRecListHandle bufferList;
    // Bail if you can't do asynchronous grabs.
    if (!gCanAsync) return digiUnimpErr;
    // Make sure the buffer list is set up first with VDSetupBuffers.
    if (!(bufferList = (**storage).bufferList)) return badCallOrder;
    // Make sure the buffer request is within bounds.
    if (buffer > (**bufferList).count) return paramErr;
    // Get the buffer to draw into from the last one saved.
    // This is the one saved in pendingAsyncBuffer.
    if ((**storage).pendingAsyncBuffer != -1) {
        short aBuf = (**storage).pendingAsyncBuffer;
        if (aBuf == buffer) 
            DebugStr("\pasync grab into incomplete buffer");
        drawVideoFrame(storage, (**bufferList).list[aBuf].location,
            (**bufferList).list[aBuf].dest);
    }
    // Set up the next buffer to use when called.
    (**storage).pendingAsyncBuffer = buffer;
    return noErr;
}

IDENTIFYING DIGITIZER TYPES AND CAPABILITIES
No two video digitizers are alike. To make sure your 'vdig' component works smoothly with QuickTime, it's critical to identify the capabilities your hardware provides.

The 'vdig' component interface attempts to be very flexible in allowing you to indicate what your card can do. A 'vdig' specifies its type and capabilities in the DigitizerInfo structure, shown below. Two calls -- VDGetDigitzerInfo and VDGetCurrentFlags -- give a client (normally the sequence grabber) access to information contained in this structure.

typedef struct {
    short       vdigType;           
    long        inputCapabilityFlags;
    long        outputCapabilityFlags;  
    long        inputCurrentFlags;
    long        outputCurrentFlags;
    short       slot;                   
    GDHandle    gdh;           // For vdigs with preferred screen 
    GDHandle    maskgdh;       // For vdigs that have mask planes 
    short       minDestHeight; // Smallest resizable height 
    short       minDestWidth;  // Smallest resizable width 
    short       maxDestHeight; // Largest resizable height 
    short       maxDestWidth;  // Largest resizable height 
    short       blendLevels;   // # of blend levels = 2 if 1-bit mask
    long        reserved;           
} DigitizerInfo;

In the vdigType field, you specify which of the four types of digitizer you are. Either you're a basic rectangular digitizing device or you're a device that supports clipping -- an alpha channel device, a mask plane device, or a key color device. In the capability flags fields, you indicate to clients what capabilities a particular digitizer instance provides.

The current flags fields have the same attribute bit fields as the capability flags, but they indicate the currently available capabilities, not the total possible capabilities. By nature, some capabilities are mutually exclusive. For instance, if you support NTSC and PAL input formats, at any given time you're actively doing only one of them. The bit corresponding to the active standard is the one that would be set in the current flags, while both would be set in the capability flags.

Figure 5 lists each of the attribute flags for input and output capabilities, with the flags added to the API in QuickTime 1.5 shown in bold. Complete descriptions of the flags can be found in Chapter 7, "Video Digitizer Components," ofInside Macintosh: QuickTime Components.

One point we'd like to emphasize is that clients of your digitizer will be much happier if you truthfully state what you can and can't do. The sequence grabber, in particular, will function much better. So don't broadcast that you can support hardware play through or hardware DMA unless you can! In addition, when designing your device's feature set, it's better not to make your capabilities modal. For instance, don't build a card that can resize in 32-bit-per-pixel mode but not in 16-bpp mode.

Applications also need to know about your capabilities. An application makes preflight calls, like VDPreflightDestination, to your digitizer to determine whether its request will be honored, denied, or changed. Digitizers are required to support all the preflight calls.

Some examples. To see how the attribute flags are set, let's take a look at three completely different fictional video digitizer implementations. One of the examples includes the support provided by QuickTime 1.5 for digitizer hardware compression -- a nifty feature we'll discuss when we look at the QuickTime 1.5 additions later in this article.

As you consider the examples, pay particular attention to the use of the following structure members and attribute flags, which seem to be among the least understood:

vdigInfo->inputCapabilityFlags[digiInVTR_Broadcast]
vdigInfo->outputCapabilityFlags[digiOutDoesDMA]
vdigInfo->outputCapabilityFlags[digiOutDoesDouble]
vdigInfo->outputCapabilityFlags[digiOutDoesQuad]
vdigInfo->outputCapabilityFlags[digiOutDoesHWPlayThru]
vdigInfo->outputCapabilityFlags[digiOutDoesAsyncGrabs]
vdigInfo->gdh
vdigInfo->maskgdhvdigInfo->blendLevels

The size of the MaxAuxBuffer is also very important if the device supports one. The only way for a client to determine this is to make the VDGetMaxAuxBuffer call to see if it returns valid data. See "What the $#%!! Is a MaxAuxBuffer, and Do I Have One?" for details.

Figure 5 Video Digitizer Attribute Flags

For our first example of a fictional digitizer, let's suppose SuperOps announces an entry-level video digitizing card with the following capabilities: The card has one video input (an RCA-style connector) and can support only the NTSC video standard. The card is a combined video digitizer and frame buffer. The frame buffer can support 1, 2, 4, 8, 16, and 32 bpp on a monitor that's 640 by 480 pixels. The video digitizer, however, can display real-time video on its own frame buffer only in the 16-bpp and 32-bpp modes. Our SuperOps card doesn't support compression, but it does support clipping via an alpha channel. Because it can resize the video but not zoom, scaling only makes the video smaller. The card has additional local memory for grabbing in 16-bpp mode, but not in 32-bpp mode.

The interesting fields and their values for the SuperOps card are:

vdigType               =  vdTypeAlpha;
inputCapabilityFlags   =  digiInDoesNTSC | digiInDoesComposite | 
                          digiInDoesColor;            
outputCapabilityFlags  =  digiOutDoes16 | digiOutDoes32 |     
                          digiOutDoesShrink | digiOutDoesMask |
                          digiOutDoesQuarter | digiOutDoesSixteenth |
                          digiOutDoesBlend | digiOutDoesHWPlayThru |
                          digiOutDoesAsyncGrabs;
gdh                    =  // Handle to frame buffer graphics device
blendLevels            =  video16 ? 2 : (video32 ? 256:0); 

In 32-bpp mode, there's no room for a MaxAuxBuffer, so the call to VDGetMaxAuxBuffer fails. In 16-bpp mode, a MaxAuxBuffer is available because only half of the on-board local frame buffer memory is being used, and the size of the unused memory is equivalent to the display size, 640 by 480 pixels.

The second fictional card, the OneShot-O-Matic, is designed to do only single-frame grabs. The card has three inputs -- composite, S-Video, and component RGB -- and can support both the NTSC and PAL video standards on all three. The maximum size that can be grabbed is a function of which input standard the card uses (for example, the maximum size would be 768 by 576 pixels when decoding the PAL standard). Because the OneShot-O-Matic can grab only to its local memory, it can't do hardware DMA -- that is, it doesn't support play through to any frame buffer. The OneShot-O-Matic supports only 32-bpp grabs. Resizing, clipping, and hardware compression aren't supported. The board is populated with 2 megabytes of memory, which is enough to support the maximum size required by the PAL standard.

For the OneShot-O-Matic card, the interesting fields and their values are:

vdigType                =   vdTypeBasic;
inputCapabilityFlags    =   digiInDoesNTSC | digiInDoesPAL |
                            digiInDoesComposite | digiInDoesSVideo |
                            digiInDoesComponent | digiInDoesColor;
outputCapabilityFlags   =   digiOutDoes32 | digiOutDoesAsyncGrabs;
gdh                     =   nil;
blendLevels             =   0;

In this case, the MaxAuxBuffer will consume the entire 2 MB local buffer area. This works out to be 524,288 available pixels in 32-bpp mode (2097152 / 4 bytes per pixel). The representation of this as a width and height must be large enough to support the largest grab size; thus 800 by 655 pixels would be valid, as would 700 by 748 pixels. Hardware resizing would have allowed multiple buffers to reside in the MaxAuxBuffer. With this card, however, the video digitizer will be relying on the sequence grabber to do resizing, color conversion, and on-screen placement operations. Our third fictional card, the Verne-Motion, is designed to perform not only video digitizing, but also hardware compression using a highly proprietary algorithm called SqueezeMe. (We'll discuss compressed-source devices shortly.) The card has one S-Video input and supports all input standards -- NTSC, PAL, and SECAM. In addition, the Verne-Motion can simultaneously support two channels of video: an RGB pixel stream formatted for display, and a compressed SqueezeMe stream to be saved as a QuickTime movie. The card has a general-purpose DMA engine that can direct these data streams either to memory on the card or to system memory. The on-card memory is 4 MB and gives better performance than you get when you send the video data to system memory. The Verne-Motion supports hardware clipping with an 8-bit mask plane, which can also be stored in local memory or in system memory. The video input circuitry has a mode that distinguishes between clean broadcast-quality input signals and noisier VCR-type signals. Finally, the Verne-Motion lets you arbitrarily scale video images to make them up to two times larger in both the horizontal and vertical directions.

The interesting fields and their values for the Verne-Motion card are:

vdigType              =   vdTypeMask;
inputCapabilityFlags  =   digiInDoesNTSC | digiInDoesPAL |
                          digiInDoesSECAM | digiInDoesSVideo |
                          digiInVTR_Broadcast | digiInDoesColor;
outputCapabilityFlags =   digiOutDoes8 | digiOutDoes16 |
                          digiOutDoes32 | digiOutDoesStretch |
                          digiOutDoesShrink | digiOutDoesMask |
                          digiOutDoesDouble | digiOutDoesQuarter |
                          digiOutDoesSixteenth | digiOutDoesHW_DMA |
                          digiOutDoesHWPlayThru |
                          digiOutDoesAsyncGrabs | digiOutCompress |
                          digiOutPlayThruDuringCompress;
gdh                   =   nil;
maskgdh               =   // Handle to local mask plane — 8 bpp deep
blendLevels           =   256;

The size of the MaxAuxBuffer in this case would be 1024 by 1024 pixels in 32-bpp mode (1024 * 1024 * 4 bytes per pixel equals 4 MB, which is the total available memory). In the 16-bpp mode, the MaxAuxBuffer would be 2048 by 1024 pixels, and in the 8-bpp mode, it would be 2048 by 2048 pixels.

As you can see from these examples, there are many different classes of video digitizers out there. Your card may look very similar to one of the ones we've presented, but it's more likely that you'll have to set different flags to describe your card's features.

THE POWER OF THE SEQUENCE GRABBER

The sequence grabber makes life easier for application programmers by handling all the messy details of controlling video digitizers, sound input devices, and compressors. This marvelous piece of QuickTime has a very rich API that we won't even pretend to cover in this article. However, the sequence grabber plays such an important role in a video digitizing application that a brief introduction is in order. Note that while the code excerpts below demonstrate how the sequence grabber can preview and record a video channel, the HackTV application on the CD includes an audio channel as well.

Before we discuss previewing and recording, we want to briefly describe how the sequence grabber can make up for functionality missing from a digitizer. For the sequence grabber to be able to do this, the video digitizer must be able to use off-screen memory -- either local memory on the digitizing device or, in the more general case, any memory. (Recall that you can use the VDGetMaxAuxBuffer call to locate local device memory.) As an example, let's see how the sequence grabber can support color conversion.

Suppose a video digitizer can display video only to its own integrated frame buffer. In a multimonitor system, this would mean a user could display video on only one of the monitors. The sequence grabber comes to the rescue because it can use off-screen memory for the grab and copy it to thedesired destination buffer, making the video appear on one of the unsupported displays. Now suppose the digitizer is asked to display video in a depth it doesn't support (say the user changes the graphics mode from 32 bpp to 4 bpp with the Monitors control panel). The sequence grabber can again use off-screen memory to do the right thing. Of course, you never get something for nothing. All this buffer copying will adversely affect performance.

PREVIEWING
Previewing video with the sequence grabber is the equivalent of setting up the 'vdig' in play-through mode to display live video on the computer screen. The code to make this happen is shown below. Remember that the sequence grabber can simulate live video play through for devices that don't support hardware play through.

// Find and open a sequence grabber.
gSeqGrabber = OpenDefaultComponent(SeqGrabComponentType, (OSType) 0);
// If we get a sequence grabber, set it up.
if (gSeqGrabber != 0L) {
    // Get the monitor — in this case, the dialog 
    // window in which video is displayed.
    gMonitor = GetNewDialog(kMonitorDLOGID, nil, (WindowPtr) -1L);
    if (gMonitor != nil) {
        // Initialize the sequence grabber.
        GetPort(&savedPort);
        SetPort(gMonitor);
        ShowWindow(gMonitor);
        result = SGInitialize(gSeqGrabber);
        if (result == noErr) {
            result = SGSetGWorld(gSeqGrabber, (CGrafPtr) gMonitor,
                nil);
            // Get a video channel.
            result = SGNewChannel(gSeqGrabber, VideoMediaType, 
                                            &gVideoChannel);
            if ((gVideoChannel != nil) && (result == noErr)) {
                    short   width;
                    short   height;
                    gQuarterSize = false;
                    gHalfSize = true;
                    gFullSize = false;
                    result = SGGetSrcVideoBounds(gVideoChannel,
                        &gActiveVideoRect);
                    width = (gActiveVideoRect.right - 
                        gActiveVideoRect.left)/2;
                    height = (gActiveVideoRect.bottom - 
                        gActiveVideoRect.top)/2;
                    SizeWindow (gMonitor, width, height, false);
                    // Preview the video only.
                    result = SGSetChannelUsage(gVideoChannel,
                        seqGrabPreview);
                    result = SGSetChannelBounds(gVideoChannel,
                        &(gMonitor->portRect));
            }
            // Go!
            if (result == noErr)
                result = SGStartPreview(gSeqGrabber);
        }
        SetPort(savedPort);
    }
}

We establish a connection to the sequence grabber component in the usual way, with OpenDefaultComponent. We initialize the sequence grabber, set up the graphics environment, and allocate a new channel for video. We want the displayed video to be one-quarter size, so we call SGGetSrcVideoBounds to see what size the source video is. This function calls VDGetDigitizerRect, which returns the source video size equal to the digitizer rectangle. We scale the height and width accordingly, and the new size is sent to the sequence grabber via the SGSetChannelBounds routine. Finally, we call SGStartPreview, which turns on the video digitizer by calling the digitizer function VDSetPlayThruOnOff, and previewing begins.

RECORDING
Recording is very similar to previewing and is almost as simple. The following code highlights the differences between recording and previewing.

// Start exactly the same way as for previewing.
// Find and open a sequence grabber.
gSeqGrabber = OpenDefaultComponent(SeqGrabComponentType, (OSType) 0);
// If we get a sequence grabber, set it up.
if (gSeqGrabber != 0L) {
    . . .
        if ((gVideoChannel != nil) && (result == noErr)) {
            // Set up size the same as in previewing case.
            short   width; 
            short   height;
            gQuarterSize = false;
            gHalfSize = true;
            gFullSize = false;
            result = SGGetSrcVideoBounds(gVideoChannel,
                &gActiveVideoRect);
            width = (gActiveVideoRect.right -
                gActiveVideoRect.left)/2;
            height = (gActiveVideoRect.bottom - 
                gActiveVideoRect.top)/2;
            SizeWindow(gMonitor, width, height, false);
            // Record and play images instead of just previewing
            // them.
            result = SGSetChannelUsage(gVideoChannel, seqGrabRecord |
                seqGrabPlayDuringRecord);
            result = SGSetChannelBounds(gVideoChannel, 
                &(gMonitor->portRect));
            // Get ready. . .
            result = SGSetDataOutput(gSeqGrabber,&gMovieFile,
                seqGrabToDisk);
            result = SGPrepare(gSeqGrabber, true, true);
            // Go!
            result = SGStartRecord(gSeqGrabber);
            while (!Button() && !result) {
                result = SGIdle(gSeqGrabber);
            }
            result = SGStop(gSeqGrabber);
        }
    . . .
}

There are several differences between recording and previewing. First we set up the sequence grabber to record and to play through while recording. We next specify a file for the movie to be written to, indicating that the movie be grabbed directly to disk. For a short movie, we could grab to memory if we wanted. By default, the recording time is limited by the system resources available -- in this case, disk space.

The SGStartRecord call initiates the grab to disk. SGIdle is called repetitively to provide processing time to the sequence grabber. You should call SGIdle as often as possible while recording. When the user clicks the mouse button, or when the disk is full, recording will stop, and we call SGStop to complete the recording process.

That's all there is to simple recording. If you want to do more sophisticated tasks with the sequence grabber, such as replacing the standard sequence grabber disk- or compression-bottleneck routines with your own, consultInside Macintosh: QuickTime Components, or refer to the SGSample sample code on the QuickTime 1.0 Developer's CD-ROM.

WHAT'S NEW FOR DIGITIZERS IN QUICKTIME 1.5

QuickTime 1.5 provides expanded support for sophisticated new types of video digitizing hardware. First, it defines a number of routines that allow digitizers to describe their capabilities to clients more completely. It also defines routines that give clients more control over the digitization process. QuickTime 1.5 adds to the video digitizer API by supporting devices that are capable of producing compressed-image data. Finally, a standard user interface has been introduced for choosing and configuring video digitizers.

EXPANDED INFORMATION SERVICES
QuickTime 1.5 encompasses a greater range of digitizer capabilities than did QuickTime 1.0. The latest API defines new flags and several new routines to allow applications to determine whether a digitizer supports these new capabilities.

New digitizer information flags. Four new flags have been defined in the outputCapabilityFlags field of the DigitizerInfo record.

• The digiOutDoesUnreadableScreenBits flag indicates that the video digitizer may put pixels on the screen that are visible but can't be used when compressing images. In other words, image data can't be read directly from the screen as source material for the compression phase.
• The digiOutDoesCompress flag indicates that the digitizer is capable of producing compressed-image data directly.
• The digiOutDoesCompressOnly flag indicates that the digitizer provides compressed-image data only and that it's unable to provide image data for display.
• The digiOutDoesPlayThruDuringCompress flag indicates that the digitizer can display image data while it's compressing it.

New information routines. Four new functions allow applications to get additional information about the digitizer's capabilities.

• The VDGetSoundInputDriver call gives video digitizers a way to tell applications which sound input driver they're associated with. Several digitizer boards now available have sound as well as video digitizing capability.
• The VDGetPreferredTimeScale routine allows digitizers that can time-stamp the data they create to communicate their preferred time scale to applications. The sequence grabber, in particular, uses this call to establish the time scale of the video data it creates from the digitizer output.
• The VDGetDataRate routine is extremely useful because it gives clients a way to determine the performance capabilities of a digitizer. The call returns three values. The first value, milliSecPerFrame, indicates the number of milliseconds of overhead involved in digitizing a single frame. Overheadis defined as the average delay between the time the digitizer requests a frame from its associated hardware and the time the device actually delivers the frame. The second value, framesPerSecond, is the maximum rate at which the digitizer can capture video frames in its current configuration. This value is not affected by the VDSetFrameRate call, described later on. The last value, bytesPerSecond, indicates the data rate at which a compressed-source digitizer can produce compressed-image data. This value varies depending on parameters for spatial and motion quality, image size, and depth. In other words, unlike milliSecPerFrame and framesPerSecond, bytesPerSecondisn'ta static value.
• The VDGetDMADepths routine allows an application to determine the pixel depths a DMA-style digitizer can support. The depthArray parameter is a pointer to a long word of flags, with each flag representing a possible depth. A flag with a value of 1 indicates that the corresponding depth is supported. The preferredDepth parameter is a pointer to a long word indicating -- big surprise -- the preferred depth of the digitizer. A value of 0 indicates that all depths are equally acceptable. Note that if a DMA-style digitizer doesn't support this call, the digitizer is assumed to be capable of handling off-screen buffers at all depths indicated in the outputCapabilityFlags field of its DigitizerInfo record.

ENHANCED DIGITIZATION CONTROL
Two new routines in QuickTime 1.5 give applications greater control over how a video digitizer performs its task. Video digitizers that can time-stamp the video frames they produce should implement the new VDSetTimeBase routine so that applications can specify the time coordinate system the video digitizer should use when time-stamping video frames.

The second routine, VDSetFrameRate, allows applications to tell a digitizer the precise frame rate to use for capture. Digitizers used to capture video at only one frame rate -- as fast as possible. However, the advent of full-frame-rate digitizing hardware and compressed-source devices has made it increasingly important for clients to manage the tradeoff between frame rate, image size, and compression quality. The rate in VDSetFrameRate is expressed as a fixed-point value, typically between 0 and 29.97 (see "Frame Rates and Motion Quality" earlier in this article for a discussion of frame rate values).

COMPRESSED-SOURCE DEVICE SUPPORT
OK, pay attention here -- this enhancement alone is worth the price of admission for QuickTime 1.5. QuickTime is finally poised to silence all complaints about "postage stamp video sizes" and "jerky frame rates." By expanding the video digitizer API to support compressed-source video digitizers, QuickTime gives users access to full-size, full-frame-rate digital video. A number of new video capture boards with on-board compression and decompression capabilities are already shipping, and there will undoubtedly be others soon.

QuickTime 1.5 includes eight new routines for servicing compressed-source video digitizers. Several of these calls should look familiar to you, since they're largely compressed-source versions of existing calls and serve very similar purposes.

• VDGetCompressionTypes is a straightforward call. It simply returns a handle to the list of compressors that your video digitizer implements. Each element of the list contains the component ID, type, name, format, and capabilities of a compressor.
• The VDSetCompression routine specifies which of all the possible compressors a digitizer should use. The parameters for VDSetCompression specify the spatial quality, temporal quality, depth, and other characteristics of the compression.
• VDCompressOneFrameAsync starts the digitizing process for compressed-source devices, just as VDGrabOneFrameAsync starts the process for regular digitizers. The major difference is that a compressed-source digitizer handles all the management of data buffers itself, without external assistance from the caller.
• VDCompressDone is similar to VDDone in that it allows a caller to determine when a frame has been completed (in this case, digitized and compressed).
• The VDReleaseCompressBuffer tells a compressed-source device to free the buffer returned by the VDCompressDone call.
• The VDResetCompressSequence call instructs a digitizer to insert a key frame into a frame-differenced image sequence as soon as possible after it receives this call.
• The VDGetImageDescription routine prompts the digitizer to return an image description structure corresponding to the current settings. This structure is defined in the Image Compression Manager chapter ofInside Macintosh: QuickTime and is the same structure that's used to describe image data in movie files.
• The VDSetCompressionOnOff routine starts and stops the digitizer. To give the digitizer adequate time to prepare itself for the requested operation, clients must call this routine before calling VDSetCompression or VDCompressOneFrameAsync.

Typically, compressed-source devices are able to act as hardware decompressors and have a corresponding QuickTime image decompressor component that clients can use to play back the compressed images. However, if your hardware produces compressed data that can't be read by any of the standard QuickTime image decompressor components, you need to provide an appropriate software-only decompressor component. This way, users who don't have your hardware will still be able to play movies produced with your compressor.

A STANDARD USER INTERFACE FOR CONFIGURING VIDEO DIGITIZERS
QuickTime 1.5 introduces changes to the sequence grabber component, which many developers rely on to reduce the complexity of dealing with video and sound digitizers. One of the more significant additions to the sequence grabber component is standard dialog boxes for configuring video and sound digitizers. Both Apple and third parties can extend the controls presented in these dialog boxes through sequence grabberpanel components. Although this article won't delve into the details of writing a panel component, you should be aware of them and how they relate to video digitizers.

Figure 6 shows the Source panel, one of the three panel components built into QuickTime 1.5. The other two panel components are the Image panel and the Compression panel. You navigate to the different panels through the pop-up menu at the top of the panel area.

Figure 6 The Source Panel

If your video digitizer has capabilities that aren't addressed by the standard panels in QuickTime 1.5, and it's important to give users access to these features, we strongly urge you to write a panel component. Any applications that use the sequence grabber dialog boxes can then pick up the functionality in your panel component. There's less work for everybody this way -- video digitizer manufacturers don't have to write applications that show off their card-specific features, and application developers don't have to write card-specific code.

You'll find a description of how to write your own sequence grabber panel components in Chapter 6 of Inside Macintosh: QuickTime Components. As a bonus, we've included the source code for an example panel component on this issue's CD.

THE IDEAL VIDEO DIGITIZER

To bring a video digitizer board to market, developers must make numerous compromises between design, features, and cost. It's not unusual for a few hasty design decisions to lead to a less-than- wonderful QuickTime digitizer product. For example, if the SuperOps company creates a video digitizing card that can do video play through at only 24 bpp in a display size of 640 by 480 pixels, and that can capture at only 8 bpp in a display size of 240 by 180 pixels, the card probably isn't going to be a prime candidate for the next QuickTime product of the year. Why? Because the video digitizer API is designed to encourage the creation of devices that have more or less symmetrical operating characteristics in both the play- through and capture modes. Obviously, the card just described doesn't meet this goal -- it behaves quite differently depending on whether it's performing play through or capture. In addition, the choice of 240 by 180 pixels as the board's capture size is unwise. In general, an ActiveSrcRect size that's evenly divisible by 2 produces the best quality without the support of sophisticated filtering (see "What's So Magic About Magic Sizes?").

Table 1 Characteristics of an Ideal Video Digitizer

Feature	Benefit
Hardware DMA support	The ability to display video on any screen (or off-screen) and not be captive to a local frame buffer
Enhanced resizing algorithms	Improved image quality
(anti-aliasing, improved
line filtering, and so on)
Arbitrary resizing in hardware	Fast resizing to any size
16-bit pixel support	Reduced data rate with minimal loss of color fidelity
Enhanced color control	Finer control over compression efficiency and improved image quality
Hardware compression and decompression	Improved live-capture frame rates, movie playback rates, and bigger movie sizes
Arbitrary hardware zoom (stretch, shrink)	Better cropping
Multiple buffering	Better performance
Signal lock sensing and standard detection	Improved user experience
Symmetrical play-through and	Improved user feedback
record characteristics
Hardware special effects (flips, warps, skews)	Fun stuff to create that jaw-dropping impact on users
Key color	Fast masking, blue screen effects
Hardware clip mask	Video window clipping in graphics environments
Simultaneous compression and decompression	Video conferencing and video phone of multiple video channels applications

So how does a developer decide what sorts of features to include in a digitizer board? Well, we've got some pretty solid ideas about what characteristics make for a really great digitizer. To get a sense of the kind of features we believe an ideal QuickTime video digitizer should have, take a look at Table 1. Keep in mind that each of the features of this ideal digitizer has a very real associated cost, which gets reflected in the retail price. Our ideal digitizer would probably be priced out of the reach of most users today. But then again, two years ago we wouldn't have believed we could afford personal computers with 32 MB of RAM, so who knows?

WHAT'S SO MAGIC ABOUT MAGIC SIZES?
When the vertical dimension of the ActiveSrcRect can be evenly divided by 2, the resulting sizes are referred to as magic sizes. Actually, there's nothing magic about them. They just happen to be easily produced by some simple tricks in hardware and software. It takes two fields to produce a complete frame of full-size NTSC video (for example, 640 by 480 pixels), with each field holding half the lines of the video frame. The most common resizing trick is to drop a field from each video frame to produce the requisite number of lines for a half-size image (for example, 320 by 240 pixels). This makes it very simple to produce good-quality half-size video without performing more sophisticated image filtering in software. Because software isn't burdened with the additional task of performing a more complex filtering of the image, the effective capture rate of the digitizer is increased at this "magic" size.

GO GRAB A MOVIE
Wow, we've covered a lot of ground here! Start thinking about integrating video data types into your applications, and experiment with the sample code to get started. The main message we want to pass on to application writers is that you should use the sequence grabber as your entry point into the world of video. Taking advantage of the enormous capability of the sequence grabber leaves you more time to spend on creating features that differentiate your product. As for video digitizer developers, we hope this article has given you more insight into the grab process and gotten you ready to start implementing some of the cool features listed in Table 1. We're counting on seeing many of these capabilities in the next generation of digitizer boards. Finally, to the video neophyte, all we can say is that there aren't too many things that are more entertaining than going out and grabbing a movie. So go do it!

REFERENCES

• Inside Macintosh: QuickTime and Inside Macintosh: QuickTime Components. These are included in the QuickTime Developer's Kit v. 1.5.
• "Inside QuickTime and Component-Based Managers" by Bill Guschwan, develop Issue 13.
• "Techniques for Writing and Debugging Components" by Gary Woodcock and Casey King, developIssue 12.
• "Time Bases: The Heartbeat of QuickTime" by Guillermo A. Ortiz, develop Issue 12.
• Electronic Cinematography: Achieving Photographic Control over the Video Image by Harry Mathias and Richard Patterson (Wadsworth, 1985).
• Film Art: An Introduction, 3rd ed., by David Bordwell and Kristin Thompson (McGraw-Hill, 1990).
• Graphics Gems Volume I edited by Andrew S. Glassner (Academic Press, 1990), Chapter 3, "Useful 1-to- 1 Pixel Transforms."
• Raster Graphics Handbook, 2nd ed., Conrac Corporation (Van Nostrand Reinhold, 1985).
• Television Production, 3rd ed., by Alan Wurtzel and Stephen R. Acker (McGraw-Hill, 1989).

FRAME RATES AND MOTION QUALITY
The term frame rate is frequently tossed about in discussions of the pros and cons of video digitizers. Frame rate is the rate at which frames appear during video playback.

It's commonly held that the frame rate corresponding to full-motion video is 30 frames per second (fps). However, 30 fps is not the only interesting frame rate or even the only "true" full-motion frame rate.

• 30 fps is the normal frame rate for NTSC video and broadcast production. The precise rate is actually 29.97 fps. When a QuickTime product promises "full-motion" video, this is the rate that's typically implied.
• 25 fps is the normal frame rate for PAL and SECAM video and broadcast production.
• 24 fps is the normal frame rate for theatrical film production.
• 10 to 12 fps is widely regarded as the minimum acceptable frame rate for a QuickTime movie. At rates below this threshold, the motion is generally perceived to be too jerky. In addition to frame rate, there are two other terms you should know about. If the frame rate measures the speed at which the movie is played back for the viewer, the capture rateis the rate at which the 'vdig' hardware is capable of capturing frames. The effective capture rate is the number of frames per second that end up in a QuickTime movie. Many factors can make the effective capture rate less than the intrinsic rate the hardware can support. We'll discuss these factors later in the article.

WHAT THE $#%!! IS A MAXAUXBUFFER, AND DO I HAVE ONE?VDGetMaxAuxBuffer is an extremely misunderstood 'vdig' call. For a video digitizer that implements this call, the sequence grabber can provide improved capture rates and enhanced capabilities, even if the digitizer hardware wasn't originally designed with off-screen buffering in mind.

Simply put, a MaxAuxBuffer is the total unused off-screen local memory on a digitizer card. The sequence grabber can use the MaxAuxBuffer to implement resizing, clipping, and color conversion operations for digitizers that don't support these operations in hardware. The MaxAuxBuffer can also be used to provide a low-performance equivalent to DMA for devices that don't support hardware DMA. Alternatively, the sequence grabber can use the MaxAuxBuffer for multiple buffering by partitioning it into one or more smaller buffers.

Even if your card supports high-performance DMA-to-system memory operations, we suggest that video digitizers support and implement this feature whenever possible. Transferring data into your local memory system will be faster than transferring it across NuBus^TM into system memory. This, in turn, makes it possible for clients to achieve a faster effective capture rate.

A couple of rules of thumb for using VDGetMaxAuxBuffer: First, because a MaxAuxBuffer is composed of local memory only from your device, you should not allocate system memory to support this call. Second, the sequence grabber will sometimes act like a snapperhead and make this call multiple times. Just be consistent with your replies. You aren't responsible for managing or partitioning this memory -- you simply have to say it's around.

CASEY KING AND GARY WOODCOCK are considered fictitious by Apple Computer, Inc. (at least, they're never around their offices when anybody's looking for them). Any similarity to actual persons, living, dead, or somewhere in between, is unintentional, unlikely, and if true, probably unfortunate. All persons appearing in this article are over 18 years of age (physically, anyway). This article was written entirely on location in Austin, TX, and Cupertino, CA, usually between the hours of 10 P.M. and 4 A.M.*

Casey King and Gary Woodcock are trademarks of Apple Computer, Inc.

Great QuickTime movies are tough to produce due to the number of tradeoffs that need to be considered. See the article "Making Better QuickTime Movies" in this issue for more information on managing frame rate, frame size, compression, image quality, and sound to achieve the best results. *A single full-sized video frame consists of two fields, one containing the odd-numbered scan lines and the other containing the even-numbered scan lines.*

For an example of how to set up destination characteristics, see the description of VDSetPlayThruDestination in Inside Macintosh: QuickTime Components.*

VDSetPlayThruDestination is one of the required calls defined in the video digitizer component API. For a complete list of all the calls that a digitizer component must implement, see the section "Required Functions" in Chapter 7 of Inside Macintosh: QuickTime Components.*For more information on time scales, see Inside Macintosh: QuickTime Components and "Time Bases: The Heartbeat of QuickTime" in develop Issue 12.*

THANKS TO OUR TECHNICAL REVIEWERS Bill Guschwan, Peter Hoddie, Guillermo Ortiz, John Wang *