TweetFollow Us on Twitter

Oct 01 QT Toolkit

Volume Number: 17 (2001)
Issue Number: 10
Column Tag: QuickTime Toolkit

F/X 2

by Tim Monroe

Using Video Effects with Movie Segments, Images, and Sprites


In the previous QuickTime Toolkit article ("F/X" in MacTech, September 2001), we investigated a few of the most basic ways to use the QuickTime video effects architecture, which allows us to apply video effects to tracks in movies and to images. We saw how to work with generators (zero-source effects) and how to apply a filter to a video track and a transition to a pair of video tracks. We also saw how to specify effects parameters and use the effects parameters dialog box to elicit an effect and some effects parameters from the user.

In this article, we're going to continue working with the QuickTime video effects architecture. We'll see how to apply an effect to part of a movie and how to use an effect as the image for a sprite. We're also going to see how to apply an effect to an image (that is, not to a track in a movie). This will lead us, for the first time, to work directly with image decompressors (since, as we saw last time, effects are rendered by image decompressor components). In fact, the techniques we learn for decompressing image sequences will be useful in the future (perhaps even in our very next article).

Our sample application once again is QTEffects (the same one as in the previous article); its Test menu is shown in Figure 1.

Figure 1: The Test menu of QTEffects.

In this article, we'll see how to handle the fourth menu item and the final two. Let's begin by seeing how to add an effect to a movie segment.

Video Effects and Movie Segments

We saw in the previous article that it's fairly simple to add a video effect to an entire track. We just add an effects track that has the same track offset and duration as the source track, and we link the effects track to the source track by creating track references from the effects track to the source track and by setting the input map of the effects track appropriately. The media handler for the source track feeds all of its decompressed frames to the component specified in the effects track, which processes those frames further.

To apply a video effect to only part of a source track requires a bit more work. As we saw briefly last time, we can do this by creating a copy of the track segment that we want to apply the effect to; the effect then uses the track segment copy its source, as shown in Figure 2.

Figure 2: A filter applied to part of a video track

When we want to add a two-source effect to part of a movie, the ideas are fundamentally the same. Suppose we've got a movie with two video tracks that overlap for some part of the movie (as shown in Figure 3).

Figure 3: Two overlapping video tracks

We want to apply a transition during the time the two tracks overlap; to do this, we can make copies of the appropriate track segments and use them as sources for the effects track, as seen in Figure 4.

Figure 4: A transition applied to parts of two video tracks

Given what we learned in the previous article, all we really need to do now is learn how to create a new track that holds only part of the data of an existing track. But in fact we already know how to do that. When we were discussing data references ("Somewhere I'll Find You" in MacTech, October 2000), we saw how to use the InsertTrackSegment function to copy media data from one track to another. We can use that function here to create the video track segment copy, as shown in Listing 1. Notice that we also call CopyTrackSettings to copy the source track matrix, clipping region, graphics mode, and other properties into the destination track.

Listing 1: Creating a copy of a video track segment

mySrcTrack1 = NewMovieTrack(theMovie, myWidth, myHeight, 
if (mySrcTrack1 == NULL)

mySrcMedia1 = NewTrackMedia(mySrcTrack1, VideoMediaType, 
            myTimeScale, NULL, 0);
if (mySrcMedia1 == NULL)

myErr = BeginMediaEdits(mySrcMedia1);
if (myErr != noErr)
myErr = CopyTrackSettings(myVidTrack1, mySrcTrack1);
myErr = InsertTrackSegment(myVidTrack1, mySrcTrack1, 
            theStartTime, theDuration, theStartTime);
if (myErr != noErr)


The value of the compiler flag COPY_MOVIE_MEDIA determines whether the new track segment contains a copy of the media data in the original video track or the new track segment contains only references to that media data. In QTEffects, we set the value of that flag to 0, to minimize the resulting file size.

To make sure that the original video track is hidden behind the new effects track for the duration of the effect, we need to set the track layer of the effects track to be lower than the track layer of the video track. Toward the beginning of QTEffects_AddEffectToMovieSegment, we call the EffectsUtils_GetFrontmostTrackLayer function to retrieve the lowest layer of any video track in the movie, like this:

myLayer = EffectsUtils_GetFrontmostTrackLayer(theMovie, 

Once we've created the effects track, we then set its layer like this:

SetTrackLayer(myEffectTrack, myLayer - 1);

Listing 2 shows our definition of EffectsUtils_GetFrontmostTrackLayer.

Listing 2: Finding the lowest layer of a track of a certain kind

short EffectsUtils_GetFrontmostTrackLayer (Movie theMovie, 
            OSType theTrackType)
   short      myLayer = 0;
   short      myIndex = 1;
   Track      myTrack = NULL;

   // get the layer number of the first track of the specified kind;
   // if no track of that kind exists in the movie, return 0
   myTrack = GetMovieIndTrackType(theMovie, 1, theTrackType, 
            movieTrackMediaType | movieTrackEnabledOnly);
   if (myTrack == NULL)

   myLayer = GetTrackLayer(myTrack);

   // see if any of the remaining tracks have lower layer numbers
   while (myTrack != NULL) {
      if (myLayer > GetTrackLayer(myTrack))
         myLayer = GetTrackLayer(myTrack);
      myTrack = GetMovieIndTrackType(theMovie, myIndex, 
            theTrackType, movieTrackMediaType | 


See the file QTEffects.c for the complete definition of the QTEffects_AddEffectToMovieSegment function, which is called in response to the "Add Effect to Movie Segment" menu item.

Video Effects and Images

Up to now, we've considered QuickTime video effects only as applied to movies. It's also possible to apply effects to still images. For instance, Figure 5 shows a still image that's had the emboss effect applied to it.

Figure 5: An image with the emboss effect

Figure 6 shows the same image, with the x-ray version of the color tint effect.

Figure 6: An image with the x-ray color tint effect

In this section, we'll see how to apply a filter to an image. We won't actually learn how to apply a transition to a pair of images, but we'll write our code in such a way that it will be easy for the motivated reader to extend it to do so. See the end of this article for pointers to code that does in fact implement transitions between pairs of images.

Decompressing Images

When we apply an effect to a track in a movie, the data describing the effect is stored in the movie itself (in an effects track, of course) and the effect is rendered automatically by QuickTime when the movie is played. Our job, as we've seen, is simply to create the effects track and link it to its source tracks. QuickTime takes care of the nitty-gritty details of retrieving the effect description, interpreting the effects track input map, and applying the effect to the source tracks.

When we want to apply an effect to an image, however, we're more or less on our own. Our application is going to have to keep track of the relevant effects data (that is, the effect description and the image description, along with the source data) and render the effect itself. Since QuickTime effects are implemented as image decompressor components, we need to open an image decompressor and apply it to the source data (the original image). We've previously worked with image compressors, to compress single images and sequences of images. (See "Honey, I Shrunk the Kids" in MacTech, February 2001.) Now it's time to tackle the other end of the compression/decompression process.

Let's begin by learning how to decompress a single image. Remember that we can compress an image by calling the Image Compression Manager (ICM) functions GetMaxCompressionSize and CompressImage. GetMaxCompressionSize tells us the maximum size of the buffer we'll need to hold a compressed image, and CompressImage actually compresses the image. The source data is stored as a pixel map, and the compressed data is written into a buffer. To decompress an image, we can call DecompressImage, which takes a buffer of data and expands it into a pixel map. DecompressImage is declared essentially like this:

OSErr DecompressImage (Ptr data, ImageDescriptionHandle desc,
            PixMapHandle dst, const Rect *srcRect, 
            const Rect *dstRect short mode, RgnHandle mask);

The data parameter points to the compressed data that we want to decompress, and the dst parameter is a handle to a pixel map into which the data will be decompressed. The desc parameter is a handle to an image description, which specifies (among other things) the format of the compressed data and the bounds of the image. The srcRect parameter specifies which part of the image rectangle we want to decompress. This rectangle must lie within the rectangle whose upper-left corner is (0,0) and whose lower-right corner is ((**desc).width, (**desc).height). To specify the entire source rectangle, we can pass the value NULL for the srcRect parameter.

The dstRect parameter specifies the rectangle into which the image is to be decompressed. Typically we'll decompress into the entire destination pixel map, so we would pass the value (**dst).bounds. The mode parameter indicates the desired transfer mode, which is often srcCopy. Finally, the mask parameter is a handle to a region that specifies a drawing mask (or clipping region) for the destination pixel map; only pixels that lie within the mask are drawn into the destination pixel map. To draw into the entire pixel map, set this parameter to NULL.

Here's a typical call to DecompressImage:

myErr = DecompressImage(myData, myDesc, myPixMap, NULL, 
            (**myPixMap).bounds, srcCopy, NULL);

If this call completes successfully, then we could use myPixMap anywhere we'd use a pixel map; for instance, we could copy it into a window by calling the CopyBits function.

For greater control of decompression operations, we can use the FDecompressImage function. FDecompressImage takes all the parameters of the DecompressImage function, plus a handful of additional parameters that allow us to translate or scale the image during decompression, select a particular image quality, specify a progress function that displays a progress dialog box during lengthy decompressions, and so forth.

It turns out, however, that neither DecompressImage nor FDecompressImage allows us to handle QuickTime video effects. Their main limitation is that they provide no easy way to specify an effect's source or sources. To do that, we need to use ICM functions that decompress an image sequence.

Decompressing Image Sequences

In a previous article ("Honey, I Shrunk the Kids", cited earlier), we compressed a sequence of images using the three standard image compression dialog component functions SCCompressSequenceBegin, SCCompressSequenceFrame, and SCCompressSequenceEnd. The ICM also provides the more general functions CompressSequenceBegin, CompressSequenceFrame, and CDSequenceEnd for initiating and managing a compression sequence. To decompress a sequence of images, we'll use DecompressSequenceBeginS, DecompressSequenceFrameWhen, and CDSequenceEnd. (Notice that CDSequenceEnd can be used to end both an image compression sequence and an image decompression sequence.)

To begin a decompression sequence, we call DecompressSequenceBeginS, which is declared essentially like this:

OSErr DecompressSequenceBeginS (ImageSequence *seqID,
            ImageDescriptionHandle desc, Ptr data, long dataSize,
            CGrafPtr port, GDHandle gdh, const Rect *srcRect,
            MatrixRecordPtr matrix, short mode, RgnHandle mask,
            CodecFlags flags, CodecQ accuracy,
            DecompressorComponent codec);

Some of the parameters here are identical to the parameters of DecompressImage. As with DecompressImage, we pass in a buffer of data, a source rectangle, a transfer mode, and a drawing mask. As with FDecompressImage, we pass in a transformation matrix and a quality setting (in the accuracy parameter). The port and gdh parameters specify the graphics port and graphics device into which the decompressed data will be written. (We shall decompress our data into an offscreen graphics world, in which case we can set gdh to NULL.) The codec parameter specifies the image decompressor component that we want to be used for the decompression sequence; since the image description already indicates the relevant codec, we'll pass NULL in this parameter. Finally, the flags parameter is used to specify any special memory-allocation requirements for the decompressor component; we'll pass 0 to indicate no special requirements here.

DecompressSequenceBeginS uses the image data passed in the data parameter and the other information to preflight the decompression sequence. An instance of the specified decompressor component is opened and initialized, and any additional buffers are allocated. If DecompressSequenceBeginS completes successfully, it returns in the seqID parameter a sequence identifier, which we'll use in subsequent calls to manage the decompression sequence. A sequence identifier is of type ImageSequence, which is declared like this:

typedef long            ImageSequence;

Once we've set up a decompression sequence, we can decompress individual frames of the image sequence by calling DecompressSequenceFrameWhen. We pass in the sequence identifier, the data to be decompressed, and some information about the frame's time location in the sequence. For most filters (one-source effects), the notion of time is not really relevant. But for transitions, the effects components do need to know where in the complete image sequence a particular frame lies. So we need to attach some timing information to the image sequence. We'll do this by creating a time base.

Storing the Decompression Data

Let's see how we can tie this all into our sample application, QTEffects. As you know, the shell application upon which we've built QTEffects is able to open image files in a window, using a graphics importer to draw the image whenever necessary — namely, whenever the image window receives an update event (on Macintosh) or a WM_PAINT message (on Windows). By default, the graphics importer is configured (by a call to GraphicsImportSetGWorld) to draw directly into the image window. To support adding a filter to an image, we need to set the graphics importer to draw into some other location (an offscreen graphics world), which we then use as the source for the effect. When we call DecompressSequenceBeginS, we'll set the image window as the drawing destination. So the original image is first drawn into an offscreen graphics world and then "decompressed" (using an effects component) into the onscreen image window.

Each image window opened by QTEffects therefore needs to have some additional data associated with it. As usual, we store such additional window-specific data in an application data record, a handle to which is stored in the fAppData field of the window data record. Here's how we'll declare the ApplicationDataRecord structure for QTEffects:

typedef struct ApplicationDataRecord {
   OSType                           fEffectType;
   ImageDescriptionHandle   fSampleDescription;
   ImageSequence               fEffectSequenceID;
   QTAtomContainer               fEffectDescription;
   TimeBase                        fTimeBase;
   GWorldPtr                     fGW;
   ImageDescriptionHandle   fGWDesc;
} ApplicationDataRecord, *ApplicationDataPtr, 

The fEffectType field specifies the type of filter we want to apply to the image; in QTEffects, this is always kFilmNoiseImageFilterType. The fSampleDescription field is a sample description for the effect, and the fEffectDescription field is an effect description for the effect. The fEffectSequenceID field holds the sequence identifier returned by DecompressSequenceBeginS. The fGW field holds a pointer to the offscreen graphics world that serves as the effect source, and the fGWDesc field is a handle to a second image description, which describes the image in the offscreen graphics world. Finally, the fTimeBase field specifies the time base that we'll use for timing information. Once again, this field is largely nugatory for filters, but I've included it to make it easier to extend this code to support transitions.

When we open a new image window, we'll execute this line of code to create the application data record:

(**theWindowObject).fAppData = 

The QTEffects_InitWindowData function is defined in Listing 3.

Listing 3: Initializing the data for an image window

Handle QTEffects_InitWindowData 
            (WindowObject theWindowObject)
   ApplicationDataHdl         myAppData = NULL;

   // if we already have some window data, dump it
   myAppData = (ApplicationDataHdl)
   if (myAppData != NULL)

   // allocate and initialize our application data
   myAppData = (ApplicationDataHdl)


As you can see, we clear out any existing data attached to the window (by calling QTEffects_DumpWindowData, discussed later) and then call NewHandleClear to allocate a new block of memory to hold an application data record. When the user selects the "Add Film Noise To Image" menu item, we call the QTEffects_AddFilmNoiseToImage function, defined in Listing 4.

Listing 4: Filling in the application data record

void QTEffects_AddFilmNoiseToImage 
            (WindowObject theWindowObject)
   ApplicationDataHdl            myAppData = NULL;
   GraphicsImportComponent      myImporter = NULL;
   Rect                                 myRect;

   if (theWindowObject == NULL)

   myAppData = 
   if (myAppData == NULL)

   myImporter = (**theWindowObject).fGraphicsImporter;
   if (myImporter == NULL)

   GraphicsImportGetBoundsRect(myImporter, &myRect);

   // set up the initial state
   (**myAppData).fSampleDescription = 
            myRect.right - myRect.left,
            myRect.bottom -;
   (**myAppData).fEffectDescription = 
            (kImageEffectType, kSourceOneName, kSourceNoneName, 
   (**myAppData).fEffectType               = kImageEffectType;
   (**myAppData).fEffectSequenceID      = 0L;
   (**myAppData).fTimeBase                  = NULL;


QTEffects_AddFilmNoiseToImage creates the sample description and the effect description, using utility functions defined in the file EffectsUtilities.c. It also sets the effect type to the film noise effect, using the constant kImageEffectType (which is defined as kFilmNoiseImageFilterType in the file QTEffects.h). Finally, QTEffects_AddFilmNoiseToImage calls the function QTEffects_SetUpEffectSequence to complete the effects set-up process.

Setting Up the Effect

So far we've managed only to allocate the storage we need to maintain the information about our decompression sequence and to create the sample description and the effect description for the film noise effect. We can go ahead and call DecompressSequenceBeginS, like this:

myErr = DecompressSequenceBeginS(
               NULL, NULL, NULL, ditherCopy, NULL, 0,
               codecNormalQuality, NULL);

The first parameter is the location in which a sequence identifier will be returned to us. The next two parameters specify the sample description and effect description, which we created earlier in the QTEffects_AddFilmNoiseToImage function. Notice that the effect description is the buffer of data that is "decompressed" to render the effect. That's right: the effects component takes as its input data the effect description. The image data to which the effect is applied is specified as the source of the effect. (We'll see how to do that in a moment.) We tell the effects component to draw the rendered effect into the onscreen image window using this expression:


Now we need to allocate the offscreen graphics world into which the graphics importer will draw the image and from which the effects component will take its source data. We can do that like this:

GraphicsImportGetBoundsRect(myImporter, &myRect);

// allocate a new GWorld
myErr = QTNewGWorld(&(**myAppData).fGW, 32, &myRect, NULL, 
            NULL, kICMTempThenAppMemory);

The kICMTempThenAppMemory flag tells QuickTime to try to allocate the offscreen graphics world from any available memory that's not assigned to any running process; if there isn't enough of that memory, QuickTime allocates the graphics world from the application's heap.

Once we've successfully allocated the offscreen graphics world, we want to draw the original image into it. We can accomplish this with two easy graphics importer calls:

GraphicsImportSetGWorld(myImporter, (**myAppData).fGW, NULL);

Next, we need to set the image in this offscreen graphics world to be the source data for the image sequence. We'll use the CDSequenceNewDataSource function to create a new data source and the CDSequenceSetSourceData function to install that source as the image sequence source. CDSequenceNewDataSource takes the sequence identifier and an image description for the source and returns a value of type ImageSequenceDataSource:

myErr = CDSequenceNewDataSource
            ((**myAppData).fEffectSequenceID, &mySrc, 
            kSourceOneName, 1, (Handle)(**myAppData).fGWDesc, 
            NULL, 0);

We can create the image description contained in (**myAppData).fGWDesc by calling the MakeImageDescriptionForPixMap function. Once we've got the new source identifier, we can call CDSequenceSetSourceData:

CDSequenceSetSourceData(mySrc, GetPixBaseAddr(mySrcPixMap), 

We're almost done setting up the decompression sequence. All that remains is to create a time base and attach it to the decompression sequence.

(**myAppData).fTimeBase = NewTimeBase();

SetTimeBaseRate((**myAppData).fTimeBase, 0);
myErr = CDSequenceSetTimeBase

Notice that we set the time base rate to 0, since the effect is going to be run outside of a QuickTime movie. We can't count on QuickTime to run the effect for us, so we're going to have to call DecompressSequenceFrameWhen ourselves. Before we get to that, however, let's take a look at the complete definition of QTEffects_SetUpEffectSequence (Listing 5).

Listing 5: Setting up the effect decompression sequence

static OSErr QTEffects_SetUpEffectSequence 
            (WindowObject theWindowObject)
   ApplicationDataHdl            myAppData = NULL;
   ImageSequenceDataSource      mySrc = 0;
   PixMapHandle                     mySrcPixMap = NULL;
   GraphicsImportComponent      myImporter = NULL;
   Rect                                 myRect;
   OSErr                              myErr = paramErr;

   myAppData = (ApplicationDataHdl)
   if (myAppData == NULL)
      goto bail;

   // if an effect sequence is already set up, end it
   if ((**myAppData).fEffectSequenceID != 0L) {
      (**myAppData).fEffectSequenceID = 0L;

   // if there is a timebase already set up, dispose of it
   if ((**myAppData).fTimeBase != NULL) {
      (**myAppData).fTimeBase = NULL;

   // make an effects sequence

   // prepare the decompression sequence for playback
   myErr = DecompressSequenceBeginS(

   if (myErr != noErr)
      goto bail;

   // create the offscreen GWorld holding the original image data
   myImporter = (**theWindowObject).fGraphicsImporter;
   if (myImporter == NULL)
      goto bail;

   // set the size of the GWorld
   GraphicsImportGetBoundsRect(myImporter, &myRect);


   // allocate a new GWorld
   myErr = QTNewGWorld(&(**myAppData).fGW, 32, &myRect, NULL, 
            NULL, kICMTempThenAppMemory);
   if (myErr != noErr)
      goto bail;

   // lock the pixmap

   GraphicsImportSetGWorld(myImporter, (**myAppData).fGW, 

   // get the pixel maps for the GWorlds
   mySrcPixMap = GetGWorldPixMap((**myAppData).fGW);
   if (mySrcPixMap == NULL)
      goto bail;

   // make the effect source
   if ((**myAppData).fGW == NULL)
      goto bail;

   myErr = MakeImageDescriptionForPixMap(mySrcPixMap, 
   if (myErr != noErr)
      goto bail;

   myErr = CDSequenceNewDataSource
            ((**myAppData).fEffectSequenceID, &mySrc, 
            kSourceOneName, 1, (Handle)(**myAppData).fGWDesc, 
            NULL, 0);
   if (myErr != noErr)
      goto bail;

   CDSequenceSetSourceData(mySrc, GetPixBaseAddr(mySrcPixMap), 

   // create a new time base and associate it with the decompression sequence
   (**myAppData).fTimeBase = NewTimeBase();
   myErr = GetMoviesError();
   if (myErr != noErr)
      goto bail;

   SetTimeBaseRate((**myAppData).fTimeBase, 0);
   myErr = CDSequenceSetTimeBase



Running the Effect

The essential step that remains is to call DecompressSequenceFrameWhen to draw the image, with the film noise effect, into the onscreen image window. For most filters, applying an effect is a one-shot deal. That is to say, we really need to call DecompressSequenceFrameWhen only once to get the full visual effect (ignoring of course any redrawing that is required to handle update events and paint messages). But the film noise effect is an oddball here, since the hairs and scratches applied to the image change over time. The film noise effect isn't a transition, but it is sensitive to the passage of time. So we want to call DecompressSequenceFrameWhen repeatedly. Our standard way to do that is to add some code to the QTApp_Idle function; Listing 6 shows the lines we'll add to our idle-time handler.

Listing 6: Tasking the effect decompression sequence

if ((**myWindowObject).fGraphicsImporter != NULL) {
   ApplicationDataHdl         myAppData;

   myAppData = 
   if (myAppData != NULL)
      if ((**myAppData).fEffectSequenceID != 0L)
         QTEffects_RunEffect(myWindowObject, 0);

If myWindowObject picks out an image window that has an active effect, then we call the function QTEffects_RunEffect to run the effect.

QTEffects_RunEffect is fairly simple; indeed, it consists largely of a call to DecompressSequenceFrameWhen. The only complication is that we need to specify a time value when we call DecompressSequenceFrameWhen, passing in an ICM frame time record (of type ICMFrameTimeRecord). The ICM frame time record is declared like this:

struct ICMFrameTimeRecord {
   wide                  value;
   long                  scale;
   void                  *base;
   long                  duration;
   Fixed               rate;
   long                  recordSize;
   long                  frameNumber;
   long                  flags;
   wide                  virtualStartTime;
   long                  virtualDuration;

For rendering a filter, most of these fields can be set to 0 (except for recordSize, which should of course be sizeof(ICMFrameTimeRecord)). The QTEffects_RunEffect function, shown in Listing 7, sets them to values that are appropriate when running a transition.

Listing 7: Running the effect decompression sequence

OSErr QTEffects_RunEffect 
            (WindowObject theWindowObject, TimeValue theTime)
   ApplicationDataHdl         myAppData = NULL;
   ICMFrameTimeRecord         myFrameTime;
   OSErr                           myErr = paramErr;

   myAppData = (ApplicationDataHdl)
   if (myAppData == NULL)
      goto bail;

   if (((**myAppData).fEffectDescription == NULL) || 
            ((**myAppData).fEffectSequenceID == 0L))
      goto bail;

   // set the timebase time to the step of the sequence to be rendered
   SetTimeBaseValue((**myAppData).fTimeBase, theTime, 

   myFrameTime.value.hi         = 0;
   myFrameTime.value.lo         = theTime;
   myFrameTime.scale               = gNumberOfSteps;
   myFrameTime.base               = 0;
   myFrameTime.duration         = gNumberOfSteps;
   myFrameTime.rate               = 0;
   myFrameTime.recordSize      = sizeof(myFrameTime);
   myFrameTime.frameNumber      = 1;
   myFrameTime.flags               = 
   myFrameTime.virtualStartTime.lo      = 0;
   myFrameTime.virtualStartTime.hi      = 0;
   myFrameTime.virtualDuration            = gNumberOfSteps;


   myErr = DecompressSequenceFrameWhen(
               0, NULL, NULL, &myFrameTime);


   if (myErr != noErr)
      goto bail;


Notice that we've passed NULL as the fifth parameter to DecompressSequenceFrameWhen. If instead we were to pass a pointer to a variable of type CodecFlags (which is an unsigned short integer), then DecompressSequenceFrameWhen would return in that location a set of decompression status flags that give us information about the just-completed decompression operation. As of QuickTime 5.0, these flags are defined:

enum {
   codecFlagOutUpdateOnNextIdle                  = (1L << 9),
   codecFlagOutUpdateOnDataSourceChange      = (1L << 10),
   codecFlagSequenceSensitive                     = (1L << 11),
   codecFlagOutUpdateOnTimeChange               = (1L << 12),
   codecFlagImageBufferNotSourceImage         = (1L << 13),
   codecFlagUsedNewImageBuffer                  = (1L << 14),
   codecFlagUsedImageBuffer                        = (1L << 15)

We can inspect the codecFlagOutUpdateOnTimeChange flag (which is, alas, currently undocumented) to see whether we should render the effect repeatedly, as time changes. If that flag is clear, then we need to call DecompressSequenceFrameWhen again only when the source image changes. I'll leave it as an exercise for the reader to modify QTEffects to avoid calling DecompressSequenceFrameWhen unnecessarily.

Finishing Up

So we've completed the work required to apply a video effect to an image. When the user closes the image window, we need to call CDSequenceEnd to end the decompression sequence and then dispose of any additional memory we allocated to run the sequence. Listing 8 shows our definition of QTEffects_DumpWindowData, which handles all this clean-up.

Listing 8: Disposing of the application data

void QTEffects_DumpWindowData (WindowObject theWindowObject)
   ApplicationDataHdl      myAppData = NULL;

   myAppData = (ApplicationDataHdl)
   if (myAppData != NULL) {
      if ((**myAppData).fGWDesc != NULL)

      if ((**myAppData).fGW != NULL)

      if ((**myAppData).fSampleDescription != NULL)

      if ((**myAppData).fEffectDescription != NULL)

      if ((**myAppData).fEffectSequenceID != 0L)

      if ((**myAppData).fTimeBase != NULL)

      (**theWindowObject).fAppData = NULL;

Video Effects and Sprites

Let's continue our investigation of the QuickTime video effects architecture by learning how to use a video effect as a sprite image. In a previous article ("An Extremely Goofy Movie" in MacTech, April 2001), we saw how to use a video track as a sprite image override, so that the sprite uses the frames in the video track as the source for its images. It's just as easy to use an effect as the source for a sprite's images, and this opens up the door to some truly impressive QuickTime movies. Figure 7 shows a simple example, where the image of a sprite is provided by the fire effect.

Figure 7: A sprite image overridden by the fire effect

Figure 8 shows another simple example, where the image of a sprite is provided by the ripple effect. The ripple effect makes it appear that the penguin is submerged in a pool of water that gently undulates. (This is rather difficult to see from a single screen shot, however.)

Figure 8: A sprite image overridden by the ripple effect

In this section, we'll see how to create a sprite movie that uses an effect to supply the images for one of the sprites in the movie. We'll also see how to pass user actions to an effects component, to take advantage of any special capabilities of that component.

Using Effects as Image Overrides

In the movie shown in Figure 8, the sprite track contains two sprites: (1) our standard penguin sprite and (2) a sprite whose bounding box fills the entire movie rectangle. We use the ripple effect as an image override for the second sprite, so that the entire movie, including the penguin, appears to be under water.

When the user selects the "Make Sprite Effect Movie..." menu item, QTEffects calls the QTEffects_MakeSpriteEffectMovie function. We won't consider this function in detail, as it's virtually identical to functions we've considered in the past (for instance, see QTSprites_CreateSpritesMovie in "A Goofy Movie" in MacTech, March 2001). The important step in QTEffects_MakeSpriteEffectMovie consists of a single line of code, which we use to add the appropriate samples to the sprite media:


Recall that a sprite track consists of one or more key frame samples, which contain the images for the sprites in the track and which also specify the initial properties of those sprites. (Sprite tracks can also contain override samples, to animate the sprite by specifying changes to the sprite properties; our current sprite movie does not contain any override samples.) The image data for the penguin sprite is stored in an atom of type kSpriteImageDataAtomType in the key frame sample and consists of an image description followed immediately by the sprite image data. We specify the penguin image data in our standard way, by calling the utility function SpriteUtils_AddPICTImageToKeyFrameSample. The image data for the ripple sprite is also stored in an atom of type kSpriteImageDataAtomType; in this case, however, the atom data consists of an image description followed immediately by the effect description for the desired effect. To add the image data for the ripple sprite, we call the function QTEffects_AddRippleEffectAsSpriteImage, defined in Listing 9.

Listing 9: Adding an effect as a sprite image

void QTEffects_AddRippleEffectAsSpriteImage 
            (QTAtomContainer theKeySample, QTAtomID theImageID)
   ImageDescriptionHandle      mySampleDesc = NULL;
   QTAtomContainer                  myEffectDesc = NULL;
   OSType                              myType = kWaterRippleCodecType;
   OSErr                              myErr = noErr;

   // create a sample description
   mySampleDesc = EffectsUtils_MakeSampleDescription(myType, 
            kPenguinTrackWidth, kPenguinTrackHeight);
   if (mySampleDesc == NULL)
      goto bail;

   // create an effect description
   myEffectDesc = EffectsUtils_CreateEffectDescription(myType, 
            kSourceNoneName, kSourceNoneName, kSourceNoneName);
   if (myEffectDesc == NULL)
      goto bail;

            (theKeySample, mySampleDesc, 
            GetHandleSize(myEffectDesc), *myEffectDesc, 
            theImageID, NULL, NULL);

   if (mySampleDesc != NULL)

   if (myEffectDesc != NULL)


This is entirely straightforward: create a sample description for the ripple effect, create an effect description with no sources, and then call SpriteUtils_AddCompressedImageToKeyFrameSample to add the sample description and the effect description as the ripple sprite's image data.

For this movie to work properly, the ripple sprite must be situated in front of the penguin sprite, since the ripple effect is applied only to the movie area that lies underneath the ripple sprite. We can accomplish this, of course, by appropriately setting the sprite layer. In QTEffects, we'll set the penguin sprite's layer to 0 and the ripple sprite's layer to –1 when we create the sprite key frame sample (in QTEffects_AddPenguinMovieSamplesToMedia, defined below).

Passing Clicks to an Effects Component

The ripple effect component has a very cool feature: if the user clicks on a sprite whose image is supplied by the ripple effect, then additional, concentric ripples are drawn to simulate a stone's having been dropped in the water at the point of the mouse click. Figure 9 shows a few frames of the penguin movie immediately after the user has clicked the mouse button.

Figure 9: New ripples from a user click

Remember that the user's clicks on the ripple sprite are intercepted by the movie controller and passed to the sprite media handler for processing. We can instruct the sprite media handler to send them to the ripple effect component by adding some wiring to the sprite, like this:

            kParentAtomIsContainer, kQTEventMouseClick, 
            kActionSpritePassMouseToCodec, NULL);

The kActionSpritePassMouseToCodec action tells the sprite media handler to pass the current location of the cursor to whatever component is drawing the sprite's image. Not all components can do anything useful with that information; in fact, the ripple component is currently the only effects component that accepts mouse locations.

Listing 10 shows the complete definition of QTEffects_AddPenguinMovieSamplesToMedia.

Listing 10: Adding samples to the ripple penguin sprite track

static void QTEffects_AddPenguinMovieSamplesToMedia 
               (Media theMedia)
   QTAtomContainer         mySample = NULL;
   QTAtomContainer         mySpriteData = NULL;
   RGBColor                  myKeyColor;
   Point                     myLocation;
   short                     isVisible, myIndex, myLayer;
   OSErr                     myErr = noErr;

   // create a new, empty key frame sample
   myErr = QTNewAtomContainer(&mySample);
   if (myErr != noErr)
      goto bail; = = = 
            0xffff;      // white

   // add images to the key frame sample
            kPenguinPictID, &myKeyColor, 1, NULL, NULL);
   QTEffects_AddRippleEffectAsSpriteImage(mySample, 2);

   myErr = QTNewAtomContainer(&mySpriteData);
   if (myErr != noErr)
      goto bail;

   // the penguin sprite
   myLocation.h      = 0;
   myLocation.v      = 0;
   isVisible         = true;
   myIndex            = 1;
   myLayer            = 0;

   SpriteUtils_SetSpriteData(mySpriteData, &myLocation, 
            &isVisible, &myLayer, &myIndex, NULL, NULL, NULL);
   SpriteUtils_AddSpriteToSample(mySample, mySpriteData, 1);


   myErr = QTNewAtomContainer(&mySpriteData);
   if (myErr != noErr)
      goto bail;

   // the ripple sprite
   myLocation.h      = 0;
   myLocation.v      = 0;
   isVisible         = true;
   myIndex            = 2;
   myLayer            = -1;

   SpriteUtils_SetSpriteData(mySpriteData, &myLocation, 
            &isVisible, &myLayer, &myIndex, NULL, NULL, NULL);
            kParentAtomIsContainer, kQTEventMouseClick, 
            kActionSpritePassMouseToCodec, NULL);
   SpriteUtils_AddSpriteToSample(mySample, mySpriteData, 2);

   SpriteUtils_AddSpriteSampleToMedia(theMedia, mySample, 
            kSpriteMediaFrameDurationPenguin, true, NULL);

   if (mySample != NULL)

   if (mySpriteData != NULL)

For fun, you might try changing kQTEventMouseClick into kQTEventMouseMoved. In that case, the additional concentric ripples will occur every time you move the cursor when it's over the movie rectangle. Surfs up!

Low-Level Video Effects Functions

Before we leave the topic of video effects, it's worth mentioning that QuickTime provides a set of low-level APIs that we can use in certain cases where we need greater control over the standard effects parameters dialog box. These low-level calls begin with the prefix "ImageCodec" instead of "QT"; so, for instance, we can call ImageCodecIsStandardParameterDialogEvent in places we previously called QTIsStandardParameterDialogEvent. The parameter lists for these two functions are identical except that the low-level function adds a parameter for a component instance. This allows us to restrict the operation to a specific effects component. For example, consider the standard effects parameters dialog box, shown once again in Figure 10.

Figure 10: The standard effects parameters dialog box

As you can see, all the one-source effects are listed in the upper-left corner of the dialog box. For certain purposes, we might want to display only the parameters that are relevant to a single effect. In that case, a dialog box like the one shown in Figure 11 is preferable.

Figure 11: An effects parameters dialog box for a single effect

We can display the dialog box shown in Figure 11 by calling the ImageCodecCreateStandardParameterDialog function, whose first parameter is a component instance for the desired effects component. Listing 11 shows some code that we might use to do this.

Listing 11: Showing the parameters dialog box for a specific effects component

ComponentDescription      myCD;
Component                     myComponent = NULL;
ComponentInstance            myInstance = NULL;
QTAtomContainer               myParamDesc = NULL;
QTParameterDialog            myEffectsDialog = 0L;

// set up a component description
myCD.componentType               = decompressorComponentType;
myCD.componentSubType            = kBlurImageFilterType;
myCD.componentManufacturer   = 0;
myCD.componentFlags               = 0;
myCD.componentFlagsMask         = 0;

// find the required component
myComponent = FindNextComponent(myComponent, &myCD);
if (myComponent == NULL)

// open the component
myInstance = OpenComponent(myComponent);

// get the list of parameters for the effect
myErr = ImageCodecGetParameterList(myInstance, &myParamDesc);

// display the dialog box
myErr = ImageCodecCreateStandardParameterDialog(myInstance,
               myParamDesc, myEffectDesc, 0, NULL, 0, 

The low-level APIs are also useful if we want to embed the effects parameter dialog items into a custom dialog box, as illustrated in Figure 12. To do this, we need to call ImageCodecCreateStandardParameterDialog, as in Listing 11, passing a pointer to an existing dialog box as the fifth parameter and the dialog item index of a user item as the sixth parameter. The user item is replaced by the controls from the standard parameters dialog box.

Figure 12: Effects parameters dialog items embedded in a custom dialog box

In general, you should use either the low-level interfaces or the high-level interfaces, but not both. So if we call ImageCodecCreateStandardParameterDialog to display the effects parameters dialog box or to embed some effects parameters controls into a custom dialog box, then we should also call ImageCodecStandardParameterDialogDoAction to process events in the dialog box and ImageCodecDismissStandardParameterDialog to close the dialog box. Certain high-level functions, however, can safely be intermixed with the low-level functions. A good example is QTGetEffectsList, for which there is no low-level equivalent.


The QuickTime video effects architecture provides an easy-to-use but extremely powerful set of tools for adding video effects to movies and images. We can use it to access over 100 different generators, filters, and transitions. In this article and the previous one, we've seen how to add effects to movies, images, and sprite tracks. We've also seen how to display the effects parameters dialog box to allow the user to fine-tune an effect. And we've briefly touched on the low-level effects functions that QuickTime provides. For a more complete example of using these low-level functions, see the QTShowEffect sample code package found at (along with a handful of other effects-related sample code packages). QTShowEffect also shows how to apply a transition to two images.


Thanks are due once again to Tom Dowdy for reviewing this article and suggesting some improvements.

Tim Monroe is a member of the QuickTime engineering team. You can contact him at


Community Search:
MacTech Search:

Software Updates via MacUpdate

Adium - Popular instant messagi...
Adium is a fast and free instant messaging client which supports AIM, ICQ, Jabber, MSN, Yahoo!, Google Talk, Yahoo! Japan, Bonjour, Gadu-Gadu, Novell Groupwise, SIP/SIMPLE (Text), and Lotus Sametime... Read more
CleanMyMac 3.8.1 - $39.95
CleanMyMac makes space for the things you love. Sporting a range of ingenious new features, CleanMyMac lets you safely and intelligently scan and clean your entire system, delete large, unused files... Read more
BusyCal 3.1.7 - Powerful calendar app wi...
BusyCal is an award-winning desktop calendar that combines personal productivity features for individuals with powerful calendar sharing capabilities for families and workgroups. Its unique features... Read more
Lyn 1.8.9 - Lightweight image browser an...
Lyn is a fast, lightweight image browser and viewer designed for photographers, graphic artists, and Web designers. Featuring an extremely versatile and aesthetically pleasing interface, it delivers... Read more
Tweetbot 2.5 - Popular Twitter client.
Tweetbot is a full-featured OS X Twitter client with a lot of personality. Whether it's the meticulously-crafted interface, sounds and animation, or features like multiple timelines and column views... Read more
Monolingual 1.7.8 - Remove unwanted OS X...
Monolingual is a program for removing unnecesary language resources from OS X, in order to reclaim several hundred megabytes of disk space. If you use your computer in only one (human) language, you... Read more
Dash 4.0.3 - Instant search and offline...
Dash is an API documentation browser and code snippet manager. Dash helps you store snippets of code, as well as instantly search and browse documentation for almost any API you might use (for a full... Read more
Posterino 3.3.6 - Create posters, collag...
Posterino offers enhanced customization and flexibility including a variety of new, stylish templates featuring grids of identical or odd-sized image boxes. You can customize the size and shape of... Read more
Apple Numbers 4.1.1 - Apple's sprea...
With Apple Numbers, sophisticated spreadsheets are just the start. The whole sheet is your canvas. Just add dramatic interactive charts, tables, and images that paint a revealing picture of your data... Read more
Apple Pages 6.1.1 - Apple's word pr...
Apple Pages is a powerful word processor that gives you everything you need to create documents that look beautiful. And read beautifully. It lets you work seamlessly between Mac and iOS devices, and... Read more

Latest Forum Discussions

See All

Die With Glory (Games)
Die With Glory 1.2.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.2.0 (iTunes) Description: Die with Glory is an epic adventure game where your goal is to die in glorious fashion. You must help Sigurd, a brave old... | Read more »
Get Ike in the new Fire Emblem: Heroes u...
One of the most popular Fire Emblem characters is finally available in a new update to Nintendo'sFire Emblem: Heroes. [Read more] | Read more »
Die With Glory in a new viking adventure...
If you're a fan of classic adventure games you'll do well to pick upDie With Glory, the gorgeous new title from Cloud Castle inc. Die With Glory updatesthe gameplay of the same kind of adventure classics such asMonkey Island for modern, mobile... | Read more »
Get up to speed with everything you need...
In case you haven’t heard, MU Origin just got a colossal new update with new all-server events, battle modes, and systems making their way to the land of MU. Here’s a handy guide to everything you need to know about the latest content. [Read... | Read more »
Minimalist puzzle game, Cuts, free on iO...
If you're looking for a gorgeous puzzle experience on iOS devices, developer's aesthetically interesting puzzler, Cuts, is discounted to free on the iOS App Store right now. [Read more] | Read more »
Anime tactical RPG, War of Crown, comes...
If you're looking for another tactical RPG fix to go alongside your Fire Emblem Heroes campaigns check out Gamevil's newest, anime-inspired tactics RPG, War of Crown, which comes out tomorrow. [Read more] | Read more »
Fantasy MMORPG MU Origin adds new modes,...
MU Origin, Webzen’s highly popular fantasy MMORPG is getting ready to shake things up for the second time this year, as a new update makes its way to the Google Play and App Store from today. Introducing new systems, modes, and events, the land of... | Read more »
Blizzard is looking to hire a mobile dev...
A new thread on the popular video game rumor forum, NeoGAF, uncovered an interesting job listing over at Blizzard Entertainment. It appears the studio behindStarCraft, World of WarCraft, Hearthstone,andOverwatch is looking to bring on a new hire... | Read more »
Legend of Zelda meets Cooking Mama in ne...
Dungeon Chef is what happens when you mix the RPG elements (and style) of a Legend of Zelda game, with cooking elements. Although, now that The Legend of Zelda: Breath of the Wild also has cookingelements, so maybe the gameplay is not so novel.... | Read more »
ChordFlow (Music)
ChordFlow 1.0.0 Device: iOS Universal Category: Music Price: $6.99, Version: 1.0.0 (iTunes) Description: ChordFlow is a chord sequencer with a unique 4-track polyphonic arpeggiator, extensive chord library, MIDI out and Ableton Link... | Read more »

Price Scanner via

Digital Paper Tablet Offers Distraction Free...
I typically spend 8-10 hours a day gazing at the screens in my laptops and iPad, as tools of my livelihood, I don’t as a rule use electronic devices for pleasure reading. I subscribe to a daily... Read more
“Today at Apple” Bringing New Educational Ses...
Apple has announced plans to launch dozens of new educational sessions next month in all 495 Apple Stores ranging in topics from photo and video to music, coding, art and design, and more. The hands-... Read more
Smart Finance Free Comprehensive Personal Fin...
Moscow-based indie developer, Alexander Survillo has announced the release and immediate availability of Smart Finance: Personal Finance, Budget & Money 1.1.4, an update to his comprehensive... Read more
12-inch 1.1GHz Retina MacBooks on sale for $1...
B&H has 12″ 1.1GHz Retina MacBooks on sale for $100 off MSRP. Shipping is free, and B&H charges NY & NJ sales tax only: - 12″ 1.1GHz Space Gray Retina MacBook: $1199.99 $100 off MSRP - 12... Read more
13-inch 2.7GHz Retina MacBook Pro on sale for...
B&H Photo has the 13″ 2.7GHz Retina MacBook Pro on sale for $130 off MSRP. Shipping is free, and B&H charges NY & NJ tax only: - 13″ 2.7GHz/128GB Retina MacBook Pro (MF839LL/A): $1169 $... Read more
15-inch 2.2GHz Retina MacBook Pros available...
B&H Photo has the 15″ 2.2GHz Retina MacBook Pro available for $200 off MSRP including free shipping plus NY & NJ sales tax only: - 15″ 2.2GHz Retina MacBook Pro (MJLQ2LL/A): $1799.99 $200 off... Read more
13-inch Touch Bar MacBook Pros on sale for up...
B&H Photo has the 2016 Apple 13″ Touch Bar MacBook Pros in stock today for up to $150 off MSRP. Shipping is free, and B&H charges NY & NJ sales tax only: - 13″ 2.9GHz/512GB Touch Bar... Read more
Apple refurbished Apple TVs available for up...
Apple has Certified Refurbished 32GB and 64GB Apple TVs available for up to $30 off the cost of new models. Apple’s standard one-year warranty is included with each model, and shipping is free: -... Read more
12-inch 1.2GHz Retina MacBooks on sale for up...
B&H has 12″ 1.2GHz Retina MacBooks on sale for up to $160 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 12″ 1.2GHz Space Gray Retina MacBook: $1439.99 $160 off MSRP - 12″ 1... Read more
HyperX Ships Pulsefire FPS Gaming Mouse, Winn...
Your reporter is a longtime fan of gaming mice for general purpose coomnputing use, finding them typically superior in comfort and performance. HyperX, a division of Kingston Technology Company, Inc... Read more

Jobs Board

*Apple* Mac Computer Technician - GeekHampto...
…complex computer issues over the phone and in person? GeekHampton, Long Island's Apple Premium Service Provider, is looking for you! Come work with our crew Read more
Product Manager, *Apple* Platforms - Viacom...
…Product Manager to drive the execution of its iOS and AppleTV experiences. The Apple Platform Product Manager will be a leader in our Agile/Scrum environment and Read more
Geek Squad *Apple* Master Consultation Agen...
**500662BR** **Job Title:** Geek Squad Apple Master Consultation Agent **Location Number:** 000286-Canton-Store **Job Description:** **What does a Geek Squad Read more
*Apple* Mobile Master - Best Buy (United Sta...
**500710BR** **Job Title:** Apple Mobile Master **Location Number:** 000279-North Olmsted-Store **Job Description:** **What does a Best Buy Apple Mobile Master Read more
*Apple* Engineering Specialist - CSRA (Unite...
Apple Engineering Specialist All times are in Eastern Daylight Time Requisition ID Job Locations US DC Washington DC Posted Date Category Engineering Sciences Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.