Volume Number: 21 (2005)
Issue Number: 7
Column Tag: Programming

QuickTime Toolkit

Back To The Future, Part III

by Tim Monroe

Developing Applications with the QuickTime for Cocoa Kit

In the previous two QuickTime Toolkit articles (in MacTech, May and June 2005), we've taken a look at using QTKit, Apple's new Cocoa framework for displaying and modifying QuickTime movies. We saw how to open and manipulate movies using command-line tools; then we stepped through the process of building a fairly complete document-based Cocoa application using QTKit. This application, which we called KitEez, supports the standard document-related behaviors, including saving an edited movie into its original file or into some other file, and reverting to the last saved version of a movie file. Since KitEez was built using the Cocoa class NSDocument, we didn't need to write any code to track the edited state of the document window and its associated movie or to display the standard dialog boxes that warn the user about unsaved changes and overwriting existing files.

Introduction

In this article, we'll take a look at a few more of the capabilities offered in by QTKit. In particular, we'll see how to add images to an existing QuickTime movie and work with notifications and delegates. We'll also see how to call QuickTime functions that have no corresponding QTKit method and we'll take a second look at opening files and URLs.

Adding Images to a Movie

Adding Images to an Existing Movie

Imagine that we've opened a QuickTime movie from a file and we'd like to tack a few images onto the end of the movie. The QTMovie class provides a method that makes this extremely easy, addImage:forDuration:withAttributes, whose signature is this:

-(void)addImage:(NSImage *)image 
            forDuration:(QTTime )duration
            withAttributes:(NSDictionary *)attributes;

The image parameter of course specifies the image to add to the end of the movie, and the duration parameter is a QTTime structure that specifies the desired duration of the new frame. The attributes parameter is a dictionary that specifies the codec to be used to compress the image and possibly also the quality of the compressed image. For instance, Listing 1 shows a simple method that reads an image from the application's nib file and adds it to the end of the movie, with a duration of one second.

Listing 1: Appending an image to a movie

- (void)addImageFromNibFile:(NSString *)name
{
   NSImage *image = [NSImage imageNamed:name];
   QTTime duration = QTMakeTime(600, 600);
   NSDictionary *dict = [NSDictionary dictionaryWithObject:
         QTStringForOSType('pxlt') forKey:QTAddImageCodecType]

   [_movie addImage:image forDuration:duration withAttributes:dict];
}

As you can see, the dict dictionary contains one key-value pair, where the key is QTAddImageCodecType. This key specifies the codec to be used to compress the image before it's added to the movie. Notice that we have used the function QTStringForOSType to convert a codec OSType (here, for the Pixlet compressor) to an NSString object. QTKit also provides a utility function for going in the opposite direction: QTOSTypeForString will return an OSType corresponding to a given NSString.

The dictionary of attributes can also contain a key-value pair in which the key is QTAddImageCodecQuality. If this pair is present, then the specified value, which should be an NSNumber, is used as the quality setting for the compressor. This key-value pair is optional; if it is not included in the dictionary, the quality setting codecNormalQuality is used. The acceptable values are listed in this enumeration in the header file ImageCompression.h in the framework QuickTime.framework:

enum {
  codecLosslessQuality   = 0x00000400,
  codecMaxQuality        = 0x000003FF,
  codecMinQuality        = 0x00000000,
  codecLowQuality        = 0x00000100,
  codecNormalQuality     = 0x00000200,
  codecHighQuality       = 0x00000300
};

Adding Images to a New Movie

Suppose that instead of adding an image to the end of an existing movie, we want to create a slideshow movie from some images, starting this time from an empty movie. You might think that we could first create a new, empty QTMovie object and then tack on the images, like this:

QTMovie *movie = [[QTMovie alloc] init];
[movie addImageFromNibFile:@"A.jpg"];
[movie addImageFromNibFile:@"B.jpg"];			
// and so forth...

In fact, however, this would fail. The underlying reason is that a new QTMovie object created in this way has no writable data reference associated with it. In layman's terms, this means that the associated QuickTime Movie has not yet been assigned a place to store any new media data that is added to the movie. Since there is no place to write the media data, the call to addImage:forDuration:withAttributes fails.

You might think that this is a bug in QTMovie's init method, which could easily assign a block of data in memory as the writable data reference for a new QTMovie object. But that revised behavior would lead to problems of its own. For instance, adding a number of large images could soon fill up the available physical memory or at least put the virtual memory system into severe thrashing.

Future versions of QTKit will almost certainly provide one or more new methods in QTMovie to create a new empty QTMovie object that is associated with a writable file or block of memory. In the meantime, it is possible to work around this limitation by creating a new empty file and associating it with a new QTMovie object using the movieWithQuickTimeMovie:disposeWhenDone:error: method, as shown in Listing 2. As you can see, we use the movie storage APIs (discussed in "Modern Times" in MacTech, May, 2004) to create a new empty movie file.

Listing 2: Creating a new empty movie that is writable

- (QTMovie)movieWithWritableFile:(NSString *)filename handler:(DataHandler *)dataHandler
{
   OSErr err = noErr;
   Handle dataRef = nil;
   OSType dataRefType;
   Movie qtMovie = NULL;
   QTMovie movie = nil;

   // make sure we are passed a location to return the data handler identifier
   if (!dataHandler) return nil;

   // create a file data reference for the specified filename
   err = QTNewDataReferenceFromFullPathCFString(
                     (CFStringRef)fileName,
                     kQTNativeDefaultPathStyle,
                     0, &dataRef, &dataRefType);
   if (err != noErr) return nil;
	
   // create a QuickTime movie from the file data reference
   err = CreateMovieStorage(dataRef, dataRefType, 'TVOD',
                     smSystemScript, newMovieActive, 
                     dataHandler, &qtMovie);
   if (err != noErr) return nil;

   // instantiate a QTMovie from the QuickTime movie
   movie = [QTMovie movieWithQuickTimeMovie:qtMovie 
                     disposeWhenDone:YES error:nil];

   // mark the movie as editable
   [movie setAttribute:[NSNumber numberWithBool:YES] 
                     forKey:QTMovieEditableAttribute];

   return movie;
}

We need to return to the caller both the new QTMovie object and the identifier of the data handler associated with the open movie file. That's because, when we are finished adding frames to the movie and have called updateMovieFile, we need to call CloseMovieStorage to close the movie:

CloseMovieStorage(dataHandler);

Creating a Movie from a Single Image

It's worth mentioning one further twist on this issue, namely using QTKit to create a movie from a single image. Suppose we want to create a 10-second movie from a single image. We could use the strategy employed in the previous section, first creating a new empty movie file and then adding the specified image to its associated QTMovie object. But there is in fact a much simpler way to do this, using the dataReferenceWithReferenceToData:name:MIMEType: method in the QTDataReference class. In a nutshell, this method creates a data reference to some block of memory addressed using an NSData object and optionally attaches a filenaming extension or a data reference extension of type 'mime' to the data reference. As we have seen in earlier articles (particularly in "Somewhere I'll Find You" in MacTech, October, 2000), these extensions help QuickTime find the appropriate movie or graphics importer when a data reference is to a block of memory.

In the present case, we'll create an NSData object that contains a TIFF representation of the image and then create a QTDataReference object that has, as its filenaming extension, an arbitrary name with the extension .tiff. Listing 3 shows the code we can use to do this.

Listing 3: Creating a movie from a single image

- (void)createMovieFileFromImage:(NSString *)filename 
                                       (NSImage *)image
{
   if (!filename || !image)
      return;

   NSData *data = [image TIFFRepresentation]; 
   QTDataReference *dataRef = [QTDataReference 
               dataReferenceWithReferenceToData:data 
               name:@"some_image.tiff" MIMEType:nil]; 
   QTMovie *movie = [QTMovie movieWithDataReference:dataRef 
               error:nil]; 

   // make the movie editable 
   [movie setAttribute:[NSNumber numberWithBool:YES] 
                     forKey:QTMovieEditableAttribute]; 

   // set the duration to 10 seconds 
   QTTimeRange range = QTMakeTimeRange(QTZeroTime, 
                                                      [movie duration]); 
	[movie scaleSegment:range newDuration:QTMakeTime(10, 1)]; 

   // export as a 3gpp file
   NSDictionary *dict= [NSDictionary 
            dictionaryWithObject:[NSNumber numberWithBool:YES] 
            forKey:QTMovieFlatten];
   [movie writeToFile: filename withAttributes:dict];
}

This works because we are not actually editing the movie's media data. Instead, we are creating a QTMovie object that gets its media data from the NSData block. We are editing the movie when we call the scaleSegment:newDuration: method, but that's okay even when the movie does not have a writable data reference.

QuickTime Functions

Even though QTKit exposes a fairly large number of classes and methods, and even though it does quite a bit of movie-related processing automatically, it's fairly likely that we will want to use QuickTime capabilities that it does not currently support. For instance, we've already seen that QTKit does not yet provide a method for creating a new, empty movie that has a writable data reference associated with it. So we used the underlying QuickTime APIs to create one, which we then used to initialize a QTMovie object. It can also happen that we've already got a QTMovie object and we want to perform some operation not yet supported by QTKit. In this case, the QTMovie class provides two useful methods that allow us to retrieve the Movie and MoveController identifiers associated with a QTMovie object:

-(Movie)quickTimeMovie;
-(MovieController)quickTimeMovieController;

Suppose, for example, that we want to set the magnification (or "zoom") level of a Flash movie. We can do so by using the quickTimeMovie method to retrieve the QuickTime Movie associated with a QTMovie object and then using Movie Toolbox and Flash media handler functions, as shown in Listing 4.

Listing 4: Setting the zoom level of a Flash movie

- (void)setZoom:(float)zoomPct
{
   Track flashTrack = NULL;
   Media flashMedia = NULL;
   MediaHandler flashHandler = NULL;

   flashTrack = GetMovieIndTrackType([self quickTimeMovie], 
                        1, FlashMediaType, movieTrackMediaType | 
                        movieTrackEnabledOnly);
   if (flashTrack) {
      flashMedia = GetTrackMedia(flashTrack);
   flashHandler = GetMediaHandler(flashMedia);
      FlashMediaSetZoom(flashHandler, zoomPct);
   }
}

In general, it should be safe to call virtually any QuickTime functions on the Movie or the MoveController associated with a QTMovie object. Some operations, however, might not be safe and should probably be avoided. In particular, QTKit generally assumes that it will be disposing of the Movie and MovieController associated with a QTMovie object, so you should not call DisposeMovie or DisposeMovieController. (The exception to this rule is when you pass the value YES as the disposeWhenDone parameter in QTMovie's movieWithQuickTimeMovie:disposeWhenDone:error: method, which tells QTKit that you want to dispose of the movie yourself.) Also, deleting tracks from a movie using the Movie Toolbox function DisposeMovieTrack is likely to confuse QTKit and may even lead to crashes. It's likely that QTKit will in future versions add methods that provide a way to delete tracks, since this is a reasonably common operation for some kinds of applications.

Notifications and Delegate Methods

As you no doubt already know, a notification is a way for one Cocoa object to inform other objects about changes in its state or its properties. The QTMovie class defines a large number of notifications that other objects can listen for; here are the notifications that are currently defined:

NSString *QTMovieEditabilityDidChangeNotification;
NSString *QTMovieEditedNotification;
NSString *QTMovieLoadStateDidChangeNotification;
NSString *QTMovieLoopModeDidChangeNotification;
NSString *QTMovieMessageStringPostedNotification;
NSString *QTMovieRateDidChangeNotification;
NSString *QTMovieSelectionDidChangeNotification;
NSString *QTMovieSizeDidChangeNotification;
NSString *QTMovieStatusStringPostedNotification;
NSString *QTMovieTimeDidChangeNotification;
NSString *QTMovieVolumeDidChangeNotification;
NSString *QTMovieDidEndNotification;
NSString *QTMovieChapterDidChangeNotification;
NSString *QTMovieChapterListDidChangeNotification;
NSString *QTMovieEnterFullScreenRequestNotification;
NSString *QTMovieExitFullScreenRequestNotification;
NSString *QTMovieCloseWindowRequestNotification;

Most of these are fairly obvious. For instance, QTMovieDidEndNotification is posted when a QTMovie object reaches its end. And QTMovieEditedNotification is posted when a QTMovie object has been edited or changed in some way. A few of these are however somewhat less obvious. The QTMovieTimeDidChangeNotification notification is not, for instance, posted whenever the movie time changes; rather, it's posted whenever the movie time changes to a value different from what it would be during normal movie playback. These sorts of time changes include operations like the user clicking in the movie controller bar to change the movie time or a wired action setting the movie time to some specific time.

In most cases, these notifications are posted using the NSNotificationCenter method postNotificationName:object:, where no accompanying userInfo dictionary is passed. The expectation is that the notification listener will be able to retrieve whatever information about the QTMovie object is needed at the time the notification is received. So, for instance, a listener receiving the QTMovieVolumeDidChangeNotification notification could call the QTMovie method volume to retrieve the current volume of the movie. In three cases, however, a dictionary of information is passed along with the notification, because there is no easy way for the receiver to ask for that information; these are:

QTMovieMessageStringPostedNotification
QTMovieRateDidChangeNotification 
QTMovieStatusStringPostedNotification

We've already seen an example of registering for notifications in the previous article. When we initialize a movie view to hold a movie, we want the associated document to be informed of any size changes that occur programmatically or via wired actions; we did that by registering for the QTMovieSizeDidChangeNotification notification, like this:

[[NSNotificationCenter defaultCenter] addObserver:self 
         selector:@selector(boundsDidChange:) 
         name:QTMovieSizeDidChangeNotification object:movie];

Listing 5 shows our implementation of the boundsDidChange: method, which is invoked when we receive this notification.

Listing 5: Handling size-changed notifications

- (void)boundsDidChange:(NSNotification *)notification
{
   // set the size of the movie window to exactly enclose the movie at its current size
   NSSize size = [[[movieView movie] attributeForKey:
                        QTMovieCurrentSizeAttribute] sizeValue];

   [[movieView window] setContentSize:
                        [self windowContentSizeForMovieSize:size]];
}

Notifications provide a very loose sort of coupling between objects in Cocoa. One object posts a notification and one or more other objects can listen for that notification and respond appropriately. By contrast, a much tighter association of two objects can be formed by designating one of them as the delegate of the other. Currently QTKit defines only five public delegate methods, all in the QTMovie class, of which only three are likely to be of use to developers:

- (BOOL)movie:(QTMovie *)movie linkToURL:(NSURL *)url;
- (QTMovie *)externalMovie:(NSDictionary *)dictionary;
- (BOOL)movie:(QTMovie *)movie
                     shouldContinueOperation:(NSString *)op 
                     withPhase:(QTMovieOperationPhase)phase 
                     atPercent:(NSNumber *)percent 
                     withAttributes:(NSDictionary *)attributes;

A delegate's movie:linkToURL: method is called when the movie controller is about to open a movie specified by a URL. (This corresponds to the movie controller processing an mcActionLinkToURL action in the Carbon world.) For most applications, the QTKit's normal processing of URLs is fine for most applications; you would define this delegate method if you wanted to reroute URLs to some other location or cancel URL opening altogether (by returning NO).

A delegate's externalMovie: method is called when the movie controller needs to find a movie external to some given movie, most often to send that other movie a wired action. Once again, QTKit contains code to find external movies and most applications can rely on that automatic processing.

The only QTMovie delegate method likely to be defined by applications is movie:shouldContinueOperation:withPhase:atPercent: withAttributes:, which provides a Cocoa wrapper for a movie progress procedure. Currently this is useful only when calling the writeToFile:withAttributes method, though it's possible that more operations in QTMovie will support this delegate method in the future.

Opening Movies

In our first QTKit article (mentioned earlier), we opened a movie specified by a filename using the initWithFile:error: method, like this:

QTMovie *movie = [QTMovie initWithFile:filename error:nil];

We can also open a movie specified by a URL using the initWithURL:error: method, like this:

QTMovie *movie = [QTMovie initWithURL:url error:nil];

It's useful to know that there is a more general method for opening movies which offers several advantages over these methods, namely initWithAttributes:error:. The following lines of code do exactly the same thing as the line of code that uses initWithFile:error: above.

NSDictionary *dict = [NSDictionary dictionaryWithObject:
         filename forKey:QTMovieFileNameAttribute]
QTMovie *movie = [QTMovie initWithAttributes:dict 
                                       error:nil];

The idea is that we pass into initWithAttributes:error: a dictionary containing one or more key-value pairs that specify the attributes we want the new QTMovie object to have. One of those attributes must specify the location of the movie data, either via a filename or a URL or a QTDataReference object or a pasteboard or an NSData block. But there can be an indefinite number of other attributes in that dictionary, which are applied to the movie object before it's returned to the caller. For instance, we can ensure that all movies we open are editable by creating the dictionary like this:

dict = [NSDictionary dictionaryWithObjectsAndKeys:
   filename, QTMovieFileNameAttribute,
   [NSNumber numberWithBool:YES], QTMovieEditableAttribute,
   nil];

The initWithAttributes:error: method is not merely a convenience that saves us from setting attributes on the QTMovie object returned by calls to initWithFile:error: or initWithURL:error:. There are at least two important reasons we might need to call it rather than those other methods. First, we can pass in attributes that override some of the default behaviors of QTKit when opening movies. For example, we learned in an earlier article that QTKit opens all movies asynchronously. (That is, a call to initWithURL:error: returns almost immediately, so that the application can continue with its processing while the movie data loads.) But it's possible that an application would want to load movies synchronously. In that case, if could construct an attributes dictionary like this:

dict = [NSDictionary dictionaryWithObjectsAndKeys:
   url, QTMovieURLAttribute,
   [NSNumber numberWithBool:NO], QTMovieOpenAsyncOKAttribute,
   nil];

Setting NO as the value of the QTMovieOpenAsyncOKAttribute attribute indicates that we want the movie to be opened synchronously. Obviously, this attribute needs to be set before we start opening the remote movie, and initWithAttributes:error: is the only way to do that using QTKit methods.

The second case in which this method is sometimes necessary is to set a delegate for the QTMovie object being created so that that delegate is operational during the movie loading. In particular, it's quite possible that an mcActionLinkToURL action is encountered by the movie controller before the initWithURL:error: method returns to the caller. So this sequence of calls might not work if the delegate needs to handle all link-to-URL actions:

QTMovie *movie = [QTMovie initWithURL:url error:nil];



[movie setDelegate:self];

Instead, we can call initWithAttributes:error:, adding the QTMovieDelegateAttribute and its associated object to the dictionary of attributes:

dict = [NSDictionary dictionaryWithObjectsAndKeys:
   url, QTMovieURLAttribute,
   self, QTMovieDelegateAttribute,
   nil];
QTMovie *movie = [QTMovie initWithAttributes:dict 
                                       error:nil];

By setting the delegate in this way, we can guarantee that all link-to-URL actions will be seen by the movie:linkToURL: delegate method.

If your application does not need to use any of the delegate methods defined in the QTMovie class and if it does not need to override any of the default behaviors provided by QTMovie, then the initWithFile:error: and initWithURL:error: methods are just fine.

Conclusion

This article and the previous two articles should help convince you that QTKit is a powerful framework that allows you to develop robust QuickTime applications with a minimum of code. QTKit provides the most comprehensive set of built-in capabilities and the most complete automatic processing of any QuickTime framework that I have yet encountered. It's easy to use, it dovetails nicely with existing Cocoa frameworks, and it's already serving as the QuickTime underpinnings of powerful in-house and commercial multimedia applications.

In the next article, we'll take one more look at QTKit, to investigate how to execute QTKit methods on a secondary thread. That will give us the machinery we need to perform computationally intensive operations (like exporting large movies or creating slideshow movies from a large number of images) without negatively impacting the responsiveness of the application's user interface.

References

The movieWithWritableFile:handler: method shown earlier is based on sample code that is available at http://developer.apple.com/samplecode/QTKitCreateMovie/QTKitCreateMovie.html.

Tim Monroe is a member of the QuickTime engineering team at Apple. You can contact him at monroe@mactech.com. The views expressed here are not necessarily shared by his employer.