Developing Applications With The QuickTime For Cocoa Kit
Volume Number: 21 (2005)
Issue Number: 5
Column Tag: Programming
QuickTime Toolkit
Back To The Future
by Tim Monroe
Developing Applications With The QuickTime For Cocoa Kit
In several earlier articles ("The Cocoanuts" in MacTech, December 2002 and "Animal Crackers" in January
2003), we took a look at developing QuickTime applications using the Cocoa classes NSMovie and NSMovieView. We
found that these classes provide reasonably good support for displaying and editing QuickTime movies. They are
tightly integrated with the Cocoa application framework as a whole, and they succeed marvelously at shielding
the Cocoa programmer from the Carbon data structures and APIs native to QuickTime.
Introduction
In several earlier articles ("The Cocoanuts" in MacTech, December 2002 and "Animal Crackers" in January
2003), we took a look at developing QuickTime applications using the Cocoa classes NSMovie and NSMovieView. We
found that these classes provide reasonably good support for displaying and editing QuickTime movies. They are
tightly integrated with the Cocoa application framework as a whole, and they succeed marvelously at shielding
the Cocoa programmer from the Carbon data structures and APIs native to QuickTime.
Nonetheless, the two classes NSMovie and NSMovieView provide only a limited selection of methods for
managing QuickTime movies. We saw that we fairly quickly needed to subclass NSMovieView to override its
default behaviors (for instance to support a contextual menu that was sensitive to the type of movie being
displayed in a view). And we needed to add categories to these classes to extend them in fairly simple ways
(for instance to show and hide specific buttons in the movie controller bar). NSMovie and NSMovieView are fine
as far as they go; the main problem with them is that they don't go very far at all. They are intentionally
limited to movie playback and simple editing, and even there they sidestep the opportunity to incorporate
movie opening and playback behaviors that even a simple application would probably like to exhibit (such as
loading movies asynchronously or automatically finding external target movies for wired actions).
Apple has recently introduced an enhanced set of Cocoa classes that support QuickTime movie playback and
editing, called the QuickTime For Cocoa Kit or (more briefly) the QTKit. This new framework first shipped with
Mac OS X 10.4 ("Tiger") and is also available for Mac OS X 10.3.9 ("Panther") as part of QuickTime 7. In this
article and the next several articles, I want to examine the QTKit framework in some detail. The QTKit is an
important piece of new Cocoa technology that is already used by key applications such as QuickTime Player (on
Tiger and Panther) and Spotlight (on Tiger). It promises to significantly simplify the process of adding
multimedia capabilities to Cocoa applications.
In this article, we'll take an introductory look at QTKit's classes and their methods, as well as the
structures and functions it uses to manipulate times and time ranges. We'll see how to use the QTKit to
display a QuickTime movie in a Cocoa document window and control the playback of that movie. We'll also see
how to use QTKit methods in a command-line tool. In the following articles, we'll look at more advanced uses
of these new classes. We'll learn how to use the QTKit to export and flatten movies, and how to perform more
refined movie editing.
QuickTime for Cocoa Kit Overview
The QTKit is a Cocoa framework that provides support for opening and displaying QuickTime movies, and for
performing various sorts of manipulations on those movies. Note that I did not write: "QTKit is a framework
that provides a Cocoa wrapper for all of QuickTime". There are large pieces of QuickTime functionality that
are not addressed by the version of QTKit released in QuickTime 7. For instance, there are no classes for
capturing video or sound data (thereby wrapping QuickTime's sequence grabbing capabilities), and there are no
classes for working directly with sprites or wired actions or atom containers or QuickTime VR hot spots. The
version of QTKit released in QuickTime 7 should be viewed as a very good start to the Cocoaization of
QuickTime, not the completed journey.
QTKit Classes
The QTKit framework currently exposes five classes: QTMovie, QTTrack, QTMedia, QTDataReference, and
QTMovieView. QTMovieView is just what you would expect: it's a subclass of NSView that can display QuickTime
movies and allow the user to interact with them. Figure 1 shows a Cocoa window whose content area is
completely filled by a QTMovieView.
Figure 1: A window that contains a QTMovieView
As you can see, this QTMovieView contains the standard movie controller bar. However, you can hide the bar
by invoking the setControllerVisible: method, like this:
[movieView setControllerVisible:NO];
The QTMovieView class provides methods for controlling movie playback, positioning the current movie time,
and performing edit operations on the current movie selection (cut, copy, paste, etc.). It also provides a
contextual menu that allows access to a handful of these methods, as shown in Figure 2.
Figure 2: A contextual menu in a QTMovieView
In QTMovieView, this menu is truly contextual; for a Flash movie, the menu is the one shown in Figure 3.
Figure 3: A contextual menu in a Flash movie
You associate a QuickTime movie with a QTMovieView object by calling the setMovie: method, which takes a
QTMovie object as its parameter.
QTMovie is the key class in QTKit. It is a Cocoa wrapper for a QuickTime movie and a QuickTime movie
controller. QTMovie provides an extensive selection of methods for opening QuickTime movies from files, URLs,
the Pasteboard, an NSData object, and data specified by a QuickTime data reference. For instance, if filename
is the full pathname of a QuickTime movie file, this line of code creates and initializes a QTMovie object
associated with that movie file:
[[QTMovie alloc] initWithFile:filename error:nil];
Notice the error parameter; it's a pointer to an NSError object that is used to return to the caller an
indication of any error that occurs during the movie initialization. All QTMovie initialization methods use
that parameter.
QTMovie also provides an extensive selection of methods for getting and setting movie properties,
controlling movie playback and current time, getting and setting the movie selection, and editing the movie
selection. We'll encounter a number of these methods in the following discussion.
The remaining three classes, QTTrack, QTMedia, and QTDataReference provide Cocoa representations of
QuickTime tracks, media, and data references, respectively. For simple movie playback and editing, you will
not need to use these classes and we will not discuss them further in this article.
Time Structures and Functions
Time positions and durations are managed in QTKit using two structures, QTTime and QTTimeRange. Note that
these are not objects, since creating an object each time we needed to specify a location in movie or a
duration of a movie segment would have been onerous and inefficient. Here is the declaration of the QTTime
structure:
typedef struct {
long long timeValue;
long timeScale;
long flags;
} QTTime;
The timeValue field is a 64-bit value that indicates a time value. This value is interpreted relative to
the timeScale field, which indicates the number of units per second. The flags field is usually 0; in rare
circumstances the kQTTimeIsIndefinite flag may be set, indicating that the other two fields are not to be
trusted. (This might happen if you try to do an operation on a QTTime structure whose timeScale field is 0.)
The QTKit provides a handful of utility methods for initializing and manipulating QTTime structures. For
instance, QTMakeTime takes a time value and a time scale and then returns a QTTime structure. We can specify
the four-second point in a movie like this:
QTTime startTime = QTMakeTime(4, 1);
The QTTimeRange structure specifies a range or duration of time values; it's declared like this:
typedef struct {
QTTime time;
QTTime duration;
} QTTimeRange;
QTKit Behaviors
One metric of the value of a framework is not the number and variety of the methods you can call, but (so
to speak) the methods you don't need to call -- that is, capabilities that are provided "for free" by the
framework. This is an especially valuable metric for a QuickTime framework, as the operations required to
(say) open and display a QuickTime movie can easily become quite numerous. As we've seen in earlier articles
(particularly "Loaded" in MacTech, September 2002), loading a movie from a remote server without blocking the
application (that is, asynchronously) requires a fair amount of work. We need to make special API calls to
start the asynchronous loading and then we need to monitor the movie's load state until it has reached a
certain threshold. And we need to explicitly task the movie all the while. In QTKit, all movies are loaded
asynchronously, with absolutely no extra work required by the developer. And QTKit takes care of tasking the
movie at appropriate intervals. Loading a remote movie asynchronously is as simple as executing
initWithURL:error:.
Here are a few more of the behaviors provided for free to QTKit client applications:
Multi-level undo for editing operations. QTKit takes advantage of existing Cocoa classes to support
multi-level undo and redo for editing operations on a QTMovieView object. In earlier versions of QuickTime
Player, for instance, only the most recent edit operation is undoable.
Intermovie communication. QTMovie automatically finds the target of a wired action that is targeted at another movie.
Drag and drop editing. QTKit supports dragging a movie selection out of a QTMovieView and dropping it into
any other object that can accept dragged movie data; dragging between Cocoa and Carbon applications is fully
supported.
Live resizing. A QTMovieView object can be resized while the associated movie is playing. Earlier versions
of QuickTime Player drew a ghost outline of the movie frame and window frame during resizing and redrew the
movie data only when the final size had been reached.
Constant pitch audio. In previous versions of QuickTime, selecting a faster playback rate would alter the
pitch of the audio (resulting in the infamous "chipmunk" effect). QTKit automatically takes advantage of
improvements in the QuickTime 7 audio architecture to maintain a constant pitch during accelerated movie
playback.
Keep in mind that these are only a few of the improved behaviors that all QTKit client applications exhibit
as part of using QTKit.
Movie Editing
Let's begin our hands-on exploration of the QTKit framework by seeing how to delete a segment of a
QuickTime movie using QTKit APIs. First, we need to open the movie file; if the movie file is located at
/tmp/sample.mov, we can use this line of code:
QTMovie *movie = [QTMovie movieWithFile:@"/tmp/sample.mov" error:nil];
Next we need to specify the movie segment that is to be deleted. We can use the QTTime and QTTimeRange
structures and associated functions to do that:
QTTime startTime = QTMakeTime(4, 1);
QTTime duration = QTMakeTime(2, 1);
QTTimeRange range = QTMakeTimeRange(startTime, duration);
These lines of code specify a segment that is two seconds long beginning four seconds into the movie.
We can delete the segment by using the QTMovie method deleteSegment:, like so:
[movie deleteSegment:range];
There is however one gotcha: by default, a QTMovie object is uneditable. So we first need to set the
editable state of the movie like this:
[movie setAttribute:[NSNumber numberWithBool:YES]
forKey:QTMovieEditableAttribute];
The method setAttribute:forKey: is the standard way of setting an attribute of a QTMovie object. The header
file QTMovie.h lists a large number of keys for this method.
Finally, to save the edited movie back into the original file, we can execute the updateMovieFile method:
[movie updateMovieFile];
Movie Exporting
Let's consider another easy example. Suppose we want to save an existing QuickTime movie file as a 3GPP
file (which is streamable to cell phones and other terminal devices that conform to the 3GPP standard). In
this case, we can use the writeToFile:withAttributes: method supplied by QTMovie. This method is used to
export or flatten a QuickTime movie, depending on the contents of the attributes dictionary. Listing 1 shows a
complete command-line tool that creates the dictionary, adds the appropriate objects and keys, and then
exports the movie.
Listing 1: Exporting a movie as a 3GPP file
#import <QTKit/QTKit.h>
int main ()
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
QTMovie *movie = [QTMovie movieWithFile:@"/tmp/sample.mov" error:nil];
NSDictionary *dictionary = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithBool:YES], QTMovieExport,
[NSNumber numberWithLong:kQTFileType3GPP],
QTMovieExportType, nil];
[movie writeToFile:@"/tmp/sample.3gp" withAttributes:dictionary];
[pool release];
exit(1);
}
If this code is saved into the file mov23gp.m, we can create the executable command-line tool by issuing this command:
% cc -o mov23gp -g mov23gp.m -framework Foundation -framework QTKit
Movie Display
Command-line tools are useful enough, but chances are you're more interested in opening and displaying
QuickTime movies in a window. Let's see how to work with a QTMovieView object. For this, we'll need to use
Xcode and Interface Builder.
Launch Xcode; select "New Project..." from the File menu and then select "Cocoa Document-based Application",
as shown in Figure 4.
Figure 4: Creating a new multi-document project
The Assistant then prompts us for a project name and location; let's call our project "KitEez" and save it
in our home directory. Xcode then displays the project window shown in Figure 5. (I've opened some of the
disclosure triangles to reveal the contents of each group.)
Figure 5: The KitEez project window
The "Product" -- that is, our application -- is shown in red because it hasn't been built yet. We can build
the application by selecting Build (Command-B) from the Build menu. And we can run the application by
selecting Run (Command-R). If we do that, we'll see the document window shown in Figure 6.
Figure 6: The default document window
That's certainly not what we want; remember, we want to display QuickTime movies in this window, not a
short text string.
Modifying the Document Window Appearance
Let's open MyDocument.nib with Interface Builder. We can do this by double-clicking the entry in the
project window. Figure 7 shows the main window for this nib file.
Figure 7: The document nib file
Now let's remove the text from the main document window and replace it with a QTMovieView object. By
default, the QTKit palette item is not automatically loaded into the Palettes toolbar, so we need to choose
"Preferences..." and then select the Palettes pane; click "Add..." (Figure 8).
Figure 8: The Palettes pane
Navigate to /Developer/Extras/Palettes and select the QTKit.palette item, as shown in Figure 9.
Figure 9: The QTKit palette
At this point, the Palettes toolbar looks like the one in Figure 10.
Figure 10: The QTKit palette in the toolbar
Drag the QuickTime icon from the palette into the document window and resize it to fill the content area of
the window, as shown in Figure 11. In the Size pane of the QTMovieView Inspector palette, set the movie view
autosizing so that the movie view remains pegged to the size of the window.
Figure 11: The revised document window
Now, we need a way for our custom document class (MyDocument) to keep track of this new object that we've
added to the document window. We do this by adding an outlet to the document class. An outlet is essentially
just an instance variable that holds a reference to some object. We'll add the instance variable to our class
declaration in a moment, but we also need to add it to our nib file. Click on the Classes tab in the
MyDocument.nib window and navigate to the MyDocument class, as shown in Figure 12.
Figure 12: The Class hierarchy window
Now select the "Add Outlet to MyDocument" menu item in the Classes menu. Interface Builder will display the
Class Info window shown in Figure 13.
Figure 13: The Class Info window
Change the default outlet name "myOutlet" to "movieView".
So far, we've created a new outlet for the document class. Now we need to link the outlet with the movie
view that we added to the document window. Click on the Instances tab in the MyDocument.nib window; then hold
down the Control key, click on the File's Owner icon, and drag until the cursor is over the movie view in the
document window. When we release the mouse button, Interface Builder displays the window shown in Figure 14.
Figure 14: Connections of the File's Owner
Click the Connect button to make the desired connection. Notice that the Destination field updates to
indicate that movieView is connected to an object of type QTMovieView. For the moment, we are done modifying
the nib file. Let's save our work and quit Interface Builder.
Configuring the Application
Move back to Xcode. The first thing we need to do is add the QTKit framework to the project. Select "Add to
Project..." in the Project menu and then navigate to /System/Library/Frameworks; select QTKit.framework and
click OK and then Add.
At this point, we need to add an instance variable for the movieView outlet we created in Interface
Builder. Open the file MyDocument.h and add a movieView instance variable. Let's also add an instance variable
to keep track of the QTMovie object associated with the movie view. (This is not strictly necessary, as we can
always get the QTMovie object associated with a QTMovieView object by calling its movie method.) Also, import
the QTKit.h header file from the QTKit framework. MyDocument.h now looks like this:
#import <QTKit/QTKit.h>
@interface MyDocument : NSDocument
{
QTMovieView *movieView;
QTMovie *movie;
}
@end
Now select the KitEez target in the project window and bring up the inspector window by selecting "Get
Info" in the File menu. In the Properties pane, set the document types for the MyDocument class to include the
extension "mov" and the OS type "MooV". This will allow files with names ending in ".mov" or of file type
"MooV" to appear in the file-opening dialog box. (Later we'll see how to modify KitEez to allow the user to
select any kind of file openable by QuickTime, regardless of filename extension or file type.)
Opening Movie Files
When the user selects a file in the file-opening dialog box, our multi-document application creates a new
document window, based on the information in the nib file. When it has loaded the nib and initialized all the
objects specified therein, it calls our document's windowControllerDidLoadNib method. This is where we want to
assign the movie to the movie view.
By the time windowControllerDidLoadNib is called, an instance of the MyDocument class has already been
created. All the instance variables associated with outlets, and in particular movieView, are already set up
for us. It remains only for us to do any final configuration of the movie or view before the new document
window is displayed to the user.
As you know, we need to create a QTMovie object and assign it to the movie view. Once again we'll use the
initWithFile:error: method, which initializes a movie object with a QuickTime movie specified by a filename:
movie = [[QTMovie alloc] initWithFile:[self filename] error:nil];
We'll set the movie associated with the movie view like this:
[movieView setMovie:movie];
And we'll set the movie to be editable, just like we did earlier using setAttribute:forKey:.
Setting the Window Size
Before showing the new window to the user, we need to set the size of the window's content region to
exactly enclose the movie at its natural size. We can determine the natural size of the movie like this:
NSSize size = [[movie attributeForKey:QTMovieNaturalSizeAttribute] sizeValue];
As you can see, we get a movie attribute by calling the attributeForKey: method.
We can determine the appropriate content region size for a given movie size using the
windowContentSizeForMovieSize: method defined in Listing 2. Notice that we use the isControllerVisible method
on QTMovieView to see whether the controller bar is showing; if it is, we need to increase the height of the
window by the height of the movie controller bar.
Listing 2: Finding a content region size for a movie size
- (NSSize)windowContentSizeForMovieSize:(NSSize)size
{
NSSize contentSize = size;
if ([movieView isControllerVisible])
contentSize.height += [movieView controllerBarHeight];
if (contentSize.width == 0)
contentSize.width = [[movieView window] frame].size.width;
return contentSize;
}
Notice also that we look to see whether the size passed in has a width of 0; if it does, we set the width
of the window's content region to the current width of the window.
In windowControllerDidLoadNib, we'll call windowContentSizeForMovieSize: and set the size of the movie
window like this:
[[movieView window] setContentSize:[self windowContentSizeForMovieSize:size]];
There is one final task that we need to perform. Since our movie view entirely fills the content region of
the document window, we need to disable the resize box provided by NSWindow and enable the resize box
displayed by QTMovieView. We can do that with two calls to setShowsResizeIndicator::
[[movieView window] setShowsResizeIndicator:NO];
[movieView setShowsResizeIndicator:YES];
Listing 3 shows our complete implementation of windowControllerDidLoadNib.
Listing 3: Opening a movie window
- (void)windowControllerDidLoadNib:(NSWindowController *) aController
{
[super windowControllerDidLoadNib:aController];
if ([self fileName]) {
movie = [[QTMovie alloc] initWithFile:[self fileName] error:nil];
if (movie) {
// set the view's movie
[movieView setMovie:movie];
// make the movie editable
[movie setAttribute:[NSNumber numberWithBool:YES] forKey:QTMovieEditableAttribute];
// set the size of the movie window to exactly enclose the movie at its natural size
NSSize size = [[movie attributeForKey:QTMovieNaturalSizeAttribute] sizeValue];
[[movieView window] setContentSize:[self windowContentSizeForMovieSize:size]];
// use the QTKit's resize indicator, not NSWindow's
[[movieView window] setShowsResizeIndicator:NO];
[movieView setShowsResizeIndicator:YES];
}
}
}
If we build and run the application, we will be able to open and display QuickTime movies. The movies are
fully editable, either using items in the Edit menu or their command-key equivalents.
Setting Openable Document Types
The current limitation on KitEez is that it can open only files whose filename extension is ".mov" or that
are of HFS file type "MooV". We'd really like it to be able to open any kind of file the QuickTime can open.
We could of course add more and more file extensions and types to the application's property list; a better
solution, however, is to determine dynamically what files should appear in the file-opening dialog box. To do
that, we need to add an application delegate to the project. That delegate will override the openDocument: and
panel:shouldShowFilename: methods. In particular, we'll add the override method shown in Listing 4.
Listing 4: Specifying openable document types
- (BOOL)panel:(id)sender shouldShowFilename:(NSString *)filename
{
BOOL isDir = NO;
[[NSFileManager defaultManager] fileExistsAtPath:filename isDirectory:&isDir];
return isDir ? YES : [QTMovie canInitWithFile:filename];
}
The key element here is to use the QTMovie class method canInitWithFile:, which returns a value of type
BOOL that indicates whether the specified file can be used to initialize a QTMovie object. QTMovie also
supplies the class methods canInitWithURL:, canInitWithPasteboard:, and canInitWithDataReference:.
Conclusion
In this article, we've taken a preliminary first look at the QTKit framework introduced in QuickTime 7 on
both Tiger and Panther. We've seen how to use QTKit to build a simple command-line tool that manipulates
QuickTime movies, and we've seen how to create a multi-document Cocoa application that can open and display
QuickTime movies. In the next two articles, we'll continue investigating this important new framework. We'll
finish KitEez by allowing it to save edited movies, and we'll spend some time looking at more of the
capabilities offered by QTKit.
Tim Monroe is a member of the QuickTime engineering team at Apple. You can contact him at
monroe@mactech.com. The views expressed here are not necessarily shared by his employer.
