Threads
Volume Number: 21 (2005)
Issue Number: 8
Column Tag: Programming
Threads
Performing QTKit Operations On Threads
by Tim Monroe
In the previous three articles (in MacTech, May, June, and July 2005), we have investigated the
QTKit, Apple's new framework for accessing QuickTime capabilities in Cocoa applications and tools.
We have used this framework -- introduced in Tiger and also available in Panther with QuickTime 7 --
primarily to build a sample application that can open and display one or more QuickTime movie files
(or other media files openable as QuickTime movies) and that supports the standard suite of movie
editing and document management operations.
Introduction
The sample application that we have been developing, KitEez, performs really quite well. Files
open and display quickly, and cut-and-paste editing works as smoothly as could be expected. And, as
mentioned in one of those previous articles, QTKit-based drag-and-drop performance in most instances
greatly exceeds that provided by the standard movie controller component.
But, truth be told, we haven't really asked KitEez to handle any potentially slow operations.
Currently it can open only files selectable in the file-opening dialog box, which means that we're
not likely to see any real delay when opening a movie file. If KitEez were to support files
specified by remote URLs, we'd definitely see some slowdown in movie opening. Also, some operations
that we can perform using QuickTime functions -- like importing or exporting movies and large
pictures -- can take a considerable amount of time. If KitEez were to support those sorts of
operations, we'd want to make sure that the user was able to keep working while they are churning
away. That generally means that we'd need to learn how to perform QTKit operations on a background
thread.
In this article, we're going to take a look at threading and QTKit. Since QTKit is built on top
of QuickTime, it inherits whatever limitations exist in QuickTime with regard to being able to
perform operations on background threads. QuickTime has supported threaded operations since Mac OS X
10.3 and later, with QuickTime 6.4 and later. That is to say, it's now possible to move potentially
time-consuming operations to a background thread, thereby freeing up the main thread for user
interaction. Gone are the days when importing or exporting a large file invariably ties up your
QuickTime application.
To achieve this, we'll need to use a few new QuickTime functions to set up the QuickTime
environment on a background thread. We'll need to learn how to safely move QTKit objects between
threads, and we'll need to be careful about which operations are performed on a background thread.
But with these three issues covered, we should be able to provide a far better user experience than
is possible in a single-threaded application.
Before we begin, a couple of provisos. It would be good if you were already familiar with
application threading in general and with Cocoa's NSThread class in particular. If not, don't worry;
you should be able to pick up what you need along the way. Just don't expect a detailed discussion
of threading models here. Also, I will be focusing on invoking QTKit and other Cocoa methods on
background threads; see the documents mentioned in the References section for more complete
information about threading in QuickTime generally.
Opening Movies Revisited
As mentioned above, our sample application KitEez currently can open only files selectable in the
file-opening dialog box, which generally means that it will be opening only local files. In the
previous article, we saw that we can also use the initWithURL:error: method to open a remote file
specified by a URL. To elicit a URL from the user, we might display a dialog box like the one in
Figure 1. (Adding this dialog box to KitEez is left as an exercise for the reader.)
Figure 1: A URL dialog
box
As you know, initWithURL:error: operates asynchronously, just like all the QTKit movie-opening
methods; that is to say, it returns almost immediately, so that our application can continue
processing while the movie data loads. So we might be tempted to immediately assign the QTMovie
object to the movie view in our document window, like this:
if ([QTMovie canInitWithURL:url]) {
movie = [[QTMovie alloc] initWithURL:url error:nil];
if (movie)
[movieView setMovie:movie];
}
This would work fine, from the standpoint of having QuickTime download the movie data without
further intervention from our application. Experience has shown, however, that it's best to defer
the setMovie: call until a sufficient amount of movie data has been downloaded. To do that, we can
install a notification handler for the QTMovieLoadStateDidChangeNotification notification, like
this:
if ([QTMovie canInitWithURL:url]) {
movie = [[QTMovie alloc] initWithURL:url error:nil];
if (movie)
[[NSNotificationCenter defaultCenter] addObserver:self
selector:@selector(loadStateChanged:)
name:QTMovieLoadStateDidChangeNotification
object:movie];
}
Whenever the load state of the downloading movie changes, the loadStateChanged: method will be
called. (See "Loaded" in MacTech, September 2002 for a complete discussion of movie load states.)
One easy implementation of loadStateChanged: is shown in Listing 1.
Listing 1: Handling load state-changed notifications
- (void)loadStateChanged:(NSNotification *)notification
{
if ([[movie attributeForKey:QTMovieLoadStateAttribute]
longValue] >= kMovieLoadStatePlayable) {
[[NSNotificationCenter defaultCenter]
removeObserver:self
name:QTMovieLoadStateDidChangeNotification
object:movie];
[movieView setMovie:movie];
[movie release];
[[movieView movie] play];
}
}
As you can see, we wait until the load state of the movie reaches the kMovieLoadStatePlayable
level, at which time we remove the notification listener for the specified notification, call
setMovie:, release our QTMovie object (since the movie view will retain it), and then start the
movie playing. The kMovieLoadStatePlayable constant is defined in the QuickTime header file
Movies.h. Here is the complete set of load state constants:
enum {
kMovieLoadStateError = -1L,
kMovieLoadStateLoading = 1000,
kMovieLoadStateLoaded = 2000,
kMovieLoadStatePlayable = 10000,
kMovieLoadStatePlaythroughOK = 20000,
kMovieLoadStateComplete = 100000L
};
We need to defer the call to setMovie: until the movie is playable to work around a bug in the
first release of QTKit. If we didn't do this, and instead just called setMovie: immediately after
the call to initWithURL:error:, we would find that the movie view would not automatically redraw
itself once any video data had arrived. The workaround is simple enough and indeed provides an easy
way for us to start the movie playing at an appropriate time.
It's also useful to know how to get the load state of a QTMovie object when we want to add
importing and exporting support to our applications. The reason is simple: whenever we want to
export or flatten or save a movie, we need to have all of its movie and media data on hand. The
QTMovie method that we use to export or flatten a movie, writeToFile:withAttributes:, internally
checks the movie's load state and returns NO if it's not at least kMovieLoadStateComplete. But we
might need to make this check ourselves, when adjusting some menu items. If not all the movie and
media data is available, then for instance the Export... menu item should not be enabled. Listing 2
shows a segment of a document class' override of the validateMenuItem: method.
Listing 2: Enabling menu items
- (BOOL)validateMenuItem:(NSMenuItem *)menuItem
{
BOOL valid = NO;
SEL action;
action = [menuItem action];
if (action == @selector(doExport:))
valid = ([[movieView movie]
attributeForKey:QTMovieLoadStateAttribute] longValue]
>= kMovieLoadStateComplete);
// other lines omitted...
return valid;
}
Keep in mind that we have not yet reached a position where we need to move any processing out of
the main thread and into a secondary or background thread. The periodic movie tasking that is
required to keep a remote movie steadily downloading happens automatically on the main thread and
does not generally consume so much processor time that the responsiveness of the main thread is
adversely impacted. So, although opening a movie specified by a URL may take a significant amount of
time, QuickTime (and hence QTKit) already knows how to do that asynchronously without blocking the
main thread.
Importing and Exporting
When we move into the realm of movie importing and exporting -- that is, converting potentially
large amounts of data -- we cross an important threshold. Although it is certainly possible to export
a movie by calling writeToFile:withAttributes: on the main thread, it isn't really advisable, since
the call would execute synchronously. Listing 3 shows how not to define the doExport: method
referenced in Listing 2.
Listing 3: Exporting a movie as 3GPP
- (IBAction)doExport:(id)sender
{
NSDictionary *dict = [NSDictionary
dictionaryWithObjectsAndKeys:
[NSNumber numberWithBool:YES], QTMovieExport,
[NSNumber numberWithLong:kQTFileType3GPP],
QTMovieExportType, nil];
[[movieView movie] writeToFile:@"/tmp/sample.3gp"
withAttributes:dict];
}
If the movie is very large, this method could take quite a while to complete. During that time,
the user would be unable to do anything with our application except move windows around. Not very
exciting.
A slightly better solution involves using the
movie:shouldContinueOperation:withPhase:atPercent:withAttributes: delegate method described briefly
in the previous article. As I mentioned, this is a wrapper around QuickTime's movie progress
function, which we have used in earlier articles to display a dialog box showing the progress of the
export and to allow the user to cancel the operation. Figure 2 shows the sheet we'll display from
within that delegate method.
Figure 2: A progress
sheet
We could implement this delegate method as shown in Listing 4.
Listing 4: Displaying a cancelable progress sheet
- (BOOL)movie:(QTMovie *)movie
shouldContinueOperation:(NSString *)op
withPhase:(QTMovieOperationPhase)phase
atPercent:(NSNumber *)percent
withAttributes:(NSDictionary *)attributes
{
OSErr err = noErr;
NSEvent *event;
double percentDone = [percent doubleValue] * 100.0;
switch (phase) {
case QTMovieOperationBeginPhase:
// set up the progress panel
[progressText setStringValue:op];
[progressBar setDoubleValue:0];
// show the progress sheet
[NSApp beginSheet:progressPanel
modalForWindow:[movieView window] modalDelegate:nil
didEndSelector:nil contextInfo:nil];
break;
case QTMovieOperationUpdatePercentPhase:
// update the percent done
[progressBar setDoubleValue:percentDone];
[progressBar display];
break;
case QTMovieOperationEndPhase:
[NSApp endSheet:progressPanel];
[progressPanel close];
break;
}
// cancel (if requested)
event = [progressPanel
nextEventMatchingMask:NSLeftMouseUpMask
untilDate:[NSDate distantPast]
inMode:NSDefaultRunLoopMode dequeue:YES];
if (event && NSPointInRect([event locationInWindow],
[cancelButton frame])) {
[cancelButton performClick:self];
err = userCanceledErr;
}
return (err == noErr);
}
This is certainly a better solution than having no sheet at all, but it's really not
satisfactory. Just distracting the user with a progress bar is not going to make the export go any
faster or be any less synchronous. And the manner in which we check for clicks on the Cancel button
is not really very good, even if it is the best we can hope for in a non-threaded application.
Threaded Exporting
So we really do need to move to a multithreaded application if we want to be able to provide
acceptable responsiveness in our application's user interface while performing potentially lengthy
operations like exporting a movie. As we'll see, spawning a thread to execute some code in a Cocoa
application is as easy as calling the NSThread method detachNewThreadSelector:toTarget:withObject:.
The complexities we shall encounter arise from the fact that QuickTime was not originally written to
be thread-safe, and making it work in a threaded environment requires some assistance from the
application developer. For some of the theory, consult the documents listed at the end of this
article, particularly the Tech Note on threading QuickTime applications. For the moment, we will
content ourselves with the practical implications of that theory. In summary, they are these:
(1)Before any QuickTime APIs (including QTKit methods) can be called on a background
thread, the function EnterMoviesOnThread must be called on that thread.
(2) After all QuickTime APIs (including QTKit methods) have been called on a background
thread, the function ExitMoviesOnThread must be called on that thread.
(3) A movie created on one thread that is to be accessed on some other thread must first
be detached from the first thread (by calling DetachMovieFromCurrentThread) and attached to that
other thread (by calling AttachMovieToCurrentThread).
(4)QuickTime APIs (including QTKit methods) executing on a background thread may
internally attempt to instantiate components that are not thread-safe; when that happens, the result
code componentNotThreadSafeErr (-2098) will be returned. In that case, you might want to retry the
operation on the main thread.
Implication (4) has an important corollary. Recall from the first article on QTKit that a QTMovie
object is a Cocoa representation of a QuickTime movie and a QuickTime movie controller. That is to
say, a QTMovie object is associated with a Movie instance and a MovieController instance. Currently,
no movie controller components are thread-safe. This means:
(5) All QTMovie objects must be created on the main thread.
In theory, creating a QTMovie object on the main thread and then migrating it to a background
thread is no less dangerous than creating it on a background thread. Experience has shown, however,
that it appears to be safe to call QTMovie methods on a QTMovie object that has been thus migrated.
At any rate, this is the best we can do given the current state of the QTKit and the underlying
movie controller components.
Transferring Movies Between Threads
To see how these implications play out in practice, let's walk through some code that exports a
QuickTime movie on a background thread. If the application's Export... menu item is connected to the
doExport: method, we can start the export process by calling
detachNewThreadSelector:toTarget:withObject:, passing the selector of the application's
doExportOnThread: method. The doExport: method is shown in Listing 5.
Listing 5: Starting a background export
- (IBAction)doExport:(id)sender
{
NSSavePanel *savePanel = [NSSavePanel savePanel];
Movie qtMovie = [[movieView movie] quickTimeMovie];
int result;
OSErr err = noErr;
result = [savePanel runModal];
if (result == NSOKButton) {
SEL sel = @selector(doExportOnThread:);
[[movieView movie] stop];
err = DetachMovieFromCurrentThread(qtMovie);
if (err)
return;
// show the progress sheet
[NSApp beginSheet:progressPanel
modalForWindow:[movieView window] modalDelegate:nil
didEndSelector:nil contextInfo:nil];
[NSThread detachNewThreadSelector:sel toTarget:self
withObject:[savePanel filename]];
}
}
There is nothing particularly noteworthy here except for the call to
DetachMovieFromCurrentThread. The doExport: method is called on the main thread, so we need to
explicitly detach the Movie associated with the QTMovie object from the main thread so that it can
later be attached to a background thread. Notice also that we have moved the call to display the
progress sheet out of the delegate method and into the doExport: method.
Accessing Movies on Background Threads
Now let's start building the doExportOnThread: method. Its declaration should look like this:
- (IBAction)doExportOnThread:(id)sender;
Here, the sender object is in fact the name of the file into which the exported movie is to be
written (as you can see from Listing 5). Since doExportOnThread: is to be run on a background
thread, it needs to create and release an autorelease pool, and it needs to call EnterMoviesOnThread
and ExitMoviesOnThread. Listing 6 shows the basic skeleton of a method that is to support QuickTime
calls on a background thread.
Listing 6: Exporting a movie in the background (skeleton version)
- (IBAction)doExportOnThread:(id)sender
{
NSAutoreleasePool* pool = [[NSAutoreleasePool alloc]
init];
OSErr err = EnterMoviesOnThread(0);
if (err)
goto bail;
err = AttachMovieToCurrentThread([movie quickTimeMovie]);
if (err)
goto bail;
// QuickTime/QTKit calls can go in here....
DetachMovieFromCurrentThread([movie quickTimeMovie]);
ExitMoviesOnThread();
bail:
[pool release];
[NSThread exit];
}
The EnterMoviesOnThread function is declared like this:
OSErr EnterMoviesOnThread (UInt32 inFlags);
Currently only one bit in the inFlags parameter is defined, namely
kQTEnterMoviesFlagDontSetComponentsThreadMode. Setting this flag forces no change to be made to the
Component Manager threading mode. By default, EnterMoviesOnThread automatically sets the Component
Manager threading mode to kCSAcceptThreadSafeComponentsOnlyMode, which indicates that only
thread-safe components shall be allowed. Since this is the mode we desire, we'll pass 0 when we call
EnterMoviesOnThread.
When we are finished using QuickTime API calls on a particular background thread, we need to call
ExitMoviesOnThread, which takes no parameters. Each call to EnterMoviesOnThread must be balanced by
a call to ExitMoviesOnThread.
The main thing we need to do is add some code that exports the specified movie into the filename
indicated by the sender parameter. That code might look like this:
NSDictionary *dict = [NSDictionary
dictionaryWithObjectsAndKeys:
[NSNumber numberWithInt:1], QTMovieExport,
[NSNumber numberWithInt:kQTFileType3GPP],
QTMovieExportType, nil];
[movie writeToFile:sender withAttributes:dict];
Once the writeToFile:withAttributes: method completes, we need to make sure that the progress
panel is removed and that the movie is transferred back to the main thread. We can do that by adding
one more line of code, just before the bail label:
[self performSelectorOnMainThread:
@selector(finishedExporting) withObject:nil
waitUntilDone:YES];
Listing 7 shows our implementation of the finishedExporting method.
Listing 7: Cleaning up after an export operation
-(void)finishedExporting
{
[NSApp endSheet:progressPanel];
[progressPanel close];
AttachMovieToCurrentThread([movie quickTimeMovie]);
}
And so we are done building the doExportOnThread: method and the methods it calls.
Handling the Cancel Button
One final task awaits us, namely displaying the progress sheet, updating its progress bar, and
handling clicks on the Cancel button. It turns out that we already have most of the code we need at
hand, in the form of our movie:shouldContinueOperation:withPhase:atPercent:withAttributes: delegate
method. The first thing we need to change is the cheesy way in which we detect clicks on the Cancel
button. In the nib file, we configure that button to initiate the doCancel: action, implemented in
Listing 8.
Listing 8: Handling clicks on the Cancel button
- (IBAction)doCancel:(id)sender
{
cancel = YES;
}
Then, in the delegate method, we toss all the code that looks for mouse-up events in the button
and replace it with this easy test:
if (cancel)
err = userCanceledErr;
Also, we cannot set the values of the text field and the progress bar from within the delegate
method, because this delegate method wraps a movie progress function, which is called on the same
thread as the export operation -- that is, on a background thread. In general, a background thread
must never directly alter the application's user interface. What we need to do is have the delegate
method use the NSObject method performSelectorOnMainThread:withObject:waitUntilDone:, as we did
earlier. So we'll rework the various case blocks like this:
case QTMovieOperationUpdatePercentPhase:
// update the percent done
[self updateProgress:op toNumber:percent];
break;
Listing 9 shows our definition of updateProgress:toNumber:.
Listing 9: Sending UI updates to the main thread
- (void)updateProgress:(NSString*)msg
toNumber:(NSNumber*)value
{
NSDictionary* dict = [NSDictionary
dictionaryWithObjectsAndKeys:msg, @"msg",
value, @"value", nil];
[self performSelectorOnMainThread:
@selector(updateProgressInMainThread:)
withObject:dict waitUntilDone:NO];
return ;
}
Finally, Listing 10 shows the code we run on the main thread to update the items in the progress
panel.
Listing 10: Updating the progress panel
- (void)updateProgressInMainThread:(NSDictionary*)dict
{
NSString* msg = [dict objectForKey:@"msg"];
double value = [[dict objectForKey:@"value"] doubleValue]
* 100.0;
[progressText setStringValue:msg];
[progressBar setDoubleValue:value];
}
For completeness, let's take a last look at the revised version of the delegate method we are
using to drive the progress updating (Listing 11).
Listing 11: Displaying a cancelable progress sheet (revised)
- (BOOL)movie:(QTMovie *)movie
shouldContinueOperation:(NSString *)op
withPhase:(QTMovieOperationPhase)phase
atPercent:(NSNumber *)percent
withAttributes:(NSDictionary *)attributes
{
OSErr err = noErr;
switch (phase) {
case QTMovieOperationBeginPhase:
// set up the progress panel
[self updateProgress:op toNumber:
[NSNumber numberWithLong:0]];
break;
case QTMovieOperationUpdatePercentPhase:
// update the percent done
[self updateProgress:op toNumber:percent];
break;
case QTMovieOperationEndPhase:
break;
}
if (cancel)
err = userCanceledErr;
return (err == noErr);
}
Conclusion
In this article, we've taken a look at executing QTKit methods on secondary threads, in an effort
to offload lengthy operations from the main thread and thus improve the responsiveness of our
application. In particular, we've seen how to export a large movie without blocking the playback of
other movies that we might have open and without preventing the user from opening other movies. The
basic rules we need to adhere to are relatively simple: (1) make sure to properly initialize and
deinitialize the QuickTime environment on secondary threads (by calling EnterMoviesOnThread and
ExitMoviesOnThread); (2) make sure to detach a movie from one thread and attach it to another thread
if you need to operate on it in multiple threads (by calling DetachMovieFromCurrentThread and
AttachMovieToCurrentThread); and (3) make sure to perform any user interface processing on the main
thread.
Credits and References
A few of the routines used here are based on code by Michael B. Johnson. A more exhaustive
discussion of threading in QuickTime can be found in Technical Note TN2125, "Thread-safe programming
in QuickTime", available at http://developer.apple.com/technotes/tn/
tn2125.html. You can also find a discussion of the QuickTime threading APIs in the QuickTime 6.4
API Reference.
Tim Monroe is a member of the QuickTime engineering team at Apple. You can contact him
at monroe@mactech.com. The views expressed here are not
necessarily shared by his employer.
