TweetFollow Us on Twitter

December 94 - Somewhere in QuickTime: Supporting Text Tracks in Your Application

Somewhere in QuickTime: Supporting Text Tracks in Your Application

Nick Thompson

Text media tracks were introduced with QuickTime version 1.5 and have been further improved in QuickTime 2.0 with the addition of calls for more powerful searching and a new facility called burnt text. The big news is that text tracks are now supported in QuickTime for Windows version 2.0, which makes text tracks a cross-platform solution. If you're developing content-based products that need to be cross-platform, you'll want to take a look at text tracks.

Text tracks give you the ability to embed text in a QuickTime movie, which is particularly useful if you're aiming your product at international markets or at people with hearing impairments (you can subtitle your movie) or if you want to enable your users to perform searches (by including the script of a play or movie in a text track, for instance, you can make it easy for users to find a particular scene by searching for a key piece of dialog). The possibilities for adding value to your QuickTime application and content with text tracks are limited only by your imagination.

This column shows you how to add text track support to your QuickTime application, including support for searching and editing. The sample program QTTextSample on this issue's CD demonstrates what's involved. This small application plays a movie controller-based movie in a window and offers users the ability to search for a particular sequence of characters, or edit the text, in the text track.

Text tracks are handled by the text media handler, which is documented in the "Movie Toolbox" chapter of Inside Macintosh: QuickTime. The text media handler's functionality includes the following:

  • Searching for text, using FindNextText. With QuickTime 2.0, a new routine for text searching, MovieSearchText, can be used. This call is also available with QuickTime 2.0 for Windows.

  • Adding plain or styled text, using AddTextSample or AddTESample. Both of these calls allow you to define additional text properties, such as scrolling and highlighting.
Searching is something that all applications that support text tracks will want to offer the user, while editing text is likely to be something that only a few specialized applications will want to provide. Editing of text can be accomplished with Movie Toolbox routines, as discussed later in this column.

FIRST THINGS FIRST

Your QuickTime application needs to do a few basic things at the outset, as QTTextSample demonstrates. These include checking for the QuickTime version number, growing your heap as large as possible, and checking return codes.

QuickTime version number. A Macintosh application that supports text tracks requires QuickTime version 1.5 or later; a Windows application that supports text tracks requires QuickTime 2.0 for Windows. On Macintosh platforms you can use the Gestalt selector gestaltQuickTime to check that the version number returned in the high-order word is greater than or equal to the required version (0x0150 for QuickTime 1.5; 0x0200 for QuickTime 2.0).

Our sample program bails out if the system isn't running QuickTime version 1.5 or later. If your application uses calls provided by later releases of QuickTime but you also want it to run on earlier versions, you should check for the version number and selectively enable your application's functionality accordingly.

Heap size. As discussed in the Somewhere in QuickTime column in develop Issue 13 ("Top 10 QuickTime Tips" by John Wang), you need to ensure that you grow your heap as large as possible in your initialization code by calling the MaxApplZone routine. QuickTime needs to use a lot of handle-based data structures, so you also need to ensure that there are enough master pointers allocated in the base of your heap. To do that, you should call MoreMasters a number of times (you can tell how many times by examining your heap while the application is running). If you don't do this, your heap may become fragmented, which in turn may cause certain QuickTime routines to fail due to lack of memory. This is a general Macintosh programming issue that you should be aware of for all applications.

Return codes. Finally, always check those return codes, and handle errors as required. Most QuickTime routines return a status code of type OSErr; if this isn't equal to noErr, you have problems. QuickTime is always trying to tell you something -- listen and your life will be more complete!

THE USER INTERFACE

The user interface for the sample application is pretty basic. Figure 1 shows the application window. As you can see, the text for the movie is repeated below the movie so that it can be edited. Buttons offer the user the options of finding specific text or updating the text with editing changes. (The Update Text button is dimmed unless the text in the text box has been modified.)

With text track selected

With movie selected

Figure 1. The application window for QTTextSample

Because the user can potentially edit two items -- the movie's text track or the movie itself -- the application needs to keep track of what the user selected last (either text or the movie) and highlight it in some way. As shown in Figure 1, when the text track is selected in our sample application, the text below the movie has a black box around it; when the movie is selected, the movie frame has a black box around it. When the window is inactive (for example, when you switch applications), the box surrounding the active item is rendered in gray.

TRICKS FOR EASY ACCESS

In the sample code, the movie controller is stored in a record referenced by a handle stored in the window's refCon, along with a few other bits and pieces related to the movie and the text for a window. This gives us easy access to both the movie controller and its associated movie:
aDocHdl = (DocHandle)GetWRefCon(theMovieWindow);
aController = (**aDocHdl).myController;
aMovie = MCGetMovie(aController);
When we need to locate the first track of a particular type (in our case a text track) in a movie, we can use the following handy utility routine:
Track GetFirstTrackOfType (Movie aMovie,
                              OSType trackType)
{
   Track      theTrack = nil; 
   OSType     mediaType;
   short      trackCount, index;
   
   trackCount = GetMovieTrackCount(aMovie);
   for (index=1; index <= trackCount; index++) {
      Track t = GetMovieIndTrack(aMovie, index);
      GetMediaHandlerDescription(GetTrackMedia(t),
                  &mediaType, nil, nil);
      if (mediaType == trackType) {
         theTrack = t;
         break;
      }
   }
   return theTrack;
}
A new function, GetMovieIndTrackType, was introduced with QuickTime 2.0 for both Macintosh and Windows. GetMovieIndTrackType provides an easy way to iterate through all tracks in a movie that are either of a given media type or that support a particular media characteristic. Documentation for this, and the other new QuickTime calls, can be found in the QuickTime 2.0 Developer's Kit which is available from Apple Developer Catalog

HANDLING TEXT TRACKS

By default, QuickTime displays the text for an enabled text track. We want to be able to exercise more control over the format and display of the text track and to edit the text embedded in the movie. Thus, we need to extract the text from the track and stuff it into a TextEdit record.

Our application needs to be able to access the text for a particular frame as it's displayed. We do this by defining a text-handling procedure (or textProc for short) with the following format:

pascal OSErr MyTextProc (Handle thisText, 
         Movie thisMovie, short *displayFlag, 
         long refCon)
The text is passed in the handle. To access the text, we need to determine the length of the text and store it somewhere.
// This yields the actual size of the text.
textSize = *(short*)(*thisText);
// This yields a pointer to the text.
textSamplePtr = 
            (char*)(*thisText + sizeof(short)); 
The style data for a text track is stored in one of two places. Information about the default text style, together with other items of interest (such as the background color), is stored in a text description handle (see page 2-291 of Inside Macintosh: QuickTime). Additional information may be supplied at the end of the handle passed to the textProc, in the form of 'styl' atoms (see page 2-290 of Inside Macintosh: QuickTime). Our sample code demonstrates how to access the style information. To get the text description, you need to call GetMediaSample in the textProc, as shown in Listing 1. You need to parse the handle passed into your textProc to see if additional information is supplied; the sample code illustrates how to do this.

Listing 1. Getting the text description

theErr = GetMediaSample(aMedia, myData, nil, nil, mediaCurrentTime,
         nil, sampleDescriptionH, nil, nil, nil, nil);
...
if (sampleDescriptionH) {
   scrapHdl = (StScrpHandle)NewHandle(sizeof(StScrpRec));
   if (scrapHdl == nil)
      CheckError(MemError(), 
         "\pCouldn't allocate memory for the style table");
   (**scrapHdl).scrpNStyles = 1;
   (**scrapHdl).scrpStyleTab[0] =
      (**((TextDescriptionHandle)sampleDescriptionH)).defaultStyle;

   // Delete the previous contents of the TextEditHandle.
   TESetSelect(0, (**myDocTEH).teLength, myDocTEH);
   TEDelete(myDocTEH);

   // Insert the new text.
   TEStylInsert(textPtr, textSize, scrapHdl, myDocTEH);

   DisposeHandle((Handle)scrapHdl);
}
else TESetText(textPtr, textSize, myDocTEH);
We can control whether QuickTime also displays the text, by returning a value in the displayFlag parameter. For example, if we want the default display, we set it in the following way:
*displayFlag = txtProcDefaultDisplay;
Other flags are available either to suppress output or to ensure that QuickTime always displays the text track, regardless of the settings saved in the movie. Check page 2-364 of Inside Macintosh: QuickTime for more details.

In order for the textProc we've defined to get called, we need to tell QuickTime about it. This is easily accomplished with the SetMovieTextHandler routine (shown in Listing 2), which uses the utility routine described earlier to get the first text track.

Listing 2. The SetMovieTextHandler routine

OSErr SetMovieTextHandler (WindowPtr aWindow)
{
   MediaHandler        aMediaHandler;
   MovieController     aController;
   Movie               aMovie;
   Track               aTrack;
   DocumentHandle      aDocHdl;

   aDocHdl = (DocumentHandle)GetWRefCon(aWindow);
   aController = (**aDocHdl).myController;
   if (aController != nil) {
      aMovie = MCGetMovie(aController);

      // If there's a text track in the movie, set the textProc.
      if (aMovie != nil && (aTrack = GetFirstTrackOfType
             (aMovie, TextMediaType)) != nil) {
         aMediaHandler = GetMediaHandler(GetTrackMedia(aTrack));
         if (aMediaHandler != nil) 
            SetTextProc(aMediaHandler, NewTextMediaProc(MyTextProc),
               (long)aWindow);
      }
   }
   return GetMoviesError();
}

SEARCHING FOR TEXT

One feature that should be provided in movies that have embedded text is the ability to search for words. Consider a scenario where you're providing an interactive learning experience. The video track of your QuickTime movie contains a play, and the text track contains its script. Students can search for a particular scene just by searching for a few words. If the play were Shakespeare's Julius Caesar, for example, searching for "Lend me your ears" would find Marc Antony's speech at Caesar's funeral.

Searching for text in a movie is a straightforward operation using QuickTime's FindNextText routine (which is described on page 2-298 of Inside Macintosh: QuickTime). You can control the way this routine works by passing in the following flags:

  • findTextWrapAround -- wraps the search around at the end or start of the movie and continues searching for the text

  • findTextReverseSearch -- searches backward

  • findTextCaseSensitive -- makes the search case sensitive
Under QuickTime 1.5 or 1.6, however, you shouldn't use all three of these flags together in the same call; if you do, a bug will cause a bus error. You can work around this by manually implementing a wrapped search. This was fixed in QuickTime 2.0.

The sample code illustrates the use of FindNextText in the DoSearchForStringInMovieWindow routine. This routine gets the movie controller and its associated movie from the movie window. We pass in the text to search for, the direction to search in, and whether to wrap the search. The sample code shows one way of doing this with a simple dialog.

The new routine MovieSearchText, which was added to the Movie Toolbox in QuickTime 2.0 and in QuickTime 2.0 for Windows, also aids in searching for text in a movie. It can search any track that supports the text characteristic. (For a track to support the text characteristic, it must implement the FindNextText and HiliteTextSample calls as defined in the Movies.h file.) MovieSearchText is defined like this:

pascal OSErr MovieSearchText(Movie theMovie, 
   Ptr text, long size, long searchFlags, 
   Track *searchTrack, TimeValue *searchTime,
   long *searchOffset);
In this definition, theMovie is the movie to search, text is a pointer to a block of text that contains the search string, and size is the length of the search string in bytes. The other parameters are as follows:
  • searchFlags is a combination of findText flags as defined for media handlers that support text and searchText flags that manage the higher-level searching operation.

  • searchTrack is a pointer to the first track to search (or the only track, if searchTextOneTrackOnly is set in searchFlags). If the text is found, searchTrack will be updated to point to the track in which the text was found. If nil is passed in for searchTrack or if it points to nil, the search will start from the first track in the movie.

  • searchTime is a pointer to the movie time at which to start the current search. If the text is found, searchTime will be updated to reflect the movie time at which the text was found. If nil is passed in for searchTime or if it points to -1, the current movie time will be used.

  • searchOffset is a pointer to the offset within the text sample (as defined by searchTrack and searchTime) in which to start the search. If the text is found, searchOffset will be updated to reflect the offset into the text sample where the text was found. If nil is passed in for this parameter, an offset value of 0 will be used.
If MovieSearchText doesn't succeed in finding the search string because either there were no text tracks in the movie or the text simply wasn't found, an error value is returned.

EDITING TEXT

While the text media handler provides routines to add and delete text track segments, it doesn't provide routines to edit the text contained in a text track. Most applications won't need to edit text, but in case you're interested, this section looks at the Movie Toolbox routines involved.

The DoUpdateText routine in the sample code shows how the user can edit the text contained in a movie's text track. The steps involved in this process are listed below, and the code to accomplish these steps is shown in Listing 3. Note that error checking isn't included in this simplified version of the sample code; you'll find the complete code on this issue's CD.

    1. Determine which text track to edit.

    2. Determine the segment of the track to be edited. To do this we need to find the start time and duration of the sample we want to edit.

    3. Delete the existing text using the start time and duration we've determined.

    4. Add the text from the TextEdit handle into the media, using the QuickTime routine AddTESample (described on page 2-295 of Inside Macintosh: QuickTime). Then call the InsertMediaIntoTrack routine to insert the media we just created into the track.

Like all movie editing operations, editing text will cause the movie to become fragmented. You should ensure that the final production version of your movie is flattened; this will remove any fragmentation introduced by editing.

Listing 3. The DoUpdateText routine, simplified version

// Step 1:

// Get the text track; remember to check that it's not nil.
aTrack = GetFirstTrackOfType(aMovie, TextMediaType);
...

// Step 2:

// Get the media time of the current sample.
mediaCurrentTime = TrackTimeToMediaTime(currentTime, aTrack);
...
// Get detailed information on start and duration of the current
// sample (this is used later).
MediaTimeToSampleNum(aMedia, mediaCurrentTime, &mediaSampleIndex,
   &mediaSampleStartTime, &mediaSampleDuration);
...
// Look back and find where this text starts.
theErr = GetTrackNextInterestingTime(aTrack,
         nextTimeMediaSample | nextTimeEdgeOK, currentTime,
         -kFix1, &interestingTime, nil);

currentTime = interestingTime;

// Determine the duration of this sample.
theErr = GetTrackNextInterestingTime(aTrack,
         nextTimeMediaSample | nextTimeEdgeOK, currentTime,
         kFix1, &interestingTime, &theDuration);
...

// Step 3:

// Tell the media that we're about to edit stuff.
theErr = BeginMediaEdits(aMedia);
...
// Delete whatever was there before.
theErr = DeleteTrackSegment(aTrack, interestingTime, theDuration);
...

// Step 4:

// Write out the new data to the media.
theErr = AddTESample(aMediaHandler, aTEH,
         (RGBColor *)&theTextColor, teFlushDefault, nil,
         dfClipToTextBox, 0, 0, 0, nil, mediaSampleDuration,
         &sampleTime);
...
// Insert the new media into the track.
theErr = InsertMediaIntoTrack(aTrack, interestingTime, sampleTime,
    mediaSampleDuration, kFix1);
...
theErr = EndMediaEdits(aMedia);
...

NEW IN QUICKTIME 2.0: BURNT TEXT

QuickTime 1.6 introduced the capability for applications to apply special effects to the text in text tracks, notably antialiased text and drop shadows. Antialiased text is generally easier to read and looks more attractive than fonts rendered in the normal manner. However, antialiasing text takes time, and the performance penalty that's incurred playing movies with antialiased text tracks limits their usefulness.

QuickTime 2.0 allows applications to prerender text tracks, with a new facility called burnt text. Burnt text not only incurs less of a performance penalty than antialiased text but also has the advantage that a font doesn't need to be installed on the target machine in order to be rendered correctly. Applications that want to take advantage of this facility need to write data to the movie file in the form of a number of new atoms; for information on these additional atoms, see the file Text Imaging in QuickTime 2.0, accompanying this column on the CD.

THAT'S ALL THERE IS TO IT

Adding text track support to QuickTime applications really makes sense. With just a few lines of code, you can add a great deal of functionality. Most applications that use text tracks won't need to support editing the text, but it's a good idea to support searching because it provides an easy and powerful way of indexing into a movie containing text tracks.

Take a look at the sample code on the CD. It will help you get started with adding basic searching to your applications and (if required) with more advanced text track features, such as editing. Have fun!

NICK THOMPSON (AppleLink NICKT) found his first job in a surfboard factory, gluing wetsuits together. Then he scammed a job finishing custom surfboards. Somewhere along the way he learned how to program, and he's been riding that wave ever since. Last summer he snagged a job at Apple in Developer Technical Support and moved from London to California. Now he dresses up in a neoprene seal suit on weekends and goes shark fishing in the cold Pacific, armed only with a surfboard. (The glue from his first job must have affected his brain.)

Thanks to Ken Doyle, C. K. Haun, Peter Hoddie, Don Johnson, and John Wang for reviewing this column.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

How to get coins faster in Rodeo Stamped...
There comes a time in a cowboy or cowgirl's life when all the riding and lassoing skills in the world aren't enough. You're going to need some cold, hard cash to keep your sky zoo expanding in Rodeo Stampede. [Read more] | Read more »
How to out-do Cam Newton in Can You Dab?
The thing about dance crazes is that you're never really sure when they've run their course. Take the Dab, for instance. Propelled by its adoption as the touchdown celebration of choice for Carolina Panthers quarterback Cam Newton, the Dab seemed... | Read more »
Artik Games releases Splashy Cats for An...
Splashy Cats had us hooked from the title alone, and when we found out the game was literally just zig-zagging one of our favourite pop-culture references, guised as a playable cat character, down a river – our appetites were whetted to say the... | Read more »
Battle Cars (Games)
Battle Cars 1.1 Device: iOS Universal Category: Games Price: $1.99, Version: 1.1 (iTunes) Description: Welcome to the world of Battle Cars. Battle Cars is a classic arcade top-down racing game with fast mini cars and funny weapons to... | Read more »
How to get started with live.ly
One could be forgiven for thinking that there are already plenty of streaming video apps out there. It's just that the App Store charts would insist that you're mistaken. [Read more] | Read more »
Rodeo Stampede: Guide to all Savannah an...
A "gotta catch 'em all" joke seems appropriate here, even though we're talking animals in Rodeo Stampede and not pocket monsters. By now you've probably had plenty of rides, tamed some animals and built yourself a pretty nice zoo | Read more »
Is there cross-platform play in slither....
So you've sunken plenty of hours into crawling around in slither.io on your iPhone or iPad. You've got your stories of tragedy and triumph, the times you coiled four snakes at one time balanced out by the others when you had a length of more than... | Read more »
Rodeo Stampede guide to running a better...
In Rodeo Stampede, honing your skills so you can jump from animal to animal and outrun the herd as long as possible is only half the fun. Once you've tamed a few animals, you can bring them home with you. [Read more] | Read more »
VoxSyn (Music)
VoxSyn 1.0 Device: iOS Universal Category: Music Price: $6.99, Version: 1.0 (iTunes) Description: VoxSyn turns your voice into the most flexible vocal sound generator ever. Instantly following even subtle modulations of pitch and... | Read more »
Catch Battleplans on Google Play from Ju...
Real-time strategy title Battleplans is due for release on Google Play on June 30th, following its release for iOS systems last month. With its simple interface and pretty graphics, the crowd-pleaser brings a formerly overlooked genre out for the... | Read more »

Price Scanner via MacPrices.net

MacBook Airs on sale for up to $50-$100 off M...
B&H Photo has 13″ and 11″ MacBook Airs on sale for up to $100 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 11″ 1.6GHz/128GB MacBook Air: $849 $50 off list price - 11″ 1.... Read more
Brexit Vote Result Forecast To Slash UK 2016...
Uncertainty and economic volatility can be expected to increase over the next nine months, as the Brexit and concerns over the future of the EU hit IT investment, say Canalys market analysts, with... Read more
13-inch 256GB MacBook Air on sale for $1149,...
Amazon has the 2016 13″ 1.6GHz/256GB MacBook Air (model MMGG2LL/A) on sale for $1149.99 including free shipping. Their price is also $50 off MSRP. Read more
Haven App Launches New Age Of Wirless 911 Eme...
Haven from RapidSOS represents a transformation in access to emergency services from a phone call solely dependent on voice to a robust data connection for voice, text, medical/demographic data.... Read more
Cu Parachute 1.1 Retirement Success PLanning...
Tucson, Arizona based Indie developer Bradley McCarthy has announce the release of Cu (Copper) Parachute 1.1 for iPhone, iPad, and iPod touch devices — a tool with which users can continuously... Read more
Research and Markets Releases iPhone 6s Plus...
A new analysis report from Dublin-based Research and Markets observes that with the iPhone 6s Plus, Apple introduced a new rear camera module. The new device has similar structure and technology than... Read more
Apple refurbished Retina MacBook Pros availab...
Apple has Certified Refurbished 2015 13″ and 15″ Retina MacBook Pros available for up to $380 off the cost of new models. An Apple one-year warranty is included with each model, and shipping is free... Read more
Apple refurbished 11-inch MacBook Airs availa...
Apple has Certified Refurbished 11″ MacBook Airs (the latest models), available for up to $170 off the cost of new models. An Apple one-year warranty is included with each MacBook, and shipping is... Read more
Apple price trackers, updated continuously
Scan our Apple Price Trackers for the latest information on sales, bundles, and availability on systems from Apple’s authorized internet/catalog resellers. We update the trackers continuously: - 15″... Read more
12-inch 32GB and 128GB WiFi iPad Pros on sale...
B&H Photo has 12″ 32GB & 128GB WiFi iPad Pros on sale for up to $80 off MSRP, each including free shipping. B&H charges sales tax in NY only: - 12″ Space Gray 32GB WiFi iPad Pro: $749 $50... Read more

Jobs Board

*Apple* iPhone 6s and New Products Tester Ne...
…we therefore look forward to put out products to quality test for durability. Apple leads the digital music revolution with its iPods and iTunes online store, Read more
*Apple* iPhone 6s and New Products Tester Ne...
…we therefore look forward to put out products to quality test for durability. Apple leads the digital music revolution with its iPods and iTunes online store, Read more
*Apple* Retail - Multiple Positions, Willow...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* iPhone 6s and New Products Tester Ne...
…we therefore look forward to put out products to quality test for durability. Apple leads the digital music revolution with its iPods and iTunes online store, Read more
*Apple* iPhone 6s and New Products Tester Ne...
…we therefore look forward to put out products to quality test for durability. Apple leads the digital music revolution with its iPods and iTunes online store, Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.