TweetFollow Us on Twitter

December 94 - Somewhere in QuickTime: Supporting Text Tracks in Your Application

Somewhere in QuickTime: Supporting Text Tracks in Your Application

Nick Thompson

Text media tracks were introduced with QuickTime version 1.5 and have been further improved in QuickTime 2.0 with the addition of calls for more powerful searching and a new facility called burnt text. The big news is that text tracks are now supported in QuickTime for Windows version 2.0, which makes text tracks a cross-platform solution. If you're developing content-based products that need to be cross-platform, you'll want to take a look at text tracks.

Text tracks give you the ability to embed text in a QuickTime movie, which is particularly useful if you're aiming your product at international markets or at people with hearing impairments (you can subtitle your movie) or if you want to enable your users to perform searches (by including the script of a play or movie in a text track, for instance, you can make it easy for users to find a particular scene by searching for a key piece of dialog). The possibilities for adding value to your QuickTime application and content with text tracks are limited only by your imagination.

This column shows you how to add text track support to your QuickTime application, including support for searching and editing. The sample program QTTextSample on this issue's CD demonstrates what's involved. This small application plays a movie controller-based movie in a window and offers users the ability to search for a particular sequence of characters, or edit the text, in the text track.

Text tracks are handled by the text media handler, which is documented in the "Movie Toolbox" chapter of Inside Macintosh: QuickTime. The text media handler's functionality includes the following:

  • Searching for text, using FindNextText. With QuickTime 2.0, a new routine for text searching, MovieSearchText, can be used. This call is also available with QuickTime 2.0 for Windows.

  • Adding plain or styled text, using AddTextSample or AddTESample. Both of these calls allow you to define additional text properties, such as scrolling and highlighting.
Searching is something that all applications that support text tracks will want to offer the user, while editing text is likely to be something that only a few specialized applications will want to provide. Editing of text can be accomplished with Movie Toolbox routines, as discussed later in this column.

FIRST THINGS FIRST

Your QuickTime application needs to do a few basic things at the outset, as QTTextSample demonstrates. These include checking for the QuickTime version number, growing your heap as large as possible, and checking return codes.

QuickTime version number. A Macintosh application that supports text tracks requires QuickTime version 1.5 or later; a Windows application that supports text tracks requires QuickTime 2.0 for Windows. On Macintosh platforms you can use the Gestalt selector gestaltQuickTime to check that the version number returned in the high-order word is greater than or equal to the required version (0x0150 for QuickTime 1.5; 0x0200 for QuickTime 2.0).

Our sample program bails out if the system isn't running QuickTime version 1.5 or later. If your application uses calls provided by later releases of QuickTime but you also want it to run on earlier versions, you should check for the version number and selectively enable your application's functionality accordingly.

Heap size. As discussed in the Somewhere in QuickTime column in develop Issue 13 ("Top 10 QuickTime Tips" by John Wang), you need to ensure that you grow your heap as large as possible in your initialization code by calling the MaxApplZone routine. QuickTime needs to use a lot of handle-based data structures, so you also need to ensure that there are enough master pointers allocated in the base of your heap. To do that, you should call MoreMasters a number of times (you can tell how many times by examining your heap while the application is running). If you don't do this, your heap may become fragmented, which in turn may cause certain QuickTime routines to fail due to lack of memory. This is a general Macintosh programming issue that you should be aware of for all applications.

Return codes. Finally, always check those return codes, and handle errors as required. Most QuickTime routines return a status code of type OSErr; if this isn't equal to noErr, you have problems. QuickTime is always trying to tell you something -- listen and your life will be more complete!

THE USER INTERFACE

The user interface for the sample application is pretty basic. Figure 1 shows the application window. As you can see, the text for the movie is repeated below the movie so that it can be edited. Buttons offer the user the options of finding specific text or updating the text with editing changes. (The Update Text button is dimmed unless the text in the text box has been modified.)

With text track selected

With movie selected

Figure 1. The application window for QTTextSample

Because the user can potentially edit two items -- the movie's text track or the movie itself -- the application needs to keep track of what the user selected last (either text or the movie) and highlight it in some way. As shown in Figure 1, when the text track is selected in our sample application, the text below the movie has a black box around it; when the movie is selected, the movie frame has a black box around it. When the window is inactive (for example, when you switch applications), the box surrounding the active item is rendered in gray.

TRICKS FOR EASY ACCESS

In the sample code, the movie controller is stored in a record referenced by a handle stored in the window's refCon, along with a few other bits and pieces related to the movie and the text for a window. This gives us easy access to both the movie controller and its associated movie:
aDocHdl = (DocHandle)GetWRefCon(theMovieWindow);
aController = (**aDocHdl).myController;
aMovie = MCGetMovie(aController);
When we need to locate the first track of a particular type (in our case a text track) in a movie, we can use the following handy utility routine:
Track GetFirstTrackOfType (Movie aMovie,
                              OSType trackType)
{
   Track      theTrack = nil; 
   OSType     mediaType;
   short      trackCount, index;
   
   trackCount = GetMovieTrackCount(aMovie);
   for (index=1; index <= trackCount; index++) {
      Track t = GetMovieIndTrack(aMovie, index);
      GetMediaHandlerDescription(GetTrackMedia(t),
                  &mediaType, nil, nil);
      if (mediaType == trackType) {
         theTrack = t;
         break;
      }
   }
   return theTrack;
}
A new function, GetMovieIndTrackType, was introduced with QuickTime 2.0 for both Macintosh and Windows. GetMovieIndTrackType provides an easy way to iterate through all tracks in a movie that are either of a given media type or that support a particular media characteristic. Documentation for this, and the other new QuickTime calls, can be found in the QuickTime 2.0 Developer's Kit which is available from Apple Developer Catalog

HANDLING TEXT TRACKS

By default, QuickTime displays the text for an enabled text track. We want to be able to exercise more control over the format and display of the text track and to edit the text embedded in the movie. Thus, we need to extract the text from the track and stuff it into a TextEdit record.

Our application needs to be able to access the text for a particular frame as it's displayed. We do this by defining a text-handling procedure (or textProc for short) with the following format:

pascal OSErr MyTextProc (Handle thisText, 
         Movie thisMovie, short *displayFlag, 
         long refCon)
The text is passed in the handle. To access the text, we need to determine the length of the text and store it somewhere.
// This yields the actual size of the text.
textSize = *(short*)(*thisText);
// This yields a pointer to the text.
textSamplePtr = 
            (char*)(*thisText + sizeof(short)); 
The style data for a text track is stored in one of two places. Information about the default text style, together with other items of interest (such as the background color), is stored in a text description handle (see page 2-291 of Inside Macintosh: QuickTime). Additional information may be supplied at the end of the handle passed to the textProc, in the form of 'styl' atoms (see page 2-290 of Inside Macintosh: QuickTime). Our sample code demonstrates how to access the style information. To get the text description, you need to call GetMediaSample in the textProc, as shown in Listing 1. You need to parse the handle passed into your textProc to see if additional information is supplied; the sample code illustrates how to do this.

Listing 1. Getting the text description

theErr = GetMediaSample(aMedia, myData, nil, nil, mediaCurrentTime,
         nil, sampleDescriptionH, nil, nil, nil, nil);
...
if (sampleDescriptionH) {
   scrapHdl = (StScrpHandle)NewHandle(sizeof(StScrpRec));
   if (scrapHdl == nil)
      CheckError(MemError(), 
         "\pCouldn't allocate memory for the style table");
   (**scrapHdl).scrpNStyles = 1;
   (**scrapHdl).scrpStyleTab[0] =
      (**((TextDescriptionHandle)sampleDescriptionH)).defaultStyle;

   // Delete the previous contents of the TextEditHandle.
   TESetSelect(0, (**myDocTEH).teLength, myDocTEH);
   TEDelete(myDocTEH);

   // Insert the new text.
   TEStylInsert(textPtr, textSize, scrapHdl, myDocTEH);

   DisposeHandle((Handle)scrapHdl);
}
else TESetText(textPtr, textSize, myDocTEH);
We can control whether QuickTime also displays the text, by returning a value in the displayFlag parameter. For example, if we want the default display, we set it in the following way:
*displayFlag = txtProcDefaultDisplay;
Other flags are available either to suppress output or to ensure that QuickTime always displays the text track, regardless of the settings saved in the movie. Check page 2-364 of Inside Macintosh: QuickTime for more details.

In order for the textProc we've defined to get called, we need to tell QuickTime about it. This is easily accomplished with the SetMovieTextHandler routine (shown in Listing 2), which uses the utility routine described earlier to get the first text track.

Listing 2. The SetMovieTextHandler routine

OSErr SetMovieTextHandler (WindowPtr aWindow)
{
   MediaHandler        aMediaHandler;
   MovieController     aController;
   Movie               aMovie;
   Track               aTrack;
   DocumentHandle      aDocHdl;

   aDocHdl = (DocumentHandle)GetWRefCon(aWindow);
   aController = (**aDocHdl).myController;
   if (aController != nil) {
      aMovie = MCGetMovie(aController);

      // If there's a text track in the movie, set the textProc.
      if (aMovie != nil && (aTrack = GetFirstTrackOfType
             (aMovie, TextMediaType)) != nil) {
         aMediaHandler = GetMediaHandler(GetTrackMedia(aTrack));
         if (aMediaHandler != nil) 
            SetTextProc(aMediaHandler, NewTextMediaProc(MyTextProc),
               (long)aWindow);
      }
   }
   return GetMoviesError();
}

SEARCHING FOR TEXT

One feature that should be provided in movies that have embedded text is the ability to search for words. Consider a scenario where you're providing an interactive learning experience. The video track of your QuickTime movie contains a play, and the text track contains its script. Students can search for a particular scene just by searching for a few words. If the play were Shakespeare's Julius Caesar, for example, searching for "Lend me your ears" would find Marc Antony's speech at Caesar's funeral.

Searching for text in a movie is a straightforward operation using QuickTime's FindNextText routine (which is described on page 2-298 of Inside Macintosh: QuickTime). You can control the way this routine works by passing in the following flags:

  • findTextWrapAround -- wraps the search around at the end or start of the movie and continues searching for the text

  • findTextReverseSearch -- searches backward

  • findTextCaseSensitive -- makes the search case sensitive
Under QuickTime 1.5 or 1.6, however, you shouldn't use all three of these flags together in the same call; if you do, a bug will cause a bus error. You can work around this by manually implementing a wrapped search. This was fixed in QuickTime 2.0.

The sample code illustrates the use of FindNextText in the DoSearchForStringInMovieWindow routine. This routine gets the movie controller and its associated movie from the movie window. We pass in the text to search for, the direction to search in, and whether to wrap the search. The sample code shows one way of doing this with a simple dialog.

The new routine MovieSearchText, which was added to the Movie Toolbox in QuickTime 2.0 and in QuickTime 2.0 for Windows, also aids in searching for text in a movie. It can search any track that supports the text characteristic. (For a track to support the text characteristic, it must implement the FindNextText and HiliteTextSample calls as defined in the Movies.h file.) MovieSearchText is defined like this:

pascal OSErr MovieSearchText(Movie theMovie, 
   Ptr text, long size, long searchFlags, 
   Track *searchTrack, TimeValue *searchTime,
   long *searchOffset);
In this definition, theMovie is the movie to search, text is a pointer to a block of text that contains the search string, and size is the length of the search string in bytes. The other parameters are as follows:
  • searchFlags is a combination of findText flags as defined for media handlers that support text and searchText flags that manage the higher-level searching operation.

  • searchTrack is a pointer to the first track to search (or the only track, if searchTextOneTrackOnly is set in searchFlags). If the text is found, searchTrack will be updated to point to the track in which the text was found. If nil is passed in for searchTrack or if it points to nil, the search will start from the first track in the movie.

  • searchTime is a pointer to the movie time at which to start the current search. If the text is found, searchTime will be updated to reflect the movie time at which the text was found. If nil is passed in for searchTime or if it points to -1, the current movie time will be used.

  • searchOffset is a pointer to the offset within the text sample (as defined by searchTrack and searchTime) in which to start the search. If the text is found, searchOffset will be updated to reflect the offset into the text sample where the text was found. If nil is passed in for this parameter, an offset value of 0 will be used.
If MovieSearchText doesn't succeed in finding the search string because either there were no text tracks in the movie or the text simply wasn't found, an error value is returned.

EDITING TEXT

While the text media handler provides routines to add and delete text track segments, it doesn't provide routines to edit the text contained in a text track. Most applications won't need to edit text, but in case you're interested, this section looks at the Movie Toolbox routines involved.

The DoUpdateText routine in the sample code shows how the user can edit the text contained in a movie's text track. The steps involved in this process are listed below, and the code to accomplish these steps is shown in Listing 3. Note that error checking isn't included in this simplified version of the sample code; you'll find the complete code on this issue's CD.

    1. Determine which text track to edit.

    2. Determine the segment of the track to be edited. To do this we need to find the start time and duration of the sample we want to edit.

    3. Delete the existing text using the start time and duration we've determined.

    4. Add the text from the TextEdit handle into the media, using the QuickTime routine AddTESample (described on page 2-295 of Inside Macintosh: QuickTime). Then call the InsertMediaIntoTrack routine to insert the media we just created into the track.

Like all movie editing operations, editing text will cause the movie to become fragmented. You should ensure that the final production version of your movie is flattened; this will remove any fragmentation introduced by editing.

Listing 3. The DoUpdateText routine, simplified version

// Step 1:

// Get the text track; remember to check that it's not nil.
aTrack = GetFirstTrackOfType(aMovie, TextMediaType);
...

// Step 2:

// Get the media time of the current sample.
mediaCurrentTime = TrackTimeToMediaTime(currentTime, aTrack);
...
// Get detailed information on start and duration of the current
// sample (this is used later).
MediaTimeToSampleNum(aMedia, mediaCurrentTime, &mediaSampleIndex,
   &mediaSampleStartTime, &mediaSampleDuration);
...
// Look back and find where this text starts.
theErr = GetTrackNextInterestingTime(aTrack,
         nextTimeMediaSample | nextTimeEdgeOK, currentTime,
         -kFix1, &interestingTime, nil);

currentTime = interestingTime;

// Determine the duration of this sample.
theErr = GetTrackNextInterestingTime(aTrack,
         nextTimeMediaSample | nextTimeEdgeOK, currentTime,
         kFix1, &interestingTime, &theDuration);
...

// Step 3:

// Tell the media that we're about to edit stuff.
theErr = BeginMediaEdits(aMedia);
...
// Delete whatever was there before.
theErr = DeleteTrackSegment(aTrack, interestingTime, theDuration);
...

// Step 4:

// Write out the new data to the media.
theErr = AddTESample(aMediaHandler, aTEH,
         (RGBColor *)&theTextColor, teFlushDefault, nil,
         dfClipToTextBox, 0, 0, 0, nil, mediaSampleDuration,
         &sampleTime);
...
// Insert the new media into the track.
theErr = InsertMediaIntoTrack(aTrack, interestingTime, sampleTime,
    mediaSampleDuration, kFix1);
...
theErr = EndMediaEdits(aMedia);
...

NEW IN QUICKTIME 2.0: BURNT TEXT

QuickTime 1.6 introduced the capability for applications to apply special effects to the text in text tracks, notably antialiased text and drop shadows. Antialiased text is generally easier to read and looks more attractive than fonts rendered in the normal manner. However, antialiasing text takes time, and the performance penalty that's incurred playing movies with antialiased text tracks limits their usefulness.

QuickTime 2.0 allows applications to prerender text tracks, with a new facility called burnt text. Burnt text not only incurs less of a performance penalty than antialiased text but also has the advantage that a font doesn't need to be installed on the target machine in order to be rendered correctly. Applications that want to take advantage of this facility need to write data to the movie file in the form of a number of new atoms; for information on these additional atoms, see the file Text Imaging in QuickTime 2.0, accompanying this column on the CD.

THAT'S ALL THERE IS TO IT

Adding text track support to QuickTime applications really makes sense. With just a few lines of code, you can add a great deal of functionality. Most applications that use text tracks won't need to support editing the text, but it's a good idea to support searching because it provides an easy and powerful way of indexing into a movie containing text tracks.

Take a look at the sample code on the CD. It will help you get started with adding basic searching to your applications and (if required) with more advanced text track features, such as editing. Have fun!

NICK THOMPSON (AppleLink NICKT) found his first job in a surfboard factory, gluing wetsuits together. Then he scammed a job finishing custom surfboards. Somewhere along the way he learned how to program, and he's been riding that wave ever since. Last summer he snagged a job at Apple in Developer Technical Support and moved from London to California. Now he dresses up in a neoprene seal suit on weekends and goes shark fishing in the cold Pacific, armed only with a surfboard. (The glue from his first job must have affected his brain.)

Thanks to Ken Doyle, C. K. Haun, Peter Hoddie, Don Johnson, and John Wang for reviewing this column.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

The beginner's guide to Warbits
Warbits is a turn-based strategy that's clearly inspired by Nintendo's Advance Wars series. Since turn-based strategy games can be kind of tricky to dive into, see below for a few tips to help you in the beginning. Positioning is crucial [Read... | Read more »
How to upgrade your character in Spellsp...
So you’ve mastered the basics of Spellspire. By which I mean you’ve realised it’s all about spelling things in a spire. What next? Well you’re going to need to figure out how to toughen up your character. It’s all well and good being able to spell... | Read more »
5 slither.io mash-ups we'd love to...
If there's one thing that slither.io has proved, it's that the addictive gameplay of Agar.io can be transplanted onto basically anything and it will still be good fun. It wouldn't be surprising if we saw other developers jumping on the bandwagon,... | Read more »
How to navigate the terrain in Sky Charm...
Sky Charms is a whimsical match-'em up adventure that uses creative level design to really ramp up the difficulty. [Read more] | Read more »
Victorious Knight (Games)
Victorious Knight 1.3 Device: iOS Universal Category: Games Price: $1.99, Version: 1.3 (iTunes) Description: New challenges awaits you! Experience fresh RPG experience with a unique combat mechanic, packed with high quality 3D... | Read more »
Agent Gumball - Roguelike Spy Game (Gam...
Agent Gumball - Roguelike Spy Game 1.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0 (iTunes) Description: Someone’s been spying on Gumball. What the what?! Two can play at that game! GO UNDERCOVERSneak past enemy... | Read more »
Runaway Toad (Games)
Runaway Toad 1.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0 (iTunes) Description: It ain’t easy bein’ green! Tap, hold, and swipe to help Toad hop to safety in this gorgeous new action game from the creators of... | Read more »
PsyCard (Games)
PsyCard 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: From the makers och Card City Nights, Progress To 100 and Ittle Dew PSYCARD is a minesweeper-like game set in a cozy cyberpunk... | Read more »
Sago Mini Robot Party (Education)
Sago Mini Robot Party 1.0 Device: iOS Universal Category: Education Price: $2.99, Version: 1.0 (iTunes) Description: -- Children's Technology Review Editor's Choice -- | Read more »
How to get a high score in every level o...
Sky Charms is an adorable match three puzzler that provides a decent challenge thanks to its creative level design. It regularly presents something new, forcing you to think on your feet. [Read more] | Read more »

Price Scanner via MacPrices.net

Apple restocks Certified Refurbished Mac mini...
Apple has restocked Certified Refurbished 2014 Mac minis, with models available starting at $419. Apple’s one-year warranty is included with each mini, and shipping is free: - 1.4GHz Mac mini: $419 $... Read more
15-inch 2.2GHz Retina MacBook Pro on sale for...
Amazon.com has the 15″ 2.2GHz Retina MacBook Pro on sale for $1699.99 including free shipping. Their price is $300 off MSRP, and it’s the lowest price available for this model from any reseller (and... Read more
Apple Beats Microsoft at Own Game; Amazon Pri...
First quarter seasonality combined with an overall disinterested customer base led to an annual decline of 14.7% in worldwide tablet shipments during the first quarter of 2016 (1Q16). Worldwide... Read more
Tablets Had Worst Quarter Since 2012, says St...
The global tablet market began 2016 just as 2015 left off, down. Tablet shipments fell 10% to 46.5 million units during the Q1 2016, according to the new “Preliminary Global Tablet Shipments and... Read more
Clearance 13-inch MacBook Airs, Apple refurbi...
Apple recently dropped prices on certified refurbished 2015 13″ MacBook Airs with 4GB of RAM with models now available starting at $759. An Apple one-year warranty is included with each MacBook, and... Read more
Clearance 12-inch Retina MacBooks, Apple refu...
Apple has dropped prices on Certified Refurbished 2015 12″ Retina MacBooks with models now available starting at $929. Apple will include a standard one-year warranty with each MacBook, and shipping... Read more
Aleratec Releases Mac Software Upgrade for 1...
California based Aleratec Inc., designer, developer and manufacturer of Portable Device Management (PDM) charge/sync products for mobile devices and professional-grade duplicators for hard disk... Read more
Sale! Amazon offers 27-inch iMac, 13-inch 2.9...
Amazon has the 27″ 3.2GHz 5K iMac and the 13″ 3.9GHz Retina MacBook Pro on sale for $300 off MSRP, each including free shipping, for a limited time: - 27″ 3.2GHz/1TB HD 5K iMac (model MK462LL/A): $... Read more
Apple refurbished 13-inch Retina MacBook Pros...
Apple has Certified Refurbished 13″ Retina MacBook Pros available for up to $270 off the cost of new models. An Apple one-year warranty is included with each model, and shipping is free: - 13″ 2.7GHz... Read more
13-inch 2.7GHz/128GB Retina MacBook Pro on sa...
Take $200 off MSRP on the price of a new 13″ 2.7GHz/128GB Retina MacBook Pro (model MF839LL/A) at Amazon. Shipping is free: - 13″ 2.7GHz/128GB Retina MacBook Pro: $1099.99 $200 off MSRP Act now if... Read more

Jobs Board

Restaurant Manager (Neighborhood Captain) - A...
…in every aspect of daily operation. WHY YOU'LL LIKE IT: You'll be the Big Apple . You'll solve problems. You'll get to show your ability to handle the stress and Read more
Automotive Sales Consultant - Apple Ford Linc...
…you. The best candidates are smart, technologically savvy and are customer focused. Apple Ford Lincoln Apple Valley is different, because: $30,000 annual salary Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Solutions Consultant - Apple (United...
# Apple Solutions Consultant Job Number: 48260200 Phoenix, Arizona, United States Posted: Apr. 22, 2016 Weekly Hours: 40.00 **Job Summary** As an Apple Solutions Read more
Restaurant Manager (Neighborhood Captain) - A...
…in every aspect of daily operation. WHY YOU'LL LIKE IT: You'll be the Big Apple . You'll solve problems. You'll get to show your ability to handle the stress and Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.