TweetFollow Us on Twitter

XCMD Import Text
Volume Number:4
Issue Number:10
Column Tag:HyperChat®

XCMD Corner

By Donald Koscheka, Apple Computers, Inc.

Importing Text into Hypercard

A new controversy seems to be emerging in the Hypercard community. Some Hypercard pundits are discouraging the use of XCMDs and XFCNs in stack design.. Their most convincing argument is that those of us who jump into writing XCMDs aren’t giving ourselves an opportunity to see if HyperTalk can perform the task, perhaps equally as well as an XCMD.

I frequently consider writing an XCMD solution to a programming problem without first considering whether Hypertalk can do the same job for me. Recently, I needed to import Microsoft WORD files into Hypercard. What a wonderful opportunity to write an XCMD!

When I sat down to write the script to invoke the XCMD, I realized that I could write the entire WORD import routine in HyperTalk. Ed Wischmeyer of Apple Computer Inc. pointed out that although fields in HyperTalk prefer to see straight ASCII text, there is no such restriction on the contents of containers. Hypercard also allows you to open and read any file type you want; you aren’t restricted to reading text files. Of course, you need to figure out how to translate what’s in that container into a format that can be presented in a field.

The hard part of importing text from a Word file is not reading the data into hypercard but rather figuring how Word stores its text. By committing the import code to a simple Hypertalk script, I could concentrate my efforts at decoding Word’s file format.

To simplify my search through the file format, I made the assumption that I could ignore any formatting information such as rulers, font and style changes. I was after was the text portion of the file only. This turns out to be a valid assumption since I wanted to import the file into a Hypercard field as text.

Finding the text was a snap with John Mitchell’s “FEDIT+”. I created a Word file using WORD and then examined it in FEDIT+. I noticed that the text always started at location 256 in the file. Since the size of the file was larger than the size of the text plus this 256 byte header, I needed to determine where the end of text occurred (assuming that the formatting and ruler information follows the text in the file). Since I knew how long the text was, I again used FEDIT+ to search the 256 header portion of the file. This time I was looking for any portion of the header that contained a count of the number of bytes in the text. Since I knew that my file contained exactly 100 characters (bytes), all I had to do was find this number somewhere in the header portion of the file. I found something close to what I was looking for at offset 16 in the file. This location corresponded to the number of characters in the text portion of the file plus 256 which was the length of the header.

The creators of Microsoft Word may be reading this and wondering why I’m assuming that the text size is a 16-bit entity rather than a 32 bit number. I’m not. Since Hypercard text fields are currently limited to 32K bytes, and since I knew none of my word files were longer than this, I’m only interested in the low-order word of the text length.

Reading the text portion of a Microsoft Word file into a hypercard container requires the following steps: (1) Position the mark at byte 16 of the file. Read the byte at this position and multiply it by 256 making it the high-order half of the file length. Read the next byte and add it to the hight-order half of the length. Move 238 more bytes into the file (16+2+238 = 256). This is the start of the text portion of the file. Read the number of bytes calculated minus 256. The IT container gets the imported text.

The Hypertalk script in listing 1 performs the above steps for importing up to 16K bytes of text from a Word file. I use Steisplay WORD files only in the GetFile dialog and to get the full pathname of the file from the user. This script reads in the text without any looping so an XCMD may not speed things up enough to be warranted.

{1}
on mouseup
  put filename(“WDBN”)into filename
  if filename is not empty then
     open file filename -- filename is the full pathname of a WORD file
     read from file filename for 16 -- move file mark to the text length 
word
     read from file filename for 1  -- read the upper half of the length
     put chartonum( it ) * 256 into filesize -- shift up by 8 bits
     read from file filename for 1  -- get the lower half of the length
     add chartonum( it ) mod 256 to filesize
     read from file filename for 238-- move to start of text in the file
     read from file filename for filesize-256 -- read in the text
     close file filename -- IT now contains the imported data.    
  end if
end mouseup

Listing 1. Script to Import Text from a Microsoft Word File

Not all file formats can be imported quite so simply. Macwrite uses a packed text format, storing one or two characters per byte using a simple compression scheme.

Because the text is compressed, we can’t just read the file into a container and return the result to Hypercard. We must first decompress the file a byte at a time. Such a process suggests looping and loops, as we know, are not particularly fast in HyperTalk. Although the decompression can be performed in a hypertalk script, we can write an xcmd that performs the decompression faster.

The key to reading in a MacWrite file is understanding that Macwrite stores its data by paragraph. Whereas Word files are clearly divided between the text and formatting information, Macwrite stores formatting information for each paragraph at the end of the text for that paragraph. Hypercard doesn’t do formatted text; we want to ignore the formatting information at the end of each paragraph. Our algorithm then becomes a loop that reads in a paragraph at a time, decompresses the text for that paragraph ignoring the formatting information. This process is repeated for each paragraph in the file.

One small “gotcha” to this approach stems from the fact that Rulers and pictures are also considered paragraphs. When we encounter either of these objects, we just move on to the next paragraph.

Listing 2 depicts the code for this XFCN. I chose “C” because pointer arithmetic is easier to perform in “C” and because last month’s example was written in Pascal. I made every attempt to keep the “C” isomorphic to a Pascal program so that you can easily convert the code to Pascal.

Finding the paragraph information in the file requires a little arithmetic. Bytes 2-3 in the file tell us how many paragraphs the main document contains (MacWrite makes a distinction between the main document, the header document and the footer document. For our purposes, we only want to read in the main body of text) If bytes 2-3 contain a 5 then there are 5 paragraphs in the main document.

For each paragraph, MacWrite stores an information array. We start reading the information arrays at the file position pointed to in file offset $108. An information array is an array of 16-byte elements that tell us something about each paragraph. The first two bytes in the information array tell us whether the paragraph contains text, a ruler or a picture. If this value is positive the paragraph contains text, if this value is 0 or negative the paragraph is a ruler or a picture respectively and we can ignore it.

Offset 8 in the information array contains a status byte that provides some information about the text. If bit 3 is set, the text in this paragraph is compressed. Bytes 9-11 tell us the absolute file offset for the start of the data in the paragraph and bytes 12-13 contain the length of the data (paragraph addressing is 24 bits and each paragraph contains up to 64K of characters or data). The trick is to read in the number of characters indicated in the information array, determine if the paragraph contains text and, if so, decompress the text if it’s compressed.

Once we read in the paragraph, we get some more information. The first two bytes of the paragraph tell us how many characters of text will appear in the decompressed paragraph. Following the text on an even word boundary is the formatting information for the paragraph which we ignore in this example.

MacWrite’s text compression is based on a letter frequency scheme stored as STR resource #700 in MacWrite’s resource fork. For English, this string contains “ etnroaisdlhcfp”. Macwrite maps these characters onto the array [$0..$F]. The space character ($20) has a value of 0, letter “e” has a value of 1, “t” a value of 2 and so on. Since any number less than $F can fit into a nibble, the word “eels” can be represented as “$11A8” rather than the byte-wide representation of $65656C73. In this example, we realize a 50% space saving (the best case for this algorithm).

This compression scheme only works for lower-case letters since 4 bits is not enough information to code for word frequency and case for the 14 most popular letters. This scheme also doesn’t compress non-alphabetic characters such as numerals and punctuation marks. In these cases, the 16th array element, $F, is used as a flag to tell indicate that the next 2 nibbles represent one character. “Then” would be coded as $F55906. Note that the letter “T” crosses byte boundaries, the top nibble is in byte 0 and the lower nibble is in byte 1. This is of no consequence to the algorithm.

Armed with this information, you should have little trouble understanding the XFCN. In fact, I hope you find it useful and informative! (Next month: printing from XCMDs).

{2}
/*************************\
*file:  MWRead.c *
*  *
* an XFCN that imports text *
* directly from a MacWrite file  *
* whose full pathname is passed  *
* as an input parameter.  *
**
* -------------------------------- *
* To Build this file:*
**
*C -q2 -g MWRead.c *
**
*link -sn Main=MWRead    *
*-sn STDIO=MWRead *
*-sn INTENV=MWRead  *
*-rt XFCN=301   *
*-m MWREAD MWRead.c.o    *
*“{CLibraries}”CInterface.o  *
*-o “your stack name”*
**
* -------------------------------- *
* By: Donald Koscheka*
* Date: 2-July-1988*
* ©1988, Donald Koscheka  *
*All Rights Reserved *
**
* -------------------------------- *
\*************************/

#include<Types.h>
#include<OSUtils.h>
#include<Memory.h>
#include<Files.h>
#include<Resources.h>
#include  “HyperXCmd.h”

#define INFOPOS  0x00000108
#define PPOS0x00000002
#define COMP0x0008


/* -------------------------- */
/* Define the structure of an */
/* information array element*/
/* */
/* pHite is positive if this*/
/* info array points to text, */
/* ignored otherwise.*/
/* */
/* fPos is the absolute file*/
/* position of the start of */
/* the paragraph in the file*/
/* */
/* fLen is the total length of*/
/* the file including formats */
/* -------------------------- */

typedef struct infoArr {
 short  pHite;/* parag hite */
 short  pixels;/* ignore this */ 
 long pHand;/* ignore this*/
 char status;/* chk comprsn */
 char hiMark;/* msw of mark */
 short  loMark;/* lsw of mark */
 short  fLen;  /* parag. len*/
 short  fmat;  /* ignore this */ 
}infoArr;

/* ------------------------ */

short   ReadFile();
Handle  DeCompress();


pascal void MWRead( paramPtr )
 XCmdBlockPtr  paramPtr;
/*************************
* In:ParamPtr:
*pointer to XCMD param
*block. params[0] is the
*name of the macwrite file
*  to open.
*
* Out:ParamPtr->returnValue
*empty if data could not
*be read, text portion 
*of a   Macwrite document.
*************************/
{
short 
 ref,   /* file reference */
 err,   /* io error  */
 vRef,  /* vol reference  */
 pcnt,  /* #  paragraphs  */
 tSiz,  /* text length  */
 loop;  /* loop counter */
long  
 fSiz,  /* data size */
 iSiz,  /* out data size  */
 iMark,/* iarr file pos */
 fPos;  /* para. offset */
Handle
 ImportText,
 decomp,/* decompressed*/
 temp;
infoArr
 info;
char 
 *fName,
 vName[32];

ImportText = nil;

if( paramPtr->params[0] != nil ){
HLock( paramPtr->params[0] );

GetVol( vName, &vRef );
fName = *(paramPtr->params[0]);
err = FSOpen( fName, vRef, &ref );
HUnlock( paramPtr->params[0] );

if( err == noErr ){
 ImportText = NewHandle( 0 );
 
 /* get paragraph count   */
 fSiz = sizeof( short );
 err=ReadFile(ref,fSiz,&pcnt,(long)PPOS);
 
 /* get infoArray position*/
 fSiz = sizeof( long );
 err=ReadFile(ref,fSiz,&iMark,(long)INFOPOS);
 
 /* read in the paragraphs*/
 for( loop = 0; loop < pcnt; loop++){
 fSiz = sizeof( infoArr );
 err=ReadFile(ref,fSiz,&info,iMark);
 
 if( info.pHite > 0 ){
 /* paragraph is text*/
 
 /* calc text position  */
 fPos=(info.hiMark<<0x10)+info.loMark;
 
 /* get the char count  */
 fSiz = sizeof( short );
 err = ReadFile(ref,fSiz,&tSiz,fPos);
 
 /* read in the text */
 temp = NewHandle((long)tSiz);
 HLock( temp );
 fPos += 2; 
 fSiz = (long)tSiz;
 err = ReadFile(ref,fSiz,*temp,fPos);
 
 if( info.status & COMP ){
 /* paragraph is compressed */
 HLock( temp );
 
 decomp = DeCompress( *temp, tSiz );
 
 HUnlock( temp );
 DisposHandle( temp );
 
 temp = decomp;
 tSiz = (short)GetHandleSize( decomp ); 
 }/* if( info.status & COMP ) */
 
 iSiz = GetHandleSize( ImportText );
 fSiz = (long)tSiz;
 SetHandleSize( ImportText, iSiz+fSiz );
 BlockMove(*temp,(*ImportText)+iSiz,fSiz);
 
 HUnlock( temp );
 DisposHandle( temp );
 }/* if( info.pHite > 0 ) */
 
 iMark = iMark + sizeof( infoArr );
 
 }/* FOR paragraph count  */

 iSiz = GetHandleSize( ImportText );
 SetHandleSize( ImportText, iSiz+1 );
 *((*ImportText)+iSiz) = ‘\0’;
 
 FSClose( ref );
 FlushVol( nil, vRef );
}/* if file opened ok     */

paramPtr->returnValue = ImportText;
}

}

short   ReadFile(ioRef,siz,buf,from)
short   ioRef;
long    siz;
char    *buf;
long    from;
/*************************
* read cnt bytes from the file specified by parms and put 
* the data into the buffer pointed to by buf 
*
* ioRef = file reference number
* siz = number of bytes to read
* buf = where to read in to
* from  = where in file to read from
*
* from is the file mark relative to the start of the file from
* which the read is to start.
*************************/
{
 short  err;
 
 err = SetFPos( ioRef, fsFromStart, from );
 if( err == noErr )
 err = FSRead( ioRef, &siz, buf ); 
 return( err);
}

Handle DeCompress( inp, expcnt )
 char *inp;
 short  expcnt;
/***********************
* Decompress the input handle’s data (inH) and put the result
* in the output Handle (outH).  outH is sized properly and 
* we use the following scheme:
*
*0 1 2 3 4 5 6 7 8 9 A B C D E F
*_ e t n r o a i s d l h c f p !
*
* where _ = SPACE
*! = not compressed
* 
* Consult MacWrite resource str #700 for the decompression
* string in your file (different for other languages).
* cycle through until the decompressed string count 
* matches the expected count
***********************/
{
 short  chcnt;
 register char *op;
 register char hiNib;
 register char loNib;
 char   dc[16];
 Handle outH;
 
 outH = NewHandle( (long)expcnt );

 dc[0]  = 0x020;
 dc[1]  = ‘e’;
 dc[2]  = ‘t’;
 dc[3]  = ‘n’;
 dc[4]  = ‘r’;
 dc[5]  = ‘o’;
 dc[6]  = ‘a’;
 dc[7]  = ‘i’;
 dc[8]  = ‘s’;
 dc[9]  = ‘d’;
 dc[10] = ‘l’;
 dc[11] = ‘h’;
 dc[12] = ‘c’;
 dc[13] = ‘f’;
 dc[14] = ‘p’;
 
 HLock( outH );
 op = *outH;
 chcnt = 0;
 
 while( chcnt < expcnt ){
 hiNib = loNib = *inp++;
 hiNib = hiNib >> 0x04;
 hiNib &=  0x000F;
 loNib &=  0x000F;
 
 if( hiNib < 0x0F ){
 *op++ = dc[hiNib];
 chcnt++;
 if( loNib < 0x0F ){
 *op++ = dc[loNib];
 chcnt++;
 }
 else{ /* next BYTE is a char */
 *op++ = *inp++;
 chcnt++;
 }
 }
 else{
 /* next 2 nibbles represent*/
 /* a complete char which */
 /* is on odd-nibble bounds */
 
 *op  = loNib << 0x04;
 hiNib  = *inp++;
 loNib = hiNib & 0x000F;
 hiNib = hiNib >> 0x04;
 hiNib &= 0x0F;
 *op  = *op | hiNib;
 op++;
 chcnt++;
 
 if( loNib < 0x0F )
 *op++ = dc[loNib];
 else
 *op++ = *inp++;
 chcnt++;
 }
 }
 HUnlock( outH );
 return( outH );
}

#include <XCmdGlue.inc.c>

Listing 2. XFCN to import trxt from a MacWrite Document

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Parallels Desktop 13.0.0 - Run Windows a...
Parallels allows you to run Windows and Mac applications side by side. Choose your view to make Windows invisible while still using its applications, or keep the familiar Windows background and... Read more
Mellel 4.0.0 - The word processor for sc...
Mellel is the leading word processor for OS X and has been widely considered the industry standard for long form documents since its inception. Mellel focuses on writers and scholars for technical... Read more
Backblaze 5.0.0.116 - Online backup serv...
Backblaze is an online backup service designed from the ground-up for the Mac. With unlimited storage available for $5 per month, as well as a free 15-day trial, peace of mind is within reach with... Read more
EtreCheck 3.4.4 - For troubleshooting yo...
EtreCheck is an app that displays the important details of your system configuration and allow you to copy that information to the Clipboard. It is meant to be used with Apple Support Communities to... Read more
Luminar 1.2.1 - Powerful, adaptive, conf...
Luminar is the new full-featured image editor that adapts to the way you edit photos. Over 300 essential tools to fix, edit, and enhance your photos with comfort. The future of photo editing is here... Read more
GarageSale 7.0.8 - Create outstanding eB...
GarageSale is a slick, full-featured client application for the eBay online auction system. Create and manage your auctions with ease. With GarageSale, you can create, edit, track, and manage... Read more
Adobe Muse CC 2017 2017.1.0 - Design and...
Muse CC 2017 is available as part of Adobe Creative Cloud for as little as $14.99/month (or $9.99/month if you're a previous Muse customer). Adobe Muse 2017 enables designers to create websites as... Read more
1Password 6.8.1 - Powerful password mana...
1Password is a password manager that uniquely brings you both security and convenience. It is the only program that provides anti-phishing protection and goes beyond password management by adding Web... Read more
Luminar 1.2.1 - Powerful, adaptive, conf...
Luminar is the new full-featured image editor that adapts to the way you edit photos. Over 300 essential tools to fix, edit, and enhance your photos with comfort. The future of photo editing is here... Read more
1Password 6.8.1 - Powerful password mana...
1Password is a password manager that uniquely brings you both security and convenience. It is the only program that provides anti-phishing protection and goes beyond password management by adding Web... Read more

Radiation City (Games)
Radiation City 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: Radiation City Welcome to the world of Radiation City where a great survival adventure awaits you! | Read more »
The best deals on the App Store this wee...
The summer is drawing quickly to a close, but luckily there's a game for every season. It's an excellent week for some bargain shopping if that's what you're after. There are some big names and indie darlings in this week's roundup. It's a great... | Read more »
KORG iMono/Poly (Music)
KORG iMono/Poly 1.0.0 Device: iOS Universal Category: Music Price: $19.99, Version: 1.0.0 (iTunes) Description: *** Special Sale for a limited time to celebrate the debut of KORG iMono/Poly (33% OFF) until Sep 30! *** Reviving a... | Read more »
Super Phantom Cat 2 beginner's guid...
Super Phantom Cat 2 presents a whole new world of fun platforming challenges and perplexing puzzles. It's a well-designed platformer with a bright, neon aesthetic that brings the genre up to date. [Read more] | Read more »
Shadow Fight 2 Special Edition (Games)
Shadow Fight 2 Special Edition 1.0.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0.0 (iTunes) Description: ** New story chapter! **** No Ads! **** No energy! ** The best fighting series on mobile has returned and... | Read more »
4 RPGs like Final Fantasy XV that deserv...
Square Enix announced another Final Fantasy XV spin-off today - Final Fantasy XV Pocket Edition. This mobile, episodic version of the hit RPG gives the game a chibi-fied makeover. The first episode will be free, followed by 9 more premium episodes... | Read more »
Guild sieges and soul gems in latest upd...
Webzen’s MU Origin hit app stores last year, giving fans of fantasy hack-n-slash MMOs like Diablo a new fix to fixate on. This latest update introduces a competitive guild battle, a fresh dungeon challenge, a mini-game and some elemental gems to... | Read more »
Little Red Lie (Games)
Little Red Lie 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: ARE YOU MORE AFRAID OF POVERTY THAN DEATH? Little Red Lie is a narrative-focused, interactive fiction experience that reduces... | Read more »
You can now apply to be Clash of Clans...
Earlier this month, word got out that the Builder, the trusty handiman who tirelessly built every single building inevery singleClash of Clansbase had called it quits. Sick of seeing his work destroyed endless, the Builder has set out for our world... | Read more »
Meshi Quest beginner's guide - how...
Meshi Quest is Square Enix's newest free-to-play release, and it's a real charmer. You start off as the head of a sushi restaurant, upgrading your food and equipment as you serve visitors heaping helpings of your delicious meals. As you progress,... | Read more »

Price Scanner via MacPrices.net

Low Cost Subscription Graphics App Alternativ...
I’m not a fan of the subscription software model, I don’t use any subscription apps. Used to be that you paid your license fee and the app was yours to use indefinitely, or until one opted for a paid... Read more
Clearance 2016 13-inch MacBook Airs, Apple re...
Apple has Certified Refurbished 2016 13″ MacBook Airs available starting at $809. An Apple one-year warranty is included with each MacBook, and shipping is free: – 13″ 1.6GHz/8GB/128GB MacBook Air: $... Read more
2017 13-inch MacBook Airs on sale for $100 of...
B&H Photo new 2017 13″ MacBook Airs on sale today for $100 off MSRP, starting at $899: – 13″ 1.8GHz/128GB MacBook Air (MQD32LL/A): $899, $100 off MSRP – 13″ 1.8GHz/256GB MacBook Air (MQD42LL/A... Read more
Sale! 13-inch 2.3GHz MacBook Pros for $100 of...
B&H Photo has 13″ 2.3GHz MacBook Pros in stock today and on sale for $100 off MSRP including free shipping plus NY & NJ sales tax only: – 13-inch 2.3GHz/128GB Space Gray MacBook Pro (MPXQ2LL... Read more
2016 MacBook Pros, Apple refurbished, availab...
Apple has Certified Refurbished 2016 15″ and 13″ MacBook Pros available starting at $1189. An Apple one-year warranty is included with each model, and shipping is free: – 15″ 2.7GHz Touch Bar Space... Read more
Apple offers Certified Refurbished iPhone 6s...
Apple has Certified Refurbished unlocked iPhone 6s’s and 6s Plus’s available starting at $449. An Apple one-year warranty is included with each phone, and shipping is free: – 16GB iPhone 6s: $449, $... Read more
Apple offers Certified Refurbished Pencils fo...
Apple has Certified Refurbished Apple Pencils available for $85 including free shipping. Their price is $14 off MSRP, and it’s the lowest price available for a Pencil. Read more
2016 15-inch 2.6GHz Touch Bar MacBook Pro ava...
B&H Photo has clearance 2016 15″ 2.6GHz MacBook Pros in stock today and on sale for $500 off original MSRP. Shipping is free, and B&H charges NY & NJ sales tax only: – 15″ 2.6GHz Touch... Read more
21-inch 2.3GHz iMac on sale for $999, save $1...
Amazon has the new 2017 21″ 2.3GHz iMac (MMQA2LL/A) in stock and on sale for $999.99 including free shipping. Their price is $100 off MSRP, and it’s the lowest price available for this model. Read more
Free Instant Translator 2.0 App For iOS Relea...
Mobile application development company, Neoappz has announced the release and immediate availability of Instant Translator 2.0 for iOS devices. Instant Translator is a user-friendly application which... Read more

Jobs Board

*Apple* Solutions Consultant - Apple Inc. (U...
…about helping others on a team while also delighting customers? As an Apple Solutions Consultant (ASC), you will discover customers needs and help connect them Read more
*Apple* Solutions Consultant - Apple Inc. (U...
Job Summary As an Apple Solutions Consultant, you'll be the link between our future customers and our products. You'll showcase your entrepreneurial spirit as you Read more
*Apple* Solutions Consultant - Apple Inc. (U...
…about helping others on a team while also delighting customers? As an Apple Solutions Consultant (ASC), you will discover customers needs and help connect them Read more
Sr. Software Engineer, Core Services, *Apple...
…part of the server team that powers various features within the App Store, Apple Music, iBooks, iTunes, and Podcasts. You will be working cross functionally with Read more
SW Engineer *Apple* TV Frameworks - Apple I...
Job Summary The Apple TV team is looking for a software engineer to join us as we work to define the future of the tvOS platform. Key Qualifications You're Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.