TweetFollow Us on Twitter

XModem Transfer
Volume Number:2
Issue Number:12
Column Tag:Forth Forum

Batch Text File Transfer by XMODEM

By Jörg Langowski, MacTutor Editorial Board, Grenoble, France

If you don't happen to have a tape backup unit for your Macintosh, but access to some larger computer installation over a terminal line (with, usually, lots of disk space), there is a slow but secure way to keep backups of your files by uploading them to that machine. There are all sorts of communication utilities that help in doing so, most of them using the XMODEM protocol for up- and downloading files.

The telecommunications programs I am aware of at this moment are excellent tools for transferring single files to and from the Macintosh. If one wants to back up one full disk by such a file transfer, however, the task becomes a little tedious since one has to select one file at a time; particularly time-consuming if your disk has 327 short text files on it. The XMODEM standard does offer a batch transfer mode in which the filename is transferred followed by the file, and if anyone out there knows of a terminal program that has this mode implemented, please let me know.

For the time being, the transfer of one whole volume on a HFS disk from the Mac to another system by XMODEM batch transfer is a neat little project to implement in Forth. Besides being useful, it'll help us gain some insight into HFS file handling.

We will limit ourselves to transferring text files here. Since you all are aware that files on the Macintosh consist of the data and the resource fork, applications and other resource-containing files have to be changed into text files using a utility like BINHEX first. However based on our example, it is rather easy to implement the MacBinary format [a special standard combining both forks together with the Finder information into one structure: see MacTutor V2#1], and transfer files of any type automatically. At the present time, a text file backup system is fine for me, since the documents I most often have to back up, like manuscripts or current versions of Mach 2 and NEON source files, are text files anyway.

The XMODEM Batch Transfer Protocol

How does an XMODEM file transfer proceed? Each file is split up into sectors of 128 bytes each (a relic of old CP/M times), each sector starting at one. A simple file transfer (not batch mode) has the following protocol:

Simple File Transfer Protocol

Sender Receiver

waits for NAK,

80 sec timeout sends NAK at

10 sec intervals

Loop n times for n sectors

send header:

SOH ($01)

sector no. (0-255)

sector no. complement (255-0)

send sector data, 128 bytes

send checksum:

(header bytes + sector data ) mod 255

compute checksum and

compare with checksum

byte

send ACK ($06) if OK and

NAK if not

If ACK received, increment

sector no.

otherwise resend sector

End Loop

send EOT ($04)

send ACK

The transfer may be cancelled at any time by the sender or receiver transmitting a Ctrl-X ($24) to the other end.

In batch mode, there is an additional protocol defined to transmit the filename before the file transfer starts:

Batch Mode Protocol

Sender Receiver

waits for NAK,

80 sec timeout sends NAK at

10 sec intervals

sends ACK

awaits filename characters,

1 sec timeout

on error -> above NAK

Loop 11 times

send filename char

(bit 7=0, upper case)

add to checksum receive filename char

await ACK 1 sec timeout add to checksum

ACK

End Loop

send Ctrl-Z

add to checksum receive Ctrl-Z

add to checksum

send checksum

receive checksum +

verify

ACK if OK and "u" if not await ACK

- Normal file transfer starts -

NAK

SOH

.

.

.

EOT

ACK

expect NAK to start new

filename NAK

ACK for new file name

or EOT if finished

The routines necessary to implement this protocol are printed in Listing 1. [For the single file transfer protocol, I adapted some routines from an article by Robert Taylor in Dr. Dobbs Journal 83 (9/1983) p.66.]

The program in Listing 1 is best used by having some terminal emulation like Mockterminal running on the Macintosh at the same time. Mach2 users may append this code directly to the "Terminal Emulator" example on the Mach2 demo disk and then start the terminal emulator before doing the transfer. For other Forths, the program should be easily adaptable, you'll have to change the routines for setting the baud rate and for doing the modem input and output.

Look at the definition of the words send and send-filename and the inner loop of send for the implementation of the XMODEM protocol in Forth.

In order to be able to transfer one whole volume automatically, we must have some means for accessing the files on a volume one by one. The example contains some file and directory handling words for this purpose. I have also left in there some words I needed in testing (like $openWD) which are not used in the actual implementation, but could be useful at times.

The Mach2 word $open opens a file on the default volume, given a name string, and returns a file ID number. In order to access files in any arbitrary folder (= HFS volume), we have to get a volume ID number first and set the default volume to this ID. The word promptvolID calls the standard file package to prompt the user to select a file in the folder that is to be transferred. Then the volume ID of this folder is returned. Calling setvol with a volume ID sets the default volume. getidxfile takes as its input a volume ID and an index n and will look for the n-th file in the volume. The filename is returned in a global parameter block, parblock, and can be used to open the file with $open after setting the correct default volume. send-batch simply scans the default volume for all files and transfers them one by one. So far, for text files only. Have fun implementing the MacBinary standard.

Reader Feedback

I got several comments on the floating point routines that we published lately. Ed Moskowitz from Winfield, IL points out correctly that the routines will not work with the new Mach2 since they use D5 and D6. This is correct, and D5 and D6 have to be added to the MOVEM.L save and restore list at the routine entry and exits to fix this bug.

Mike Morton from Cambridge, MA had some useful comments to make about improving the speed of those routines. Excerpts from his letter:

"• If you're doing lots of _Pack4 calls, you can do a GetTrapAddress and call the package directly, saving 30 to 50 microseconds ( )

• Instead of BTST #7, D0, you can do TST.B D0 and use BMI to see if the bit is on.

...

• when you AND #$7FFFFF to both D3 and D1 in the division routine, it's faster to move the mask to a register just once and use it twice. ( )

• When you're doing multiplications [and divisions, JL], you might optimize for cases where some of the number is zero. So your four MULUs might test whether the operands are zero and handle that case explicitly. I spent some time writing 32-bit fixed-point multiply routines a while ago, and this helped a lot because many numbers don't have any bits on the bottom of the longword. If you expect many of the f.p. numbers to not have many significant bits, this helps some.

( ) "

Thanks, Ed and Mike, for your comments.

NEON News

One letter came clear from Tasmania, from Phil Barnard:

" I am suffering from the new ROM syndrome at the moment, although the faster and more spacious drives are very pleasant after the original units. Having to replace the interrupt vectors for the interrupt button was a nuisance. As you are no doubt aware the new ROM/System does not present the familiar bomb-box, but an 'empty' dialog.

It may be that the new dialog can be used to resume in the same fashion as the old one, but I don't yet know how. G <return> sends it back into any loop that you may wish to exit. Looking at the register contents by typing A0 through D7 is of limited debugging use. Can it do anything else?"

[Why, I even installed a control panel with flashing LEDs and toggle switches on the top of my Mac. Why do you want anything else? ... Well, G 40F6D8, of course, you probably know by now that that is the address of _Exittoshell and will (sometimes) send you straight back to the Finder. I don't use it for many other things. JL]

"The following routine brings back the old familiar bomb-box on interrupt."

\ installation of custom interrupt vectors; Phil Barnard
\  ** INTERRUPT VECTORS **
\ prints out the seven vectors from the interrupt table
: get.ints ( -- ) cr
 $ 64 -base @ . cr \ VIA
 $ 68 -base @ . cr \ SCC
 $ 6C -base @ . cr \ VIA & SCC
 $ 70 -base @ . cr \ Debugging Button
 $ 74 -base @ . cr \     "
 $ 78 -base @ . cr \     "
 $ 7C -base @ . cr \     "

CREATE RTS $ 4E75 NEXT,

\ installs RTS in the last four (debugging) vectors in the interrupt 
table

: put.ints ( -- )
 $ 80 $ 70 do 'c RTS +base i -base ! 4 +loop get.ints ;

Thanks, Phil. I'll deal with more NEON stuff in the next column where I'll also have some time to write about the new release of NEON, version 2.0. Here only a few headlines:

• NEON 2.0 is fully HFS and Switcher compatible; a new class pathlist has been added to specify HFS search paths. The assembler supports the new ROM calls.

• Applications install as a one piece file, no need to keep a copy of the installed NEON kernel on the disk.

• A decompiler has been added that will handle simple definitions, classes, objects and methods.

• A number of bugs, including the famous IC! have been fixed.

More details about NEON 2.0 and other NEON things later. Till then, good holidays.

Listing 1: XMODEM batch file downloader


( Implementation of Modem7 file transfer protocol, 
 © 9/86 J. Langowski  for MacTutor )

only forth also mac also i/o also assembler
decimal

60 user fileID

10 constant maxerr   0 constant nul24 constant can
4 constant eot   6 constant ack  21 constant nak
1 constant soh   26 constant ctrl-z
117 constant bdnmch

" Select file on volume to upload:" constant seldirtext

variable time-ct variable nak-ct variable cksum    
variable filename 128 vallot
variable xbuffer 128 vallot
variable replyrecord 74 vallot

header parblock 12 allot  ( 12 bytes junk )
header iocompl   4 allot  ( at PB + 12 )
header iores     8 allot  ( at PB + 16 )
header ioref#    4 allot  ( at PB + 24 )
header iomisc    4 allot  ( at PB + 28 )
100 allot ( some elbow room )

: long-timeout  1000 time-ct ! ; (  17 sec timeout )
: short-timeout 100  time-ct ! ; ( 1.7 sec timeout )

: baud-rate ( - )
 1CC0A COMM1 mode  ( 9600 BAUD Xon/Xoff )
 ABORT" Serial Driver Error" ;  (8 bit, No Par, 2Stop)
 
: modout comm1 output emit console output ;
: fin can modout ;  : fini fin close-file ;

: timed-read { | tstart -- char }
 call tickcount -> tstart
 comm1 input
 begin
 ?terminal if key console input exit then
 call tickcount tstart -  time-ct @ > 
 until 
 console input  -1
;

: ?contrlx ?terminal 
 if key 24 = 
 if fini 1 abort" Control-X abort" then
 then 
;

: wait-ack { | errs -- }
 0 -> errs 
 begin  errs maxerr >  nak-ct @ maxerr > or
 if fini 1 abort" Max error count exceeded." then
 ?contrlx 
 timed-read case
 -1  of errs 1+ -> errs ." Timeout" cr endof
 can of 1 abort" Remote Cancel" endof
 ack of 0 nak-ct !  exit endof
 nak of 1 nak-ct +! exit endof
 ( disp-chr 0 )
 endcase
 again
;

: send-setup
 long-timeout
 standard-getfile  0= 
 if fin 1 abort" Cancelled." then
 0 nak-ct !
;

: read-sect { sector# | sect-start -- }
 sector# 128 * -> sect-start
 128 0
 do i sect-start + virtual c@ i xbuffer + c! loop
;

: send-filename
 ack modout 0 cksum c!
 12 1 do filename i + 
 c@ 127 and dup 95 > if 32 - then
 dup modout dup emit
 cksum c@ + cksum c!
 wait-ack
 loop cr
 ctrl-z modout 
 cksum c@ ctrl-z + cksum c!
;
 
: send-hdr { sector# | -- }
 soh modout
 sector# 1+ 255 and dup modout 255 xor modout
 ." Sending " sector# . cr
;

: send-sect ( send sector in xbuffer via XMODEM )
 0 ( sum )
 128 0 do xbuffer i + c@ dup modout + 255 and loop
 cksum c!
;
 
: send-cksum cksum c@ modout ;

: end-send
 close-file
 begin cr ." Sending EOT -" eot modout wait-ack
 nak-ct @ 0= until
 ." ack'd"
;

: get-file-length ( -- sectors )
 fileID w@ ['] ioref# w!  0 ['] iocompl !
 ['] parblock call geteof . cr
 ['] iomisc @ 128 /mod swap 
 if 1+ then
 dup ." File has " . ."  sectors" cr 
;
 
: send ( current file via XMODEM, closes file on exit )
 get-file-length 0
 do
 begin
 i read-sect ( into xbuffer )
 i send-hdr send-sect send-cksum
 wait-ack nak-ct @ 0=
 until
 loop
 end-send
;

: prelude
 short-timeout  begin timed-read -1 = until
 long-timeout  ." Awaiting initial NAK: "
 wait-ack cr   short-timeout
;

: send-one send-setup prelude send ;

: end-batch ack modout eot modout ;

( *** file and directory  handling routines *** )

: getVolID { index | pb -- VRefNum / errcode 0 }
 ( ID for volume no. index )
   ['] parblock -> pb
 0 ['] iocompl !
 filename pb 18 + !
 index pb 28 + w!
 pb call getvolinfo
 ?dup if 0 else pb 22 + w@ then
;

CODE SFGetFile  ( x y prompt numTypes typeList ReplyRecord )
 MOVE.L (A6)+,A1 ( reply record )
 MOVE.L (A6)+,A0 ( type list )
 MOVE.L (A6)+,D3 ( numTypes )
 MOVE.L (A6)+,D2 ( prompt )
 MOVE.L (A6)+,D1 ( y-coord )
 MOVE.L (A6)+,D0 ( x-coord )

 MOVE.W D0,-(A7)
 MOVE.W D1,-(A7) ( where )
 MOVE.L D2,-(A7) ( prompt )
 MOVE.L #0,-(A7) ( no filter proc )
 MOVE.W D3,-(A7) ( num Types )
 MOVE.L A0,-(A7) ( type list )
 MOVE.L #0,-(A7) ( no dialog hook )
 MOVE.L A1,-(A7) ( reply record )

 MOVE.W #2,-(A7)
 _Pack3
 RTS
END-CODE

: promptvolID  ( -- VRefNum flag )
 100 50 seldirtext -1 0 replyrecord 
 SFGetfile
 replyrecord 6 + w@ l_ext
 replyrecord w@ l_ext
;

CODE SFPutFile
 MOVE.L (A6)+,A1
 MOVE.L (A6)+,A0
 MOVE.L (A6)+,D3
 MOVE.L (A6)+,D2
 MOVE.L (A6)+,D1
 MOVE.L (A6)+,D0

 MOVE.W D0,-(A7)
 MOVE.W D1,-(A7)
 MOVE.L D2,-(A7)
 MOVE.L D3,-(A7)
 MOVE.L A0,-(A7)
 MOVE.L A1,-(A7)

 MOVE.W #1,-(A7)
 _Pack3
 RTS
END-CODE

: $openWD { name | pb -- VRefNum / errcode 0 }
   ['] parblock -> pb
 0 ['] iocompl !
 name pb 18 + !
 0 pb 28 + w!
 pb 1 call HFSDispatch
 ?dup if 0 else pb 22 + w@ then
;

: closeWD { volume | pb -- rescode }
   ['] parblock -> pb
 0 ['] iocompl !
 volume pb 22 + w!
 pb 2 call HFSDispatch
;

: setVol { volume | pb -- res code }
   ['] parblock -> pb
 0 ['] iocompl !
 0 pb 18 + !
 volume pb 22 + w!
 pb call setVol
;
 
: getidxfile { volume index | pb -- res code }
   ['] parblock -> pb
 0 ['] iocompl !
 filename pb 18 + !( ioNamePtr )
 volume pb 22 + w! ( ioVRefNum ) 
 0 pb 26 + w!  ( ioFVersNum )
 index pb 28 + w!( ioFDirIndex )
 pb call getfileinfo 
 dup 0= if ." Found: " filename count type cr then
;

: openidxfile { volume index | -- res code }
 volume index getidxfile
 ?dup 0= if filename $open then
;

: find-files { | volume index -- }
 promptvolID 
 if -> volume 1 -> index
 begin 
 volume index getidxfile
 index 1+ -> index
 until
 else ." cancelled" . cr 
 then
;
( *** main definitions for batch file transfer *** )
: display-filename
 12 1 do filename i + c@ 127 and dup 95 > if 32 - then emit
 loop cr
;  
: send-batch{ volume | index }
 1 -> index  0 nak-ct ! 
 begin cr
 filename 1+ 11 32 fill
 volume index getidxfile 0= 
 while
 volume setvol drop 
 filename $open dup 0> 
 if   fileID w!
 begin 
 prelude send-filename
 timed-read cksum c@ <> 
 while 
 ." Checksum Error in filename, restarting" cr
 bdnmch modout
 repeat
 ack modout
 send ( closes file )
 else drop 
 then
 index 1+ -> index
 repeat
 wait-ack end-batch
;
: upload 
 promptvolID
 if send-batch 
 else cr ." Cancelled" cr
 then
;
 

Community Search:
MacTech Search:

Software Updates via MacUpdate

iDefrag 5.0.0 - Disk defragmentation and...
iDefrag helps defragment and optimize your disk for improved performance. Features include: Supports HFS and HFS+ (Mac OS Extended). Supports case sensitive and journaled filesystems. Supports... Read more
PCalc 4.2 - Full-featured scientific cal...
PCalc is a full-featured, scriptable scientific calculator with support for hexadecimal, octal, and binary calculations, as well as an RPN mode, programmable functions, and an extensive set of unit... Read more
FileZilla 3.10.2 - Fast and reliable FTP...
FileZilla (ported from Windows) is a fast and reliable FTP client and server with lots of useful features and an intuitive interface. Version 3.10.2: Note: Now requires a 64-bit Intel processor.... Read more
The Hit List 1.1.11 - Advanced reminder...
The Hit List manages the daily chaos of your modern life. It's easy to learn - it's as easy as making lists. And it's powerful enough to let you plan, then forget, then act when the time is right.... Read more
Bartender 1.2.32 - Organize your menu ba...
Bartender lets you organize your menu bar apps. Features: Lets you tidy your menu bar apps how you want. See your menu bar apps when you want. Hide the apps you need to run, but do not need to... Read more
ClamXav 2.7.5 - Free virus checker, base...
ClamXav is a free virus checker for OS X. It uses the tried, tested, and very popular ClamAV open source antivirus engine as a back end. I hope you like and use ClamXav a lot and that it helps keep... Read more
xScope 4.1.2 - Onscreen graphic measurem...
xScope is powerful set of tools that are ideal for measuring, inspecting, and testing on-screen graphics and layouts. Its tools float above your desktop windows and can be accessed via a toolbar,... Read more
MacFamilyTree 7.3.3 - Create and explore...
MacFamilyTree gives genealogy a facelift: it's modern, interactive, incredibly fast, and easy to use. We're convinced that generations of chroniclers would have loved to trade in their genealogy... Read more
Skype 7.5.0.738 - Voice-over-internet ph...
Skype allows you to talk to friends, family and co-workers across the Internet without the inconvenience of long distance telephone charges. Using peer-to-peer data transmission technology, Skype... Read more
PushPal 3.0 - Mirror Android notificatio...
PushPal is a client for Pushbullet, which automatically shows you all of your phone's notifications right on your computer. This means you can see who's calling or read text messages even if your... Read more

Dungeon Hunter 5 Coming on March 12
Gameloft has excitedly announced that Dungeon Hunter 5 is on its way! Once again, you will adventure across the land of Valenthia exploring dungeons and fighting monsters. The game will have a new asynchronous multiplayer mode called Strongholds... | Read more »
It Came From Canada: Jurojin: Immortal N...
At this point it’s pretty safe to say that no MOBA is going to dethrone Dota 2 and League of Legends anytime soon. After all, if Batman can’t do it, nobody can. However, with a genre as popular and profitable as this one, there’s still room for... | Read more »
Tiny Farm – Animals, Tractors and Advent...
Tiny Farm – Animals, Tractors and Adventures! Review By Amy Solomon on March 3rd, 2015 Our Rating: :: LIVELY LANDSCAPESUniversal App - Designed for iPhone and iPad Tiny Farm – Animals, Tractors and Adventures! includes farm-related... | Read more »
This Week at 148Apps: February 23-27, 20...
Final February Fun at 148Apps   How do you know what apps are worth your time and money? Just look to the review team at 148Apps. We sort through the chaos and find the apps you’re looking for. The ones we love become Editor’s Choice, standing out... | Read more »
GDC 2015 – Does Not Commute is Definitel...
GDC 2015 – Does Not Commute is Definitely a Game You Should Keep an Eye on Posted by Rob Rich on March 2nd, 2015 [ permalink ] We were teased about Mediocre Games’ (Smash Hit, | Read more »
F84 Games & POW! Announce Stan Lee V...
F84 Games has announced that it is working with legendary comic creator Stan Lee and POW! Entertainment to produce Stan Lee’s Hero Command. The game will be a action adventure of heroic proportions. | Read more »
Setlyst Keeps Your Set Straight So You C...
Setlyst Keeps Your Set Straight So You Can Focus On Rocking Out. Posted by Jessica Fisher on March 2nd, 2015 [ permalink ] Universal App - Designed for iPhone and iPad | Read more »
Space is Vast, So Space Agency Has a Vas...
Space is Vast, So Space Agency Has a Vast New Update! Posted by Jessica Fisher on March 2nd, 2015 [ permalink ] Universal App - Designed for iPhone and iPad | Read more »
Size DOES Matter Review
Size DOES Matter Review By Campbell Bird on March 2nd, 2015 Our Rating: :: HARD TO BEATUniversal App - Designed for iPhone and iPad This rhythm game has a unique control scheme and performance system that make it feel like a true... | Read more »
The first ever action 3D card battler Al...
On the other hand, you probably haven’t played an action 3D card battler – until now. Step forward, All Star Legion. All Star Legion is a 3D QTE-based action RPG card battler, but fear not – the game itself isn’t as convoluted as its description.... | Read more »

Price Scanner via MacPrices.net

Another Tranche Of IBM MobileFirst For iOS Ap...
IBM has announced the next expansion phase for  its IBM MobileFirst for iOS portfolio, with a troika of new apps to address key priorities for the Banking and Financial Services, Airline and Retail... Read more
Sale! 15-inch Retina MacBook Pros for up to $...
B&H Photo has the new 2014 15″ Retina MacBook Pros on sale for up to $250 off MSRP for a limited time. Shipping is free, and B&H charges NY sales tax only: - 15″ 2.2GHz Retina MacBook Pro: $... Read more
WaterField Designs Introduces the Minimalist...
With Apple Pay gaining popularity, Android Pay coming in May 2015, and loyalty cards and receipts that can be accessed from smartphones, San Francisco’s WaterField Designs observes that it may be... Read more
Sale! 15-inch 2.2GHz Retina MacBook Pro for $...
 Best Buy has the 15″ 2.2GHz Retina MacBook Pro on sale for $1774.99 $1799.99, or $225 off MSRP. Choose free home shipping or free local store pickup (if available). Price valid for online orders... Read more
13-inch 2.5GHz MacBook Pro (refurbished) avai...
The Apple Store has Apple Certified Refurbished 13″ 2.5GHz MacBook Pros available for $170 off the cost of new models. Apple’s one-year warranty is standard, and shipping is free: - 13″ 2.5GHz... Read more
13-inch 2.5GHz MacBook Pro on sale for $100 o...
B&H Photo has the 13″ 2.5GHz MacBook Pro on sale for $999.99 including free shipping plus NY sales tax only. Their price is $100 off MSRP. Read more
27-inch 3.5GHz 5K iMac in stock today and on...
 B&H Photo has the 27″ 3.5GHz 5K iMac in stock today and on sale for $2299 including free shipping plus NY sales tax only. Their price is $200 off MSRP, and it’s the lowest price available for... Read more
Apple Launches Free Web-Based Pages and Other...
Apple’s new Web-only access to iWork productivity apps is a free level of iCloud service available to anyone, including people who don’t own or use Apple devices. The service includes access to Apple... Read more
Survey Reveals Solid State Disk (SSD) Technol...
In a recent SSD technology use survey, Kroll Ontrack, a firm specializing in data recovery, found that while nearly 90 percent of respondents leverage the performance and reliability benefits of SSD... Read more
Save up to $600 with Apple refurbished Mac Pr...
The Apple Store is offering Apple Certified Refurbished Mac Pros for up to $600 off the cost of new models. An Apple one-year warranty is included with each Mac Pro, and shipping is free. The... Read more

Jobs Board

*Apple* Solutions Consultant - Retail Sales...
**Job Summary** As an Apple Solutions Consultant (ASC) you are the link between our customers and our products. Your role is to drive the Apple business in a retail Read more
*Apple* Pay Automation Engineer - iOS System...
**Job Summary** At Apple , great ideas have a way of becoming great products, services, and customer experiences very quickly. Bring passion and dedication to your job Read more
Sr. Technical Services Consultant, *Apple*...
**Job Summary** Apple Professional Services (APS) has an opening for a senior technical position that contributes to Apple 's efforts for strategic and transactional Read more
Event Director, *Apple* Retail Marketing -...
…This senior level position is responsible for leading and imagining the Apple Retail Team's global engagement strategy and team. Delivering an overarching brand Read more
*Apple* Pay - Site Reliability Engineer - Ap...
**Job Summary** Imagine what you could do here. At Apple , great ideas have a way of becoming great products, services, and customer experiences very quickly. Bring Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.