New Upgrades for MacForth & Mach2

By Jörg Langowski, MacTutor Editorial Board, Grenoble, France

MacForth Plus and Mach 2.1

This month I'd like to give you an overview of the most recent versions of the two Forth implementations for the Macintosh that you've been reading about most in this column. That is, Mach2 (version 2.1) and MacForth (Plus). NEON, making up a category on its own, I'll leave out this time.

Regular readers of this column will have noticed that the majority of Forth examples have been written in Mach2. This shouldn't be meant as a bias towards that particular system, only that I think it is better to stay consistent and use one implementation, pointing out differences to other Forths if they arise.

Let's start with a short description of MacForth Plus. Since it is a major change compared to the older versions, it helps to print some excerpts of the release notes by Creative Solutions included with the system.

"MacFORTH Plus has been completely rewritten from scratch...Most notably, in order to support multi-tasking the file system has changed dramatically. Other changes have been made to make MacFORTH Plus run much faster, for instance with an entirely new method for doing loops. The old system of using the 68000 Trap instruction and F-line traps for defining words has been abandoned in favor of a system that will work with the 68020 and the 68881 floating point coprocessor. This new arrangement is also much faster, but is not nearly as compact as the old system. Thus, the kernel is quite a bit larger.

You will still be able to write applications that will run on 128K machines, but it is presumed that for your development environment you use at least a 512K machine.

While block files are still fully supported, MacFORTH Plus also provides the ability to write programs in ordinary text files and provides an editor as well. Proper use of the editor could change your programming environment completely, as you can execute a selection range by hitting the Enter key. Output will be directed into the file itself. [looks very similar to the MPW environment in that aspect - JL]. This means you can easily accumulate a lot of information in the file (e.g., by executing WORDS) and read it at your leisure. If you do all your work in this way you'll also have a log file recording exactly what you did.

If you use the system window to input commands, there's a command line editor. It saves the last four commands you typed and can be invoked at any time to allow you to edit or re-use them.

Errors are reported by alert boxes in true Macintosh style.

You can enable and disable the programmer's switch with EnableSwitch and DisableSwitch. This cannot be guaranteed to work on all future Macintosh products, however.

MacFORTH Plus does multi-tasking.

A nearly complete set of words is provided for running alerts, modal dialogs, and modeless dialogs.

Words to use resource templates in defining menus, menubars, controls, and windows have been added for those who prefer this method.

The way DO ... LOOP works internally has been changed dramatically. It's twice as fast as it used to be. There is some cost for the additional speed, however. First and most important, I is no longer the same as R@. You should not use I or any of its relatives (like IC@, I+, J, etc.) unless you are referring to a loop index. The loop index is no longer kept on the return stack, but is now in a register.

Some people will have some rewriting to do to get their old code to work because of this. You must be sure you have used I and R@ correctly...Second, since the new loops use registers D4, D5, and D6, these are no longer available for your own use.

And finally, the new loop system works somewhat differently in extreme cases...This is because the new system checks for the index crossing the limit rather than checking how the index is related to the limit. (This is actually the method recommended in the Forth 83 standard.)

In order to properly support the multi-tasking, it was necessary to drop the old FCB system and install a file interface like the Pascal interface.

There are also 3 new variables for I/O redirection: INFILE, OUTFILE, and DEBUGFILE. Input is taken from INFILE; most output goes to OUTFILE, except error reports, which go to DEBUGFILE. Any of these may be a file, a MacForth wptr, or a device driver.

Code fields are now 4 bytes long instead of 2. Register allocation has completely changed. Vocabularies are set up differently, as is the token table (tokens now 6 bytes each). The OBJ resource now includes both the old object and the token table (so ROOM will report much larger values -- it includes the token table). Most of these changes will impact only very advanced users.

One new feature, contributed by John Baxter, provides for a "Tools" Menu. Program development tools can reserve space on and use this common menu rather than allocating a new menu for each tool. This has already proven to be almost indispensible with the expanding number of advanced tools being uploaded to Compuserve, and available through the MacForth User's Group.

Another feature, provided by Tim Hewitt, and now included in the default snapshot token, includes any TEXT files that were double-clicked on the Finder desktop, if the Option key is held down when the files are launched. This is particularly useful for turnkey scripts,.... Macintalk support is included.

Calling procedures and functions written in other Languages:

This effort is currently in process, (examples are working fine), but we are not sure about the format we should use for the interface (Linking templates, etc.), and we would like to know what formats end-users would like to see (eg. MDS or MPW?). We'll upload the results to Compuserve when ready."

So much for the new features of MacForth Plus as described by CSI. Let's now proceed by a more detailed comparison of the two Forth systems.

Speed

Execution speed is not everything, but it matters a lot. Less if you use a lot of toolbox calls; more if you need to write time-critical code, as for instrument control, or in heavy number-crunching.

Listing 1 shows some benchmarks that I used on the two Forth systems: a routine that inverts the screen 50 times, the well-known Sieve of Erastothenes, one million empty DO...LOOPs, and a speed test of the floating point implementation. The words counter and timer are not implemented in MacForth and are therefore redefined here.

The figures - given in Table 1 - immediately show that MacForth Plus is a great leap forward compared to the MacForth K2.4 version. Mach2, however, since it offers the distinct advantage of generating 68000 code directly instead of going through an 'inner interpreter' as many other Forths, still executes twice as fast (on the average) as MacForth Plus.

The discrepancy between the two systems is evident for the floating point benchmark, with Mach2 executing SANE routines 5-10 times faster than MacForth. It is not clear to me why the floating point routines under MacForth should generate so much overhead in an empty loop, but the fact remains. The MacForth floating point words themselves (after subtracting the loop overhead) execute not too much slower than the Mach2 versions; still, there is a constant difference of approx. 250 µs per floating point operator.

The advantage of the MacForth floating point system is that rounding and exception trapping are under very good control from your program. In this aspect, Mach2 does not implement the full SANE package.

For faster numerics, there exists a good 32-bit floating point package for MacForth in the public domain (Compuserve or MacForth Users' Group); for Mach2, see my columns in MT V2#7 and V2#8 for 32-bit floating point routines.

It is not only execution speed that matters but also loading speed. Both systems are very fast in loading long files. Although I don't have one long program that will load on both systems, I tested Mach2 and MacForth Plus with several files of comparable length and got an average loading speed of about 1.5 Kbyte/sec for Mach2 and 0.8 Kbyte/sec for MacForth Plus (both from an SCSI hard disk). Mach2.1 seems to have the edge here because of its new hash-coded vocabulary structure; switching off hashing lets Mach2 load at the same speed as MacForth Plus.

Debuggers

MacForth has had a built-in debugger from the very start, and, of course, MacForth Plus has kept it. Its TRACE option will insert a trace token before each compiled word, so that on execution (with the DEBUG feature on) the name of the word executed will be printed together with the current stack picture. DEBUG output can be redirected to a debug file [this is new], in order to leave the screen output undisturbed.

The stack picture that is displayed with each word helps a great deal in analyzing algorithms that do a lot of stack manipulation; one can see very quickly whether a routine left something on the stack that wasn't supposed to be there, or whether an odd address is generated by some bug, etc.

The MacForth debugger does not really decompile the code since you have to execute it to get it displayed. For decompilation, however, there exist a number of good tools in the public domain. A Forth decompiler included with the system would be nice, though, especially since the structure of MacForth Plus definitions has changed a lot from the older MacForth versions.

Debugging under Mach2 is different, due to the different implementation that directly generates machine code. Therefore, the debugger (included in the v2.1 release) resembles Macsbug a lot. You can enter the debugger by selecting a menu option or executing DEBUG. This will give you a Macsbug-like window that accepts Macsbug-like commands. In addition, you can refer to Forth words by their names within the debugger, and when code is disassembled, the debugger displays the names of Forth words corresponding to the code. JSR-threaded words are decompiled without problems this way; if inline code has been included using the MACH option, only the most standard definitions like swap, drop, literals etc. are recognized.

Tracing code under Mach2 is more cumbersome than under MacForth; you have to set machine-level breakpoints (DEBUG <word> sets a breakpoint at the beginning of the definition of <word>) and then step through the code. Evidently, it is not so easy to monitor the stack as with MacForth - the debugger displays no stack pictures.

The interrupt switch for entering the debugger did not work in my version of Mach2.1, and Palo Alto Shipping is working on it; they promised to have it fully functional Real Soon Now (to steal an expression from one of my more famous colleagues).

My preferred mode of operation is to debug Mach2 code with TMON. Forth words are not displayed by name there, true; but all the other tools are just so much more powerful.

For testing out algorithms, I prefer MacForth with its stack display on each step while tracing code. You see immediately where you've been stupid in your coding.

Conclusion here: it's really best to have Mach2 and MacForth ready for debugging a bigger Forth project.

Assemblers

Both MacForth Plus and Mach2 include an assembler. While MacForth Plus uses the Forth philosophy of writing assembly code in reverse Polish notation, Mach2 conforms to the standard Motorola syntax.

I suppose one can get used to both assemblers. The reason why I am using the Mach2 assembler in my examples is very simple; I still retain hopes that someone from the non-Forth speaking community might find at least some of the assembly language stuff interesting (and that way gain interest in using Forth as well). Seriously, it becomes much easier to transfer code between different programming environments if at least for assembly language, one sticks to the standard Motorola syntax.

The 'forthy' MacForth assembler has the advantage of having a set of control structures like if, else, then, begin, while, repeat, etc. built in. That, again, makes machine code more readable, but raises problems when one deliberately wants to create tight 'spaghetti' code (yuck). This is often necessary to squeeze the last bit of performance out of a piece of code, and one can generate such 'unstructured' code in MacForth by playing some tricks that are explained in the manual, but the result will be even less readable than in standard assembler using labels.

Summary: if your program includes large pieces of assembly code, I suggest using Mach2, especially if you've been using 68000 assembly in other environments. For small machine code insertions, both systems are probably equivalent.

Local variables and other data structures

A great deal of time-consuming and bug-breeding stack manipulation can be avoided by using local variables and named stack parameters in Forth words.

Both Mach2 and MacForth offer these options. The syntax is not too different between the two systems. In Mach, the word name (after the colon) is followed by a parameter list enclosed in braces with a vertical bar separating the stack input parameters from the local variables:

: test { a b c | loc1 loc2 loc3 -- comment }   ;

while in MacForth the same is accomplished by

: test  0 1 2 locals| loc3 loc2 loc1 c b a |   ;

initializing loc1, loc2 and loc3 to 0, 1 and 2 at the same time. This syntax can be emulated in Mach2 by writing

: test  0 1 2 { a b c loc1 loc2 loc3  }   ;

which makes the two approaches look very similar.

Local variables (including stack parameters) are limited to a maximum of 10 in MacForth Plus, while apparently no restriction exists in Mach2. In the latter case, a stack frame is generated on a local variable stack (A2) that holds four bytes for each variable.

Calling the name of a local variable puts its value on the stack; n -> loc1 (Mach2) or n to loc1 (MacForth) stores n into the local. Sometimes it becomes necessary to pass the address of a local variable to a toolbox routine; in MacForth this is accomplished by writing addr.of loc1, and in Mach2 by ^ loc1.

MacForth also includes words for one- and two-dimensional array definitions and for data structures. These definitions are left as an exercise to the user in Mach2. See MT V2#9 on an alternate way of defining structures both in MacForth and Mach2.

Toolbox interface

The different philosophy of the implementers of MacForth Plus and Mach2 shows up very clearly in the way the Macintosh user interface is handled. While in both systems the most frequent elements of the user interface (windows, menus, controls) are supported by high-level Forth words, MacForth offers in addition, words for handling dialogs, alerts, text edit, events, scrap and the drawing routines. This is again consistent with the philosophy of keeping the syntax as Forth-like as possible, and sometimes makes the code more readable. However, it can create problems when transporting algorithms from other programs written in C or Pascal, where the toolbox routines are called more or less according to the Inside Macintosh standard.

Mach2, in addition to supporting windows, menus and controls from within Forth (and linking these structures to the multi-tasking environment), gives a very simple way to directly call toolbox routines Inside Macintosh style. This is done by writing

CALL <routine>

with the appropriate number of parameters on the data (A6) stack. Mach2 contains a table of all toolbox routines and most of the package calls together with information on how many parameters in which format are expected on the stack. The compiler automatically inserts glue code to take the parameters off the A6 stack (each one occupying 4 bytes here) and pushing them on the A7 stack in the correct format.

For toolbox calls not contained in the list (there are not many), or for calling an operating system routine asynchronously, one has to write a little piece of assembly code to transfer the parameters correctly. Examples of this can be found in almost any previous Forth column that deals with Mach2.

The difference is clear: toolbox calling very 'close to the machine' in Mach2, higher level Forth-style toolbox support in MacForth. Otherwise the two systems are equivalent, and it is again a matter of preference which one to use. For machine-level debugging, the subroutine-threaded approach of Mach2 is easier; on the other hand, with MacForth it might be easier to transport code to other window-oriented environments that do not use exactly the same toolbox calls as the Macintosh. The problem with the MacForth approach, however, is that one has to remember a great number of new Forth words for toolbox calls which are known by other names in IM.

Mach2, on the other hand, keeps the IM names (with some exceptions), and the handbook contains a very nice quick- reference overview about all the toolbox traps with their stack parameters which replaces IM in 75% of the cases (the others being mostly answered by WinTool).

Modeless dialogs, which were not supported in earlier Mach2 versions, can now be used.

Multitasking

MacForth now offers multi-tasking support, just as Mach2 had from its beginning. The source code of the multi-tasking routines is included in MacForth, but not in Mach2.

At first glance, multitasking is very similar in the two systems. Words are provided to define tasks and to set their memory allocations; tasks are activated by the word activate (Mach2) or activate< (MacForth), followed by the code the task is to execute. Task switching is initiated by calling PAUSE from within the task code. Some Mach2 and MacForth words (like input/output, GET and RELEASE) contain a PAUSE in their code; this is clearly documented in the glossaries.

In Mach2, a task can be automatically linked to a window and a menu bar; this greatly simplifies task handling. The Mach2 user variable area contains task-specific vectors, for example, for input/output or event handling, which may also be altered by other tasks.

MacForth's multitasking system offers the advantage of being able to delay a task for a specified number of ticks. The variable delay is accomplished by a VBL task associated with each Forth task, which wakes up the Forth task after the delay has expired.

In Mach2 one would still have to install such a feature, which shouldn't be very complicated; however, having it built in would make a useful addition.

Editing

For a long time, MacForth has forced users to stick with a block-style editor, almost like simple CP/M Forth systems. I suppose I'm not the only one who didn't understand why they wouldn't accept standard text files as an input. The main problem with the block files was that it was so complicated to insert things in the middle of a program that one was tempted to fill up the available block space to the limit of incomprehensibility of the code.

Now, CSI seems to have got their act together and has provided the MacForth Plus user with a four-window (maximum) integrated editor, the source code of which is provided. It offers everything that a good standard program-text editor for the Mac should have, and in addition interesting goodies like display of page breaks in the text and a pop-up window to display file size and line numbers when using the vertical scroll bar.

The MacForth Plus editor is an excellent product, and one can only hope that the editor that was promised with the Mach2 system (Real Soon Now) is comparably good. One very important feature is that by selecting a range of text and pressing ENTER you can immediately execute the selection range (i.e. compile a definition or execute some code); the result of the execution is displayed in the edit window, thus becoming part of the edited text. Very much like the MPW worksheet windows and extremely useful. Of course, the old block style editor is still provided so that older MacForth code does not become useless.

Mach2 does not yet offer an integrated editor. One either has to use the Transfer option on one of their menus to get into Edit (bypassing the Finder) or use a desk accessory editor such as miniWRITER or MockWrite. The editor task written by Juri Munkki (MT V2#11) is another possibility. Palo Alto shipping is still working on the editor as I write this; I hope the Mach2 update with the editor will be available when you read my column.

Resources

Resources that are used by your Forth code are kept in an extra file named resource.bag in MacForth Plus and MACH.RSRC in Mach2. That resource file will automatically be opened when the Forth system starts up, and the resources will be accessible by their types and IDs.

Stand-alone applications

Both systems include a TURNKEY feature that will generate a stand-alone application out of your Forth code. Mach2 will automatically copy all resources from the MACH.RSRC file into the application. In MacForth, the user has control over which resources from the resource.bag file will be included in the final application; this is accomplished by a resource list which contains the handles of the resources to be copied.

One last thing that must not be forgotten is that Mach2 is the only Forth system available for the Macintosh which allows you to program things other than applications in Forth, such as INIT routines, VBL tasks, desk accessories, etc. etc. It is even conceivable (and planned, as far as I know) to be able to link routines written in other languages to the Forth code and vice versa.

Programming examples

This is an important point. Nothing helps better to get used to a new development system than lots and lots of programming examples. Both Mach2 and MacForth offer a large variety, in fact they supplement each other; I often found useful concepts for one system in examples from the other one. It would be desirable to have more of Mach2's source code available, as MacForth has (NEON, too, is a master example of a well documented system source).

Conclusion

I won't give a recommendation for a specific Forth system. If your main issue is speed, you might be better off with Mach2, especially if your code uses a lot of floating point calculations. Also, if you want to write a desk accessory or a window definition routine in Forth, Mach2 is the answer.

MacForth seems somewhat more flexible in the way resources can be handled and has a very good built-in editor. Its assembler will definitely appeal to die-hard Forth purists who want to do everything the reverse Polish way. The trap support is good with both systems, Mach2 staying closer to Inside Mac.

If you plan to develop a large project in Forth, I suggest getting both systems....if only for the examples.

Next column I'll take up desk accessory programming again, with some useful tricks and caveats.

Listing 1: Forth benchmarks used in this column
( Screen invert, Mach2 and Mac+ version)
only forth also assembler also sane
501504 524288 + constant screenlow ( Mac Plus)
5472 constant screenwords
screenlow screenwords 4 * + constant screenhigh

code i@
    move.l  d6,a0
    move.l (a0),-(a6)
    rts
end-code
mach

code i!
    move.l  d6,a0
    move.l (a6)+,(a0)
    rts
end-code
mach

code i+
    add.l   d6,(a6)
    rts
end-code
mach

code 3+
    addq.l  #3,(a6)
    rts
end-code
mach

code center
    move.l  d6,a0
    move.l  (a0),-(a6)
    not.l   (a6)
    move.l  (a6)+,(a0)
    rts
end-code
mach

: invert screenlow 
 begin dup @ not over ! 4 + dup screenhigh = until drop ;
: test counter 50 0 do invert loop timer ;

: invert2 screenhigh screenlow do i @ not i ! 4 +loop ;
: test2 counter 50 0 do invert2 loop timer ;

: invert3 screenhigh screenlow do i@ not i! 4 +loop ;
: test3 counter 50 0 do invert3 loop timer ;

: invert4 screenhigh screenlow do center 4 +loop ;
: test4 counter 50 0 do invert4 loop timer ;

( screen inverter, definitions for MacForth, slightly altered )
anew bench
assembler
501504 524288 + constant screenlow ( Mac Plus)
5472 constant screenwords
screenlow screenwords 4 * + constant screenhigh
: SHOW.TIME  ( ticks -- )
    dup cr .
    ." ticks     "
    100 60 */ <# # # ascii . hold #s #> type
    ."  seconds" ;
: counter tickcount ;
: timer tickcount swap - show.time ;

CODE bnot 
 d0 get, d0 long not, d0 put, next 
END-CODE

: invert1 screenlow
  begin dup @ bnot over ! 4 + dup screenhigh = until drop ;
: test tickcount 50 0 do invert1 loop
  tickcount swap - show.time ;

: invert2 screenhigh screenlow do i @ bnot i ! 4 +loop ;
: test2 tickcount 50 0 do invert2 loop
  tickcount swap - show.time ;

: invert3 screenhigh screenlow do i@ bnot i! 4 +loop ;
: test3 tickcount 50 0 do invert3 loop
  tickcount swap - show.time ;

: invert4 screenhigh screenlow
  do  >CODE
   d6 a0 long move,
   d5 a0 long adda,
   a0 () long not,
   >FORTH
  4 +loop
;
: test4 tickcount
  50 0 do invert4 loop tickcount swap - show.time ;

( Eratosthenes Sieve Benchmark, stack version )                      
          
 8192 constant size       
variable flags  size vallot     ( ALLOT for MacForth Plus)         
: primes   flags  size 1 fill  ( empty array )                
  0 ( prime counter )   size 0  ( range )                       
    do  flags  i+ c@                                          
      if   i 2* 3+  dup  i+  size <  ( avoid known nonprimes)  
         if  size flags +  over i+  flags +                     
             do  0 i c!  dup  +loop  ( flick mod prime flags) 
         then     drop 1+  ( another prime )                    
       then                                                     
    loop                                                        
;                                            
                                                                
: sieve 10 0 do  primes loop ;    
: sieve.demo counter sieve3 timer ;
                                                                
( Eratosthenes Sieve Benchmark, local variable version )
: prime3 { | #primes  prime*2+3  limit -- } 
 ( note different syntax for MF+)
 ( note also that i + should be replaced by i+ etc. in MF+ )
   flags  size 1 fill                         
   flags size + -> limit  0 -> #primes 
   
   limit 1+  flags  
   do  i c@                                     
     if i flags - 2* 3 + dup -> prime*2+3       
       i + limit <                              
          if limit 1+   prime*2+3  i +          
             do 0 i c! prime*2+3 +loop  ( 0ic! is one word  in MF+ )
          then
        #primes  1+  -> #primes                
     then
   loop
   #primes .  ." primes " cr  ;

: sieve3 10 0 do prime3 loop ;
: sieve3.demo counter sieve3 timer ;

: million.loops
    counter 1000000 0 DO LOOP timer ;

( floating point benchmarks )
FP

: fmark1 pi 2.718281828e0 ." 10000 empty loops - "
         counter 10000 0 do fover fover fdrop fdrop loop timer 
         fdrop fdrop ;
: fmark2 pi 2.718281828e0 ." 10000 additions - "
         counter 10000 0 do fover fover f+ fdrop loop timer 
         fdrop fdrop ;
: fmark3 pi 2.718281828e0 ." 10000 subtractions - " 
         counter 10000 0 do fover fover f- fdrop loop timer  
         fdrop fdrop ;
: fmark4 pi 2.718281828e0 ." 10000 multiplications - "
         counter 10000 0 do fover fover f* fdrop loop timer  
         fdrop fdrop ;
: fmark5 pi 2.718281828e0 ." 10000 divisions - "
         counter 10000 0 do fover fover f/ fdrop loop timer  
         fdrop fdrop ;
: fmark6 2.718281828e0 ." 1000 square roots - "
         counter 1000 0 do fdup fsqrt fdrop loop timer  
         fdrop ;
: fmark7 2.718281828e0 ." 1000 sines - "
         counter 1000 0 do fdup fsin  fdrop loop timer  
         fdrop ;
: fmark8 2.718281828e0 ." 1000 logarithms - "
         counter 1000 0 do fdup fln fdrop loop timer  
         fdrop ;
: fmark9 2.718281828e0 ." 1000 exponentiations - "
         counter 1000 0 do fdup faln fdrop loop timer  
         fdrop ;

: fspeed.test cr
    fmark1 cr
    fmark2 cr
    fmark3 cr
    fmark4 cr
    fmark5 cr
    fmark6 cr
    fmark7 cr
    fmark8 cr
    fmark9 cr
;