TweetFollow Us on Twitter

Benchmarks 2
Volume Number:3
Issue Number:9
Column Tag:Mac Cad

Benchmarks Re-visited

By Paul Zarchan, Cambridge, Mass

With the emergence of the Mac 2 and the growing base of useful, easy to use scientific software, the field of desktop engineering will surely grow this year. The purpose of this article is to compare, from an engineering user point of view, the new Mac’s (using a Prodigy 4 as the equivalent of a Mac 2) with their counterparts in the IBM micro world, DEC mini world and IBM mainframe world. First the issue of compilation and linking will be addressed and then standardized benchmarks will be used to compare various machines from both a cost and performance point of view. Most of the non Mac results were provided to me by A. Tetewsky and D. Feenberg. These results will soon be published in Ref. 1.

Compiling and Linking

When using a compiled language for programming, such as FORTRAN, the issue of compile and link times is extremely important. In engineering applications, excessive compile and link times may make it worthwhile to develop engineering software in an interpretive language such as BASIC, and then port it to a compiled language after initial debugging and algorithm development have been completed. If switching languages may not be practical, it may be worthwhile to stay in FORTRAN but develop the engineering software on a computer with faster compilation times. After program development the source code can easily be ported to the computer of interest for final compilation.

Let’s consider an example in finding complex roots of real polynomials. The 144 lines of program source code for this example can be found in Ref. 2. This example, like that of the Butterworth example in Ref. 3, uses single precision arithmetic but unlike the Butterworth example has virtually no input/output code. In this root finding example, a solution is found for a 30th order, well-behaved polynomial. The compile and link times for the 144 lines of code, using MS FORTRAN (both in the Apple and non Apple world), are indicated in Table 1 for a variety of micros.

In this example, compilation and linking were done using a hard disk for the IBM AT and Compaq 386, while in the Macintosh world, compilation and linking were done in RAM. In the IBM world, compiling in RAM is not significantly faster than compiling from the hard disk. This will always be the case since the operating system software, DOS, is written for 64k segmented 8086/8088 processors. Although an operating system which is developed for the 80386 or OS/2 should be better and improve compilation times, it will not be available for at least one year. If history is any guide, the wait time may be significantly longer. In addition, due to memory segmentation and the lack of a FORTRAN editor (a word processor must be used), it may be difficult to fit all necessary engineering tools into RAM. In the Macintosh world, memory is linear and easily expandable with third party upgrades. For example a 512K Mac can be upgraded to 2 Megs for about $500. This permits the creation of a 1.5 Meg recoverable RAM disk which is large enough to fit FORTRAN and many other useful tools into RAM. Therefore, compiling in RAM with a Mac is much faster than compiling from a hard disk.

In addition, in the IBM world one must compile and link before the code can be executed. The user must nurse the computer through the compiling, linking and execution process. In the Macintosh world, linking is dynamic and therefore automatic from a user point of view. The user simply double clicks on “compile and execute” and the source code compiles, links and runs.

The execution time for this complex root finding example for a variety of micros appears in Table 2. In this example all the micros with the exception of the Mac Plus had math coprocessors.

The Table shows that, for this example, the Prodigy 4 is about 10 times faster than a Mac Plus, more than 5 times faster than an IBM AT and 2.5 times faster than a Compaq 386. In the IBM world, with the exception of the PC, the math coprocessor never seems to run at the same clock rate as the CPU. That is why for this example, an AT and PC (where the math coprocessor is matched to the CPU at 4.77 MHz) have similar execution times. The Compaq 386 is only twice as fast as the AT even though the Compaq has 32 bits rather than 16 bits and runs at 16 Mhz rather than 6 Mhz. In principal, when the IBM operating system software is written and a 16 MHz Intel 80387 math coprocessor becomes available, it should be in the same speed class as the Prodigy 4. Interestingly enough, the Compaq 386 is rated at 3.5 MIPs while the Prodigy 4 is only rated at 2.0 MIPs. We can see that in numerical applications, MIP ratings may not tell the whole story (see Ref. 4 for example).

Often the user may only be interested in the turn around time, which is the sum of the compile, link and execution times. For this example we can see by comparing Tables 1 and 2 that the turn around times are significantly better in the Macintosh world. Table 3 summarizes the results for the complex root example.

The sample problem only had 144 lines of FORTRAN code. If we consider a “traveling salesman” program using 1500 lines of FORTRAN code, the comparison of compile and linking times are even more dramatic. Table 4 shows that the Macintosh and Prodigy 4 are considerably faster for larger programs than either the IBM AT or Compaq 386.

Whetstone Benchmarking

The Whetstone benchmark, devised in England by Curnow and Wichman in the Feb. 1976 issue of the Computer Journal, is an attempt to cover a typical mix of all floating point operations. This benchmark contains linear arrays, and add, subtract, multiply, divide and transcendental operations. Whetstones were originally written in ALGOL, but later translated to FORTRAN in 1979 by D. Frank. Since that time, many computer manufacturers have rated their machines in terms of thousands of Whetstones per second or kw/sec. Higher Whetstone ratings mean more powerful machines. Table 5 presents single and double Whetstone ratings for a variety of micro, mini and mainframe computers. In addition, ratios referenced to Prodigy 4 speed are indicated in the Table. A ratio of 1.7 means that the computer is 1.7 times faster than the Prodigy 4. All computers, with the exception of the Mac Plus, have math coprocessors or floating point accelerators. The poor double precision Whetstone rating of the Mac Plus may, relative to the IBM PC, may be one of the reasons there has been a scarcity of scientific software for the Mac. Of course, we can see from this Table that the Prodigy 4 and hence new Mac 2 changes all that.

The Whetstone results of Table 5 (with no I/O) can be compared to the Butterworth simulation results( with considerable I/O and more representative of a realistic engineering application) of Ref. 3. Figure 1 shows that all the benchmarks, whether they be Whetstones or Butterworth simulations, yield about the same relative machine performance. Only the Mac Plus seems to yields results which are significantly benchmark dependent. It yields worse performance on the Whetstones because of it’s lack of a math coprocessor.

Figure 1 - Relative Machine Performance is Approximately Independent of Benchmark

The performance comparison of Fig. 1 can be placed into proper perspective when the cost of the host computer is considered. For simplicity, computer cost can be considered to be the machines purchase price only. This neglects the cost of the small army of technicians required to operate the larger machines and the cost of software leasing agreements. We can see from Fig. 2 that generally higher cost computers yield faster performance. However the cost is not always commensurate with the performance. For example, a VAX 11/780 is only 1.5 times as fast as a Prodigy 4 and yet is 40 times more expensive. An IBM 3084Q is 11.7 times faster than a Prodigy 4 and is 500 times more expensive. On the micro side an IBM RT is 2.5 times slower than a Prodigy 4 and yet costs twice as much.

Figure 2 - Micros are More Cost Effective Than Larger Machines

If we normalize the computer performance as measured by double precision whetstones per second to the computer purchase price we can generate “bang for the buck” information. More “bang for the buck” means that the computer yields a higher double precision Whetstone rating for less cost. Figure 3 presents this cost effectiveness information and shows that the Compaq 386, Prodigy 4 and Micro Vax 2 are very cost effective, with the Prodigy 4 yielding the most “bang for the buck”. The curve also indicates that if a micro can do the job, it is more cost effective from a performance point of view than a mainframe.

Figure 3 - Prodigy 4 Outperforms Every Other Computer

Summary

The intent of this article was to show that FORTRAN runs very efficiently on the Prodigy 4 (and hence Mac 2) when compared to non Apple micros. When compilation and linking times are taken into account, the comparison is even more dramatic. A relative performance curve is presented quantifying “bang for the buck” information for a variety of micros, minis and mainframes. As expected, the new Mac 2 appears to out- perform every other computer.

Acknowledgements

I wish to thank Micro/Systems, Av Tetewsky and Dan Feenberg for permitting me to extract from Ref. 1 the benchmark timings on all the non Apple machines and for providing the technical explanation for the “features” of the various DOS machines. In addition, I would like to thank Owen Deutsch, for providing me with the “travelling salesman” FORTRAN code.

References

1) Tetewsky, A. and Feenberg, D. “A Survey of 6 FORTRAN Compilers” to appear in Sept. 1987 edition of Micro/Systems Journal.

2) Press, N. H. et al, “Numerical Recipes The Art of Scientific Computation”, Cambridge University Press, 1986.

3) Zarchan, P. “New Mac Workstation Potential”, MacTutor, Vol. 3, No. 3, March 1987, pp 15-21.

4) Boston Computer Society IBM PC Report, “PC Technical Report: MIPs, MFlops, Benchmarks and Other Half-Truths”, May-June 1987.

5) Marshall, T., Jones, C., and Kluger, S. “Definicon 68020 Coprocessor”, BYTE, July 1986, pp 120-144.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Rodeo Stampede guide to running a better...
In Rodeo Stampede, honing your skills so you can jump from animal to animal and outrun the herd as long as possible is only half the fun. Once you've tamed a few animals, you can bring them home with you. [Read more] | Read more »
VoxSyn (Music)
VoxSyn 1.0 Device: iOS Universal Category: Music Price: $6.99, Version: 1.0 (iTunes) Description: VoxSyn turns your voice into the most flexible vocal sound generator ever. Instantly following even subtle modulations of pitch and... | Read more »
Catch Battleplans on Google Play from Ju...
Real-time strategy title Battleplans is due for release on Google Play on June 30th, following its release for iOS systems last month. With its simple interface and pretty graphics, the crowd-pleaser brings a formerly overlooked genre out for the... | Read more »
iDoyle: The interactive Adventures of Sh...
iDoyle: The interactive Adventures of Sherlock Holmes - A Scandal in Bohemia 1.0 Device: iOS Universal Category: Books Price: $1.99, Version: 1.0 (iTunes) Description: Special Release Price $1.99 (Normally $3.99) | Read more »
Five popular free apps to help you slim...
Thanks to retail and advertising, we're used to thinking one season ahead. Here we are just a week into the summer and we're conditioned to start thinking about the fall. [Read more] | Read more »
How to ride longer and tame more animals...
It's hard to accurately describe Rodeo Stampede to people who haven't seen it yet. It's like if someone took Crossy Roadand Disco Zoo and put them in a blender, yet with a unique game mechanic that's still simple and fun for anyone. [Read more] | Read more »
Teeny Titans - A Teen Titans Go! Figure...
Teeny Titans - A Teen Titans Go! Figure Battling Game 1.0.0 Device: iOS Universal Category: Games Price: $3.99, Version: 1.0.0 (iTunes) Description: Teeny Titans, GO! Join Robin for a figure battling RPG of epic proportions! TEENY... | Read more »
NinjAwesome: Tips and tricks to be a mor...
Sorry about that headline, but I'm going to go ahead and assume that GameResort would not have named its game NinjAwesome without expecting some of that. It is, in fact, pretty awesome the way it combines an endless runner and old school arcade... | Read more »
Into Mirror (Games)
Into Mirror 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: "Is all that we see or seem, but a dream within a dream?"- Edgar Allan Poe New game by Lemon Jam Studio, the team behind Pursuit... | Read more »
Perchang (Games)
Perchang 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: Welcome to Perchang! A test of skill and puzzle solving across 60 stunning levels. | Read more »

Price Scanner via MacPrices.net

15-inch Retina MacBook Pros on sale for $200-...
B&H Photo has 15″ Retina MacBook Pros on sale for up to $210 off MSRP. Shipping is free, and B&H charges NY tax only: - 15″ 2.2GHz Retina MacBook Pro: $1799.99 $200 off MSRP - 15″ 2.5GHz... Read more
Mac minis on sale for up to $100 off MSRP
B&H Photo has Mac minis on sale for up to $100 off MSRP including free shipping plus NY sales tax only: - 1.4GHz Mac mini: $449 $50 off MSRP - 2.6GHz Mac mini: $649 $50 off MSRP - 2.8GHz Mac mini... Read more
Clearance 2015 13-inch MacBook Airs available...
B&H Photo has clearance 2015 13″ MacBook Airs available for $300 off original MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 1.6GHz/4GB/128GB MacBook Air (MJVE2LL/A): $799.... Read more
Apple refurbished Mac minis available for up...
Apple has Certified Refurbished Mac minis available starting at $419. Apple’s one-year warranty is included with each mini, and shipping is free: - 1.4GHz Mac mini: $419 $80 off MSRP - 2.6GHz Mac... Read more
ABBYY TextGrabber: 1,000,000 Installs in 5 Da...
ABBYY, an international OCR technologies provider, has announced that their image-to-text application TextGrabber, got installed 1,000,000 times in just five days while being featured by the App... Read more
New SkinIt Waterproof Case For iPhone 6
With its impact and waterproof design, the Skinit Waterproof case provides security and protection to guarantee your phone will get you through even the most demanding outdoor conditions. The impact-... Read more
iMacs on sale for up to $150 off MSRP
B&H Photo has 21″ and 27″ iMacs on sale for up to $150 off MSRP including free shipping plus NY sales tax only: - 27″ 3.3GHz iMac 5K: $2181.11 $118 off MSRP - 27″ 3.2GHz/1TB Fusion iMac 5K: $1949... Read more
12-inch 1.1GHz Retina MacBooks on sale for $5...
B&H Photo has 2016 12″ 1.1GHz/256GB Retina MacBooks on sale for up to $50 off MSRP. Shipping is free, and B&H charges NY tax only: - 12″ 1.1GHz Space Gray Retina MacBook: $1249 $50 off MSRP... Read more
WWDC Announcements Revisited Still Underwhelm...
I was disappointed that no new MacBook hardware was announced at this year’s all-software World Wide Developer’s Conference. Not even a hint about what’s in the development pipeline. Of course, we... Read more
Twelve South Compass 2 iPad Stand Now Availab...
Twelve South has updated its most popular iPad stand, Compass 2, with the introduction of two new colors — Gold and Rose Gold. These new color options n perfectly complement the new Rose Gold iPad... Read more

Jobs Board

*Apple* New Products Testers Needed - Apple...
…we therefore look forward to put out products to quality test for durability. Apple leads the digital music revolution with its iPods and iTunes online store, Read more
*Apple* Solutions Consultant - APPLE (United...
Job Summary As an Apple Solutions Consultant, you'll be the link between our future customers and our products. You'll showcase your entrepreneurial spirit as you Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Retail - Multiple Positions, Towson...
Job Description:SalesSpecialist - Retail Customer Service and SalesTransform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Professional Learning Specialist - A...
Job Summary The Apple Professional Learning Specialist is a full-time position for one year with Apple in the Phoenix, AZ area. This position requires a high Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.