TweetFollow Us on Twitter

Jul 96 Factory Floor
Volume Number:12
Issue Number:7
Column Tag:From The Factory Floor

A Little CodeWarrior History

By Dave Mark

This month, we’re going to talk with John McEnerney, one of the compiler writers
at Metrowerks.

Dave: How did you first hook up with Metrowerks?

John: I first met Greg Galanos when I was the development manager at Symantec’s Language Products Group. Greg was trying to get me interested in doing some sort of deal with the fledgling Metrowerks, and I mostly ignored him because they were trying to compete aggressively with my first product, THINK Pascal. I would have never guessed that a few years later he would offer me the best opportunity of my career.

Dave: When did you leave Symantec?

John: I left Symantec in October ’92, taking about 6 months off to figure out what I wanted to do next. I didn’t have any real plans, but I figured I’d find some way to do PowerPC work. I didn’t relish the thought of trying to write an entire C++ compiler, so I considered doing a Pascal product on my own.

Around this time, Greg had heard from Rich Siegel (of BBEdit fame) that I was no longer at Symantec, and he called me right away. The first thing he said to me was, “describe your dream job,” and I told him I wanted to write a PowerPC code generator for the upcoming Power Macintoshes. I flew to Montreal to meet him and his partner, Jean Belanger. We had some Italian food, drank some wine, and they told me a little about their Pascal and Modula products. I was really hot to write a PowerPC backend, but I was not that impressed with their technology. We talked about various contracts, but I didn’t have a really solid feeling yet.

Dave: What finally convinced you to go with Metrowerks?

John: In February ’93, Greg asked me to meet with him in Palo Alto to get a look at a C compiler that they had just acquired; a guy named Andreas Hommel in Hamburg had been writing it as a hobby. It ran on the Macintosh, had a simple but nice IDE reminiscent of early versions of THINK C, and it was fast. I spent about an hour looking through the source code: it was well organized, the compiler front-end and back-end were cleanly separated, the code was easy to follow, and in addition to being a full ANSI C compiler, it had a lot of the C++ language implemented already.

It was clear that Greg had found a diamond in the rough, the perfect platform for a native PowerMacintosh product. A few hours later we had a contract - I had about 6-8 months to write a PowerPC back-end and linker. Andreas would finish the C++ language implementation, and a few guys in Montreal (Berardino Baratta, Marcel Achim) would work on the IDE and a new Pascal front-end. We immediately hired Greg Dow, who had written the THINK Class Library for Symantec when I was there, to write a new application framework: PowerPlant.

We must have hooked up with Jordan Mattson from Apple around this time, because a week or so later he sent me one of their RS/6000s to help me get started. Between him and Alan Lillich, who I had met at all the early PowerPC meetings that Apple had been holding for their key developers, I got pretty much everything I needed from Apple.

So, I now had a contract to do the most interesting work I could imagine; all I had to do was figure out where to start.

Dave: What was it like working with Andreas’ compiler?

John: Andreas’ compiler was pretty traditional in its organization. The front-end made a single pass over the source code, performing lexical analysis as it went, and generated an intermediate representation (IR) that consisted of expression trees, labels, and branches. It took about a week to totally remove the 68K code generator from the rest of the compiler, and put in stub routines where the front-end and the back-end connected so that everything would still link. If I could fill in all the stub routines in exactly the right way, we’d have a PowerPC compiler.

The first thing I did was write a routine that dumped the IR in human-readable form - I don’t know how Andreas got his 68K code generator to work without that, I guess he can keep more in his head than I can. Looking at the expression trees on the screen allowed me to visualize how the code generator would proceed.

Most CISC compilers spend a lot of time working on the IR trees themselves. Traditional global optimizations like loop-invariant code motion or common subexpression elimination are performed by rewriting the IR trees into optimized IR trees. The code generator gathers information about the shape of the trees, deciding how many registers will be needed, which addressing modes will be used, etc. After instructions are generated they are largely ignored except for small “peephole” optimizations. (A notable exception to this is the gcc compiler, which transforms the expressions into a simple algebraic representation called RTL and uses repeated “peephole” optimizations derived from a machine description to coalesce these RTL expressions into complex instructions and addressing modes.)

Most of the RISC compilers that I’d read about in the compiler literature used a different approach: immediately transform the IR trees into a low-level representation that was similar or identical to the actual RISC instructions of the target machine, and perform all optimizations at the machine instruction level. I decided to use this technique in my PowerPC code generator.

Dave: What was your basic approach to code generation?

John: Strange as it may seem, the first part of the code generator I actually wrote was the instruction scheduler - the phase that reorders instructions to minimize latencies caused by load delays, and to permit floating-point and integer instructions to execute in parallel. I needed to know if my low-level representation - I called it a “pcode” (no relation to the UCSD Pascal pcode) - had enough information for all the phases I would eventually write, and since the scheduler needed a lot of information, it would serve to prove the design of the pcode. Of course, I had to rewrite the scheduler twice more: the first time was to fix the original one, which had some design flaws, and the second time was to make it more general to support 601, 603 and 604 CPUs.

Once I finished the scheduler, I had my data structures organized and all the support routines in place, so I started writing the instruction selection phase - the “guts” of the code generator. This phase visits the IR tree and generates pcode. It does try to recognize certain tree patterns, like opportunities for FMADD and FMSUB routines, but since there are no complex addressing modes and very few complex instructions, it is mostly a straightforward translation to PowerPC instructions.

To get short-term results, I wrote a quick-and-dirty register allocator, and some code to display the generated pcode instructions, and was able to get most of the code generation debugged this way. I decided to use a proprietary object code format, derived from the one we were already using in our 68K compiler and linker, since I could get this working faster than trying to write an XCOFF linker. I spent a few weeks getting the linker working, finished the part of the code generator that wrote the object file, and I could actually compile and link small programs.

Dave: How did the debugger fit into all this?

John: Around August ’93 the project was falling into place, but we still didn’t have a source-level debugger. In a most serendipitous event, Dan Podwall, a friend of mine from Symantec, called and asked whether there were any opportunities at Metrowerks. Greg Galanos called him right away, hired him on the phone, and 4 weeks later he had written a debugger - in PowerPlant, no less - that could single-step and set breakpoints. This would be the first commercial PowerPlant program - in fact, the first PowerPlant program of any kind aside from Greg Dow’s demos.

Dave: How did you build the compiler?

John: By September ’93 we had some prototype PowerPC hardware, and I had a working code generator and linker which ran on the 68K Macintosh and generated PEF executables that ran on the prototypes. Since this compiler was already built using our own 68K compiler, it was pretty easy to rehost it on the Power Macintosh: we made the changes for the Universal Headers and routine descriptors and such, then compiled it with itself on the 68K machine, which gave us (after some debugging!) a working PowerPC-hosted PowerPC compiler. With a little bit of trickery, mandated by differences between PowerPC floating-point hardware and the 68K SANE software floating-point architecture, we were able to rehost the 68K compiler on the Power Macintosh as well. We now had the fastest compilers on the Macintosh.

Dave: What next?

John: I still had a lot of work to do on the PowerPC code generator. The biggest task was to replace the quick-and-dirty register allocator with a graph coloring-based allocator. This is one of the great algorithms in the history of compilers. For years people had been trying to come up with an accurate way to represent the lifetimes of variables, so that variables or temporaries that did not overlap could share a register. A lot of ad hoc techniques were developed, but this guy from IBM Watson Research Center named Greg Chaitin discovered a formal approach that solved the problem better than anything that had been previously attempted: build an “interference graph” which has an edge between any two variables whose values may be live at the same time, and then try to color this graph with N colors where N is the number of available registers.

So my code generator assumes it has an infinite number of “virtual” registers, and generates the most efficient code it can under that assumption; for example, it assumes that all local variables, arguments, and TOC pointers can be assigned to a register. After the code is all generated, the register allocator tries to rewrite the virtual registers using real PowerPC registers, and generates extra code to “spill” values that couldn’t get a real register. In most cases, everything gets a register since there are so many on the PowerPC. The smarter register allocator probably makes the overall largest contribution to code quality.

The algorithm has one drawback: it has O(N^2) complexity. There are actually programs which have so many intermediate expressions that the interference graph gets too large and it takes several minutes to color it. So I had to keep around the quick-and-dirty allocator as well, which is why you’ll sometimes get an annoying message that says the code generator ran out of registers if you’re compiling without global optimizations.

Dave: And so, CodeWarrior was born!

John: By December I had pretty much everything working. After a last-minute dash to get C++ language support working on the PowerPC, we were able to burn our first public release, DR/1, starting a long Metrowerks tradition of getting things in under the wire and never missing a ship date. We introduced the product at the San Francisco Macworld Expo with our huge 8-page MacWeek advertisement, and CodeWarrior™ was born.

There were plenty of things to be cleaned up between DR/1 and DR/3, which was our real “1.0” release. But by shipping DR/1 and DR/2 when we did, and by working closely with a lot of the major Macintosh software vendors, we were able to help a lot of companies get their software ported to the PowerMac that otherwise might not have.

For me, I had accomplished what I had wanted to when I was back at Symantec: building the PowerPC compiler that most users would use to port their code to the new Power Macintoshes. And Greg had kept his promise and given me my dream job.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

VueScan 9.5.70 - Scanner software with a...
VueScan is a scanning program that works with most high-quality flatbed and film scanners to produce scans that have excellent color fidelity and color balance. VueScan is easy to use, and has... Read more
TextSoap 8.3.2 - Automate tedious text d...
TextSoap can automatically remove unwanted characters, fix up messed up carriage returns, and do pretty much anything else that we can think of to text. Save time and effort. Be more productive. Stop... Read more
ForkLift 3.0 - Powerful file manager: FT...
ForkLift is a powerful file manager and ferociously fast FTP client clothed in a clean and versatile UI that offers the combination of absolute simplicity and raw power expected from a well-executed... Read more
Amazon Chime 4.0.5528 - Amazon-based com...
Amazon Chime is a communications service that transforms online meetings with a secure, easy-to-use application that you can trust. Amazon Chime works seamlessly across your devices so that you can... Read more
Sparkle 2.1.0 - $79.99
Sparkle will change your mind if you thought building websites wasn't for you. Sparkle is the intuitive site builder that lets you create sites for your online portfolio, team or band pages, or... Read more
Bookends 12.7.9 - Reference management a...
Bookends is a full-featured bibliography/reference and information-management system for students and professionals. Bookends uses the cloud to sync reference libraries on all the Macs you use.... Read more
Dash 4.0.0 - Instant search and offline...
Dash is an API documentation browser and code snippet manager. Dash helps you store snippets of code, as well as instantly search and browse documentation for almost any API you might use (for a full... Read more
TrailRunner 3.8.834 - Route planning for...
TrailRunner is the perfect companion for runners, bikers, hikers, and all people wandering under the sky. Plan routes on a geographical map. Import GPS or workout recordings and journalize your... Read more
Dash 4.0.0 - Instant search and offline...
Dash is an API documentation browser and code snippet manager. Dash helps you store snippets of code, as well as instantly search and browse documentation for almost any API you might use (for a full... Read more
Bookends 12.7.9 - Reference management a...
Bookends is a full-featured bibliography/reference and information-management system for students and professionals. Bookends uses the cloud to sync reference libraries on all the Macs you use.... Read more

Fantasy MMORPG MU Origin’s receives a hu...
Developer Webzen are looking to take their highly popular fantasy battler MU Origin to the next level this month, with its most ambitious overhaul yet. The latest update introduces the long sought after Server Arena, new treasure dungeons, and much... | Read more »
RPG Djinn Caster (Games)
RPG Djinn Caster 1.0.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0.0 (iTunes) Description: SPECIAL PRICE 38% OFF(USD 7.99 -> USD 4.99)!!!A Fantasy Action RPG of far foreign lands! Summon the Djinns and rise to... | Read more »
Alto's Odyssey gets its first trail...
There's finally video evidence of Alto's Odyssey, the follow up to the 2015 App Store hit, Alto's Adventure. It looks just as soothing and atmospheric as Alto's last outing, but this time players will be journeying to the desert. Whereas Alto's... | Read more »
Last week on Pocket Gamer
What’s going on in the wider world of portable gaming? Each week we ask that question of our sister website Pocket Gamer. The PG team covers iOS gaming, just like 148Apps, but it also strays into the world of Android games and handheld consoles... | Read more »
Pokémon GO Generation 2 evolution guide
At long last, Niantic Labs finally unleashed the Generation 2 Pokémon into the wild. Pokémon GO trainers are scrambling to grab up this new set of 80 Pokémon. There are some special new tricks required to catch all of these new beasties, though.... | Read more »
The best new games we played this week
It feels as though the New Year got off to a creaking start as far as mobile games go, but that's changed over the past few weeks. The last few days alone have seen the debut of a number of wonderful games, so we thought we'd take the time to... | Read more »
Recruit more scallywags and discover new...
Get ready to show off your sea legs all over again in Oceans & Empires’ new grand update, which aims to make the act of rising to the role of seven seas ruler even more fresh and appealing, thanks to a richness of new content on both iOS and... | Read more »
Mage the Ascension: Refuge (Games)
Mage the Ascension: Refuge 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: The groundbreaking roleplaying game Mage: The Ascension manifests in our turbulent present with Refuge, an... | Read more »
Vampire: Prelude (Games)
Vampire: Prelude 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: The classic roleplaying game Vampire: The Masquerade returns to digital games with a Prelude of things to come. Experience a... | Read more »
Digby Forever Guide: How to dig to the d...
Digby Forever is a sparkling homage to arcade classics, and while you may be tiring of the number of arcade games being thrown at you, this endless digger finds many ways to stand out from the rest of the pack. The game manages to be challenging... | Read more »

Price Scanner via MacPrices.net

QuickerTek Announces 5TB Apple AC AirPort Tim...
QuickerTek Inc. has announced their new 5TB hard drive upgrade for Apple’s AC AirPort Time Capsule. By customer request, this upgrade also features six external antennas and offers the highest... Read more
Apple Certified Refurbished iMacs available f...
Apple has Certified Refurbished 2015 21″ & 27″ iMacs available for up to $350 off MSRP. Apple’s one-year warranty is standard, and shipping is free. The following models are available: - 21″ 3.... Read more
Apple offering Certified Refurbished Series 1...
Apple is now offering Certified Refurbished Series 1 and Series 2 Apple Watches for 14-16% off MSRP, starting at $229. An Apple one-year warranty is included with each watch. Shipping is free: Series... Read more
1.4GHz Mac mini on sale for $449, save $50
B&H Photo has the 1.4GHz Mac mini on sale for $50 off MSRP including free shipping plus NY sales tax only: - 1.4GHz Mac mini: $449 $50 off MSRP Read more
12-inch Retina MacBooks on sale for $200-$250...
Newegg has the 12″ 1.2GHz Space Gray Retina MacBook (sku MLH82LL/A) on sale for $1349.99 including free shipping. Their price is $250 off MSRP, and it’s the lowest price available for this model.... Read more
13-inch 2.0GHz Space Gray MacBook Pro on sale...
Adorama has the non-Touch Bar 13″ 2.0GHz Space Gray MacBook Pro in stock today for $100 off MSRP. Shipping is free, and Adorama charges NY & NJ sales tax only: - 13″ 2.0GHz MacBook Pro Space Gray... Read more
13-inch Touch Bar MacBook Pros on sale for $1...
B&H Photo has select 2016 Apple 13″ Touch Bar MacBook Pros in stock today and up to $100 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 2.9GHz/512GB Touch Bar MacBook... Read more
KSI-1802R SX Disinfect-able Keyboard With Wav...
KSI has unveiled a new, innovative medical keyboard, the KSI-1802R SX, at HIMSS 2017, running February 19-22 in Orlando, Florida. KSI-1802R SX is the only keyboard that combines dual factor... Read more
15-inch Touch Bar MacBook Pros on sale for up...
Amazon has 15″ Touch Bar MacBook Pros in stock today and on sale for up to $150 off MSRP including free shipping: - 15″ 2.7GHz Touch Bar MacBook Pro Space Gray: $2720.38 $79 off MSRP - 15″ 2.7GHz... Read more
Apple’s Education discount saves up to $300 o...
Purchase a new Mac or iPad using Apple’s Education Store and take up to $300 off MSRP. All teachers, students, and staff of any educational institution qualify for the discount. Shipping is free: -... Read more

Jobs Board

*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
Manager *Apple* Systems Administration - Pu...
Req ID 3315BR Position Title Manager, Apple Systems Administration Job Description The Manager of Apple Systems Administration oversees the administration and Read more
*Apple* Technician - nfrastructure (United S...
Let’s Work Together Apple Technician This position is based in Portland, ME Life at nfrastructure At nfrastructure, we understand that our success results from our Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
Manager *Apple* Systems Administration - Pu...
Req ID 3315BR Position Title Manager, Apple Systems Administration Job Description The Manager of Apple Systems Administration oversees the administration and Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.