Dinker: the Dynamic Linker

Kurt Schmucker

Many application domains naturally possess an unbounded set of desired features. A signals processing application, for example, needs an unlimited number of filters. A symbolic math package, an unlimited number of mathematical operators. A graphics package, an unlimited number of sketching tools.

Of course, no real application can present an unlimited number of anything to any given user. However, the market as a whole does appear to want an unlimited number of features. What's a developer to do?

Some developers have met this challenge by building in a capability to extend their applications. Using this feature, end users can configure a graphics package, for example, with a set of sketching tools tailored to their needs, or even modify the set of available sketching tools while the application is running.

This enables users to get the maximum use of limited RAM and disk space. Instead of running in a six megabyte MultiFinder partition, for example, an extensible application, with only the needed tools loaded, could run in, say, four megabytes. In fact, one of the best selling MacApp applications, Adobe PhotoShop, uses an extensibility approach with their plug-ins to add new image manipulation algorithms and data access procedures to the application.

Up until now, MacApp has provided no support to those wanting to add extensibility to an application. To make matters worse, the most natural way to add functionality to a MacApp application-by adding new classes to the running application- was something no one had ever done. Some people even thought this was impossible to do on the Mac.

Dinker, the results of recent work by Apple's Advanced Technology Group, adds exactly this capability to MacApp. This article describes what Dinker does and how you can use it in your application. Other articles planned for later issues of FrameWorks will cover Dinker internals and case studies of adding Dinker-based extensibility to commercial MacApp applications.

Dinker is included on the next Essentials, Tools, and Objects CD (ETO 6) as a currently unsupported MacApp experiment. It will be demonstrated at the Orlando '92 MADA Conference in February.

What is Dinker?

Dinker is a dynamic linking mechanism for the Macintosh OS. It lets you write applications whose functionality can be extended at run-time by the addition of new Object Pascal or C++ classes. In particular, Dinker lets the MacApp programmer add new 'view' or 'VIEW' resources, new view classes, new command classes or any other classes at run-time, and to use instances of these classes just like any other object in the application.

The basic run-time architecture of a Dinker application is shown in Figure 1, and then in greater detail in Figure 2 (next page). Dinker will set up entry and exit points in the base and all extensions, and will ensure that the proper global environment (A5 worlds, resource chains, etc.) is established upon crossing the boundaries between the base and the extensions, or between extensions.

Building an application that can be extended at run time via the Dinker is easy. Tell the Dinker version of MABuild to build the application to make use of Dinker. Such an application is called a "base." Then, use MABuild to build each dynamically linkable extension for the application (called an "extension" or a "component"). The total Dinker package used by a MacApp programmer-Dinker itself, the Dinker-aware modified version of MacApp (either a modified version of MacApp 2.0.1, or a modified version of MacApp 3.0ß3), the Dinker-aware version of MABuild, etc.-will cause the entry and exit points to be constructed for the base and for all extensions, and will resolve, at run-time, the invocations of these references. (See future FrameWorks articles for an explanation of the Dinker's inner details.)

A note on terminology-Dinker is a dynamic linker. It lets you "dink in" a new class into a running application. Among dynamic linkers, Dinker is unique in that it enables the dynamic linking of new classes into an object oriented application, like a MacApp application. For a time, the term "oinking" was used to describe this dynamic linking, but this term has fallen into disfavor with the Dinker implementors. A later, supported version of Dinker may have a slick, marketing-approved, copyright-obtained legal name. We hope that all savvy MacApp developers will continue to call it "Dinker" in perpetuity.

What Does dynamic linking do for users?

MacApp developers are subject to the same pressures as every other developer-users want more and more functionality in applications without the resource requirements increasing.

Extensibility lets you achieve these opposing goals. Users can pick which tools to load in the default case (by, for example, placing these in a distinguished folder), yet still have easy access to the tool needed on special occasions. In this way, Dinker lets users manage the resources needed to run your application much like Fifth Generation System's Suitcase™ II lets the user manage fonts, desk accessories, FKEYs, and sounds.

In addition, for those developers who wish to encourage a third-party market surrounding their applications, Dinker provides a way for other MacApp developers to leverage off of your application by developing extensions for it. These third parties will need the entry and exit vector files the Dinker-modified version of MABuild prepares for the application. These could be part of a developer's kit for your application. Developers of extensions do not need access to the source code of the base.

How is dinker used in an application?

To use Dinker, you willl probably need to change your application's source; most applications aren't currently designed for extensibility. Do you have a fixed number of items in a palette-items that the end user may want to add to, or replace? Do you check, upon resuming after a MultiFinder switch, if the user has added functionality to the application "behind your back?" Probably not. There will be as many ways of designing extensibility into MacApp applications as there are clever MacApp programmers.

Automatic loading of components

One way of adding new functionality to an application is to have the user configure a folder with the desired extensions in it. Dinker supports this approach. (If this sounds vaguely like the System 7 Extensions folder, it is. The Finder Extensions mechanism used in System 7 is a special case of the full Dinker design.) The Dinker version of MacApp provides a number of ways to find out which extensions have been loaded. TApplication (in this modified version of MacApp) maintains a list of the currently loaded extensions, and several utility methods and procedures let you easily query this list.

One interesting technique uses a MacApp method that you probably haven't used before: ForAllSubClassesDo in TObject. I used this technique to add an indeterminate number of dinked-in views to a window. It just happened to be the case that while each of these new subviews was a instance of a different class, all the classes were subclasses of a class unique to my application (TBannerView). The code looks like this:

PROCEDURE TBannerDocument.DoMakeViews(…);  
                                                OVERRIDE;
    VAR theWindow:  TWindow;
        theFirstView:   TBannerView;
        currentLoc:     VPoint;
        aView:          TBannerView;
        
        PROCEDURE AddAView(theClass: ObjClassID);
        BEGIN
            currentLoc.v := currentLoc.v + 135;
            aView := TBannerView(NewObjectByClassID(theClass));
            FailNIL(aView );
            aView.IBannerView(…); { this will add it as subView }
        END;
        
    BEGIN
        SetVPt(currentLoc, 0, 0);
        
        { Prime the pump by setting up the window
            with the first view }
        theWindow := NewTemplateWindow(2222, SELF);
        FailNIL(theWindow);
        theFirstView := TBannerView(theWindow.FindSubView('MNBN'));
        
        { Now add the rest of the views }
        theFirstView.ForAllSubClassesDo(AddAView)
    END;

You could use the same technique to build a palette (when all the new tools to be configured into the palette are subclasses of something like TAbstractTool), or a hierarchical menu.

Manual loading of components

Another way to add functionality is to let the user load new components at any time the application is running. Dinker also supports this. One way you can use this support is to add the appropriate menu items to your application and to process these in a DoMenuCommand something like this code from the Dinker version of DrawShapes:

PROCEDURE TShapeApplication.DoMenuCommand(
            aCommandNumber: CommandNumber); OVERRIDE;
    VAR
        anExtensionFile:    TList;
        aDomain:            TDomain;
        r:                  Rect;
        shape:              TShape;
        toolsView:          TToolsPalette;
        toolsMenu:          TToolsMenu;

        PROCEDURE DoToItemInExtensionList(item: TObject);
    BEGIN
        aDomain := GetExtension ( TFile (item) );
        IF longint(aDomain) <> 0 THEN BEGIN
            { Define the prototype shape this component }   
            SetRect(r, 9, 49, 31, 71 );                     
            OffSetRect(r, 0, 40 * ( gShapesInPalette ) );
            shape := TShape ( Instantiate( aDomain ) );
            FailNil(shape);
            gShapesInPalette := gShapesInPalette + 1;
            shape.IShapeExtension(r, gShapesInPalette );
            gShapesArray[gShapesInPalette] := shape;
        END;
    END;
    
    PROCEDURE ForceWindowRedraw ( aWindow: TWindow );
    BEGIN
        aWindow.ForceRedraw;
    END;

    BEGIN
    CASE aCommandNumber OF
        ...
        cGetComponent:                      
            BEGIN
                IF ChooseDocument( cGetComponent, anExtensionFile )
                THEN BEGIN
                    anExtensionFile.Each( DoToItemInExtensionList );
                    anExtensionFile.Free;
                    { Now add the new shape to the palette }
                    toolsMenu := TToolsMenu(gToolsMenu);
                    IF longint(toolsMenu) <> 0 THEN
                        toolsMenu.AddTool;
                    ... { Add to Tear offs, etc. here }
                    ForAllWindowsDo( ForceWindowRedraw );
                END;
    
            OTHERWISE
            INHERITED DoMenuCommand(aCommandNumber);
    END; {Case}
END;

'lView' templates in extensions

Often, TView objects are not allocated explicitly by the programmer, but rather are instantiated indirectly via 'view' resources. Extensions also have their own resource chain (forked off of the top of the base's resource chain). You can put 'view' and 'VIEW' resources in your extension. The Dinker extensions to MacApp include Dinker analogs of the familiar utility routines NewTemplateWindow and DoCreateViews, NewComponentTemplateWindow() and DoCreateDinkedViews(), which you can then use to dink in these views from other extensions. Like the standard NewTemplateWindow , NewComponentTemplateWindow lets you access a view by its resource id. However, NewComponentTemplateWindow also lets you access the new window type by the name of the extension that contains it. We have found this to be a natural way to add new functionality to an application in the case that the "important" portion of the extension was the new views being added. In this application, we made the names of the extensions the same as the names of the view.

What are the Drawbacks of using Dinker?

There is no free lunch, and Dinker, too, has some costs. These costs are not in the area of performance. Using instances of dinked-in classes carries only a tiny performance hit on the application. The gotchas of the current Dinker implementation are as follows.

Launch time

Dinker applications take longer to launch than "normal" applications. Remember that Dinker is a dynamic linker. Thus, it is doing the work of the linker at run-time. The amount of time depends mostly on the Mac model you are executing on (no surprise here) and the version of MacApp you are using (remember that MacApp 3 has about twice the number of classes and methods of MacApp 2, and there is a lot more to link in a MacApp 3 application). The additional amounts of launch time can range from less than two seconds (FX, MacApp 2) to more than 30 seconds (Portable, MacApp 3). I hope that future versions of Dinker will be able to reduce these times, but they will never go to zero.

Extensions are application specific

The Dinker internals that make it the only dynamic linker that enables the addition of new classes at run-time also require that a built extension, ready for dinking, be application specific. In addition, any significant changes to the class structure of the application require that all the application's extensions be rebuilt.

What is "significant?" Addition or removal of classes, addition or removal of methods, addition or removal of any entities that the linker needs to know about (for example, invocation of a new Toolbox routine, or moving something from the private interface to the public interface) are prime examples. "Insignificant," from a Dinker point of view, are changes to the implementations of methods or functions that don't add or remove Toolbox calls.

Debugging support

The ease of debugging a Dinker application depends on whether MacApp is used, and what version of MacApp is used. The best case is MacApp 2.0.1. The two main debugging tools designed for MacApp 2.0.1, the Inspector and the MacApp Debugger, are fully integrated with Dinker. The dinked-in classes and any instances of these classes are viewable from these tools and, in fact, you'll have a hard time distinguishing between an instance of a dynamically linked class and a statically linked class. This was a goal of the Dinker implementation, and we believe it has been met (after a lot of hard work and many late night sessions double checking on this integration).

Unfortunately, the situation is not so rosy with respect to a MacApp 3.0ß3 application, or a non-MacApp application. Yes, you can use Dinker without using MacApp. However, it isn't as easy or as convenient, mostly because we planned the first Dinker release to concentrate on the use of Dinker by a MacApp programmer.

The new MABuild, for example, handles the complexity of building the entry and exit vector segments without any work by the MacApp programmer. There are currently no such automated build facilities for the non-MacApp developer. There isn't much debugging support in either SourceBug or SADE® for dynamically linked extensions. Basically, neither of these tools yet know how to read the multiple SYM files associated with the base and all of its currently dinked in extensions, nor do they know to reassess a class hierarchy that has been modified dynamically by Dinker. However, that doesn't mean these debugging tools are useless with a Dinker application. You can use SourceBug with Dinker applications, but with these limitations:

• SourceBug is confused by the new Class hierarchy that Dinker creates. So SourceBug's browser displays the source code filenames of the base and of the MacApp library rather than its classes.
• SourceBug only really knows about the code in your base. You must go into your low level debugger if you want to step into your extension.

I hope that these limitations will be removed in the future.

Finally, a goal of Dinker was to require only minimal-and preferably pro forma-changes to a class to change it from being a statically linked piece of functionality to one that could be dynamically linked into the application. We believe this goal has been met. Thus, one possible debugging strategy is to develop and debug your application as a monolithic, statically linked application, and then when it is "bug free," to peel off various pieces of functionality into dynamically linked extensions. (Okay, okay. We know that this is kind of lame, and that your testing manager just had a heart attack even considering this approach. But it's better than nothing, and this is only the early experimental release.)

Support, in general

The Dinker ETO #6 release is an unsupported, experimental release, but Dinker or other dynamic linking mechanisms may be incorporated into future releases of MacApp, MPW, or other development tools.

While it is implemented according to the same architecture as Finder Extensions, Dinker adds significant new dynamic linking functionality to Finder Extensions and thus an extended period of testing is needed. The Dinker work was done by Apple's Advanced Technology Group and is being shared with third-party developers for experimentation and evaluation. If you find dynamic linking to be a useful feature, then it may be supported and extended in future MacApp and MPW releases. Let us know by sending an AppleLink message to the Dinker Product Manager, Tom Chavez (TOM.CHAVEZ).

FUTURE FRAMEWORKS ARTICLES

How Dinker works, and what it is like to add extensibility to an existing commercial MacApp application, will be covered in detail in future FrameWorks articles.

Acknowledgements

The Dinker folder on ETO #6 is the result of many months of work by employees from several parts of Apple, as well as some Apple contractors. Jed Harris, then in ATG, now in the Developer Tools group, did the original Dinker design. He and Scott Douglass, now in the System Software group at Apple, implemented the first proof of concept that Dinker would actually work. Frank Kurzawa of Refried Software, and Andrew Donoho of Donoho Design Group did the current Dinker implementation under contract to ATG. Frank also designed and implemented the changes to MacApp 2.0.1 and MacApp 3.0ß3 that make it work with Dinker, and he did many of the Dinker sample programs. Joost Kemink and Kurt Schmucker, both of ATG, were the "typical" Dinker users whose dinking needs drove the design, and who did all the testing of Dinker with MacApp. Kurt Schmucker was the author of two of the Dinker sample programs. Tom Chavez is the Dinker Product Manager and guided the Dinker work onto ETO.