Volume Number: 20 (2004)
Issue Number: 1
Column Tag: Programming

Rapid Development

by Richard Gaskin

In Praise of 4GLs

Making a custom Internet application in less than a day

Introduction

Fourth-generation languages (4GLs) have grown up. Once limited to narrow tasks like database queries, modern 4Gls, like Revolution and SuperCard, offer rich languages and object models suitable for complete GUI application development.

Using a 4GL can be a smart choice when developer productivity is a project's critical driver. For example, many vertical market applications are too specialized to have a broad enough audience to support development in lower-level languages like C++ or Java, but a 4GL can often get the job done in a fraction of the time.

The tradeoff for this ease of development is sometimes execution speed, as the interpretation of scripts at runtime involve steps done at compile-time in lower-level languages. However, with well-optimized 4Gls, like Revolution, this is not always the case, and for many common tasks Revolution measurably outperforms Java, and other lower-level languages.

To better appreciate the runtime efficiency of a modern 4GL like Revolution, consider what's happening under the hood. In Revolution, when a script is loaded, it's interpreted into bytecode in a form reported to be more efficient than Java's. When executing a script, this bytecode acts as a glue, connecting compiled routines in the engine written in C++. So, while a script is indeed interpreted, a single line of script often triggers the execution of hundreds of lines of compiled, optimized C++.

To illustrate the productivity gain, consider the task of creating an alias to a file. In C it takes about 18 lines, and in a Java example posted to an Apple discussion list it took 247 lines. In Revolution's scripting language, Transcript, it's a one-liner:

create alias MyAliasPath to file MySourcePath

With hundreds of commands, functions, and object properties in Transcript, things like opening windows, handling controls, reading files, and other common tasks are usually one-liners, which means that most of the code being executed is natively compiled C++. In essence, the Transcript engine acts like a precompiled object library you string together with a few lines of script.

Productivity in Action

Let's take a look at how these productivity benefits play out in a real-word example: building a custom FTP client in a few hours.

The need for this arose from a teleconference with a client. We were starting a project in which we'd need to trade a lot of files back and forth. The number of files made email a cumbersome option, and the size of some of them made using email prohibitive.

Once we decided to use FTP we reviewed the various FTP clients available, including my personal favorite, Interarchy. While the applications we looked at were all great general solutions for file transfer, the breadth of features they offered was problematic in our case, given that files would sometimes need to be transferred by people with little or no experience using FTP tools. Also, the client's office had a mix of machines running OS X and Windows, so we wanted something that was not merely easy to use, but also had the same simple interface for both platforms.

I offered to make a custom FTP tool with the goal of having the fewest features possible. It would be hard-wired to work with a single directory on our server, require nothing more than a password to log in, and work identically on both OS X and XP.

When I told him I'd have it ready by the end of the day he was surprised. What he didn't know was that the secret weapon making this possible was Revolution's Internet library, libURL.

libURL: The Heart of our application

Tim Monroe's excellent series of MacTech articles on developing QuickTime applications with Revolution provides a great introduction to the development environment and scripting language, so here we'll focus on working with libURL.

The Revolution engine contains robust support for TCP and UDP sockets, but if you've ever written an FTP client, you know how difficult it can be to handle all the error-checking needed to do it right. So, along with the other scripted libraries included with Revolution is libURL, which provides a simple interface for handling HTTP and FTP transactions.

LibURL uses Transcript's simple put command for the most common tasks. Just as you would display text in a field with put "Hello World" into field 1, you can specify a URL as a container as well.

To download a file from a Web server and display its contents in a field:

put url "http://www.fourthworld.com/data.txt" into field 1

Uploading a file to an FTP server is just as easy:

put tMyData into url \
  "ftp://user:password@ftp.domain.com/tDataFile.dat"

LibUrl also provides more than a dozen other commands to allow asynchronous transfer, logging, status updates, and more.

With libURL and Revolution's simple tools for building interfaces, putting the application together was a snap.

Development Chronology

8:30 AM Defined the requirements with the client.

9:00 AM The simple interface needed only a few objects, including a list field to display remote files, a Download button, Upload button, and of course a button to initiate the connection. Figure 1 shows Revolution's Tool palette for creating objects, and its Inspector palette for setting object properties.

Figure 2 shows a profile of our objects in Revolution's Application Browser after adding a menu bar and a hidden progress bar which we'll show during file transfers.

Once the client interface was done, I needed to build an administration window for my own use during development, so I could enter the FTP login information. I could have written those settings directly into the code, but taking a moment to make an interface for myself will make it easier to modify the application for other projects in the future (Figure 3).

I also added a simple window containing only a field to display a log of the transactions with the server to help with debugging. This exploited one of the conveniences of libURL: it includes a simple command that lets you use any field object as a log. You just pass it any valid field reference and all TCP transaction logging is automatically appended to the field's contents as it goes:

libUrlSetLogField the long ID of fld "log" \
  of stack "4wFtpLog"

The graphics in the main window were just modified versions of images from our Web site, so the interface was completed in about an hour.

10:00 AM With the interface objects in place it was time to start coding. The first thing the program needed to do, of course, was display a list of files from the remote server. As shown above we can download a file by just using the get url command, and we can download a directory listing by just specifying any valid directory on the server with the trailing slash:

  get url "ftp://user:password@ftp.domain.com/directory/"

This returns a raw listing of files in a standard directory format:

drwxr-xr-x   2 user2000 user2000      512 Oct  1 01:46 .
drwxr-xr-x   4 user2000 user2000      512 Sep 30 08:44 ..
-rw-r--r--   1 user2000 user2000   898089 Oct  1 01:48 Tutorial1.pdf
-rw-r--r--   1 user2000 user2000  1118961 Oct  1 01:46 Tutorial2.pdf
-rw-r--r--   1 user2000 user2000   557660 Oct  1 01:44 Tutorial3.pdf
-rw-r--r--   1 user2000 user2000    88486 Oct  1 01:41 workflow2.png

If order to display the file list in a simpler form for the user, the first function I wrote converted the raw data into a tab-delimited list, since Revolution fields can display tab-delimited data in a field object as a multi-column list automatically.

Converting this raw data introduced one of the strengths unique to Revolution and other 4GLs inspired by HyperTalk, affectionately called chunk expressions. These HyperTalk dialects support fast and simple text parsing with references like word (space-delimited), item (comma-delimited), and line (return-delimited). In Transcript all three of these chunk types can use custom delimiters so they can be applied to a wide variety of tasks. Similar routines in other languages usually require walking through blocks of text one character at a time, counting delimiters, and building arrays as you go. In Transcript, however, you can simply write things like:

get word 2 of item 3 of line 4

Here's the code for the reformatting function:

--
-- FormatRemoteFileList
--
-- Extracts the relevant into from each line in the
-- FTP file list passed in pList and returns a tab-
-- delimited list of just file name, size, and date
-- for display in the file list control
--
function FormatRemoteFileList pList
   put empty into tFormattedList
   repeat for each line tFile in pList
      -- Skip folders in this version:
      if char 1 of tFile = "d" then next repeat
      -- Get file name:
      put word 9 to (the number of words of tFile) \
        of tFile into tName
      -- Skip invisible files:
      if char 1 of tName = "." then next repeat
      --
      put Bytes2Size(word 5 of tFile) into tSize
      put word 6 to 8 of tFile into tDate
      put tName &tab& tSize &tab & tDate \
       &cr after tFormattedList
   end repeat
   delete last char of tFormattedList
   return tFormattedList
end FormatRemoteFileList

The Bytes2Size function simply converts bytes into a common abbreviation appropriate for user display, for example "323455" is returned as "3.2Mb":

function Bytes2Size n, pPadFlag
   if pPadFlag is empty then set the numberformat to "0.#"
   else set the numberformat to "0.0"
   --
   if n < 1024 then put n &" bytes" into n
   else
      put n / 1024 into n
      if n < 1024 then put n &"k" into n
      else
         put n / 1024 &"Mb" into n
      end if
   end if
   return n
end Bytes2Size

Since most of the controls would affect the user interface, I wrote one handler to handle all interface updates, called from most of the other UI-related code. I wanted to keep things simple, so I had it pass just one parameter to indicate if the user was connected to the server so it would refresh the file list. If called with no arguments, it cleared the list field:

--
-- UpdateConnectionUI
--
-- One-stop shopping for updating the user interface
-- whenever the user connects or disconnects
--
on UpdateConnectionUI pConnectedFlag
   libUrlSetStatusCallback
   put (pConnectedFlag = "connected") into tIsConnected
   set the enabled of grp "server" to tIsConnected
   hide scrollbar "progress"
   if tIsConnected is true then
      set the label of btn "Connect" to "Disconnect"
      --
      put ServerDirectoryUrl() into tUrl
      put url tUrl into tFileList
      if (the result is not empty) and \
       ("not completed" is not in the result) then
         answer the result
         UpdateConnectionUI
         exit to top
      end if
      put FormatRemoteFileList(tFileList) into fld "files"
      PutStatus "Connected"
      --
   else
      set the label of btn "Connect" to empty
      put empty into fld "files"
      PutStatus "Not connected"
   end if
   disable btn "Download..."
end UpdateConnectionUI

That handler called a couple of other simple handlers written as a convenience:

--
-- ServerDirectoryUrl
--
-- Returns the full URL to the server directory
-- by concatenating info stored in the hidden
-- Admin window
--
function  ServerDirectoryUrl
   return "ftp://"& \
        fld "login"     of stack "4wFtpAdmin" &":"& \
        fld "password"  of stack "4wFtpAdmin" &"@"& \
        fld "server"    of stack "4wFtpAdmin" &"/"& \
        fld "directory" of stack "4wFtpAdmin" &"/"
end ServerDirectoryUrl

--
-- PutStatus
--
-- Displays the string in s in the "Status" field
--
on PutStatus s
   put s into field "status" of stack "4wFtp"
end PutStatus

With the basics in place it was time to put them to work by adding code to handle the Connect button. To keep the interface as simple as possible, I used one button object for both connecting and disconnecting, merely changing the button's label property to reflect the change to the state. This is from the Connect button's script:

on mouseUp
   if the label of me = "Disconnect" then 
     DisconnectFromServer
   else ConnectToServer
end mouseUp

ConnectToServer and DisconnectFromServer are defined in the script of the main window, which Revolution refers to as a "stack":

--
-- ConnectToServer
--
-- Called from the Connect/Disconnect button
-- to log on to the FTP server and obtain a
-- list of files
--
on ConnectToServer
   put fld "Login" into tLogin
   if tLogin <> fld "login" of stack "4wFtpAdmin" then
      answer "Incorrect login"
      exit to top
   end if
   --
   set cursor to watch
   put fld "login" into tLogin
   ask password clear "Enter your password:" as sheet
   if it is empty then exit to top
   put it into tPassword
   if tPassword <> fld "password" of stack \
    "4wFtpAdmin" then
      answer "Wrong password for your site."
      exit to top
   end if
   --
   if "4wFtpLog" is in the windows then
      libUrlSetLogField the long ID of \
       fld "log" of stack "4wFtpLog"
   end if
   --
   UpdateConnectionUI "connected"
end ConnectToServer

--
-- DisconnectFromServer
--
-- Called from the Connect/Disconnect button to
-- clear the file list and update the interface
--
on DisconnectFromServer
   UpdateConnectionUI
end DisconnectFromServer

Most of the code is fairly self-explanatory with the background provided earlier, but it's worth calling your attention to the ask and answer commands. These Transcript commands provide one-line convenience for displaying a simple alert dialog with answer, or allowing user input with ask, returning values in a predefined local variable named it. Extensions to these commands also provide a script interface to the OS's GetFile and PutFile dialogs on each of the supported platforms, which will be used later when we script the Download and Upload buttons.

I tested the work done thus far, clicking the Connect button to see the remote file list displayed and the button relabeled to Disconnect. I clicked it again to clear the interface. So far, so good. Time for lunch.

12 Noon With things going so well I packed a picnic and took it to the deck on the roof of my office building to enjoy a long lunch in the California sunshine.

1:30 PM Returning to the office, the next step was to get the Upload and Download buttons working. Both of these make extensive use of a great feature of libURL, the libUrlSetStatusCallback command.

As we covered earlier, Transcript's get and put commands can be used to conveniently use URLs as containers. While these are very convenient, they are synchronous, suspending other script execution such as one might need for updating a progress bar, for example.

Fortunately, libURL provides commands for asynchronous data transfer, as well. For FTP, these commands include libUrlDownloadToFile and libUrlFtpUploadFile.

But, to get the most out of asynchronous execution, you'll want to be notified of the status of the transaction so you can take appropriate steps, such as updating a progress bar, and notifying the user when the transfer is completed. You set this up with libURL using the libUrlSetStatusCallback command, specifying the message name you want sent and the object it should be sent to:

libUrlSetStatusCallback message, object

While a transfer is in progress, libURL will then send that message to the object along with an argument containing the URL the callback is for, and another containing a string which describes the current status. The status argument will consist of one of the following:

queued: on hold until a previous request to the same site is completed
contacted: the site has been contacted but no data has been sent or received yet
requested: the URL has been requested
loading bytesTotal,bytesReceived: the URL data is being received
uploading bytesTotal,bytesReceived: the file is being uploaded to the URL

downloaded: the application has finished downloading the URL
uploaded: the application has finished uploading the file to the URL
error: an error occurred and the URL was not transferred
timeout: the application timed out when attempting to transfer the URL
(empty): the URL was not loaded, or has been unloaded

Since so much of the user interface that needs to be updated during a transfer is common to both uploading and downloading, I wrote one handler to handle status callbacks for both. Using a switch block to handle each type of status message appropriately, this handler was effectively the core of the application:

--
-- UpdateStatus
--
-- Callback from libURL providing status info
-- and manages each stage of the transaction
-- based on the current status
--
on UpdateStatus pUrl, pStatus
   set the itemdel to "/"
   put last item of pUrl into tFileName
   set the itemdel to comma
   put empty into tStatusDisplayString
   --
   switch item 1 of pStatus
      -- Handle progress:
   case "uploading"
      put "Uploading" into tStatusDisplayString
   case "loading"
      -- Update progress bar:
      put item 2 of pStatus into tBytesReceived
      put item 3 of pStatus into tTotalBytes
      set the endvalue of scrollbar "progress" \
        to tTotalBytes
      set the thumbpos of scrollbar "progress" \
        to tBytesReceived
      show scrollbar "progress"
      --
      if tStatusDisplayString is empty then 
        put "Downloading" into tStatusDisplayString
      end if
      put cr& tFileName &cr& \
         Bytes2Size(tBytesReceived, "pad") & \
         " of "& Bytes2Size(tTotalBytes, "pad") \
         after tStatusDisplayString
      PutStatus tStatusDisplayString, "right"
      break
      --
      -- Errors:
   case "error"
   case "timeout"
      answer pStatus
      exit to top
      break
      --
      -- Completed successfully:
   case "uploaded"
      UpdateConnectionUI "connected"
   case "downloaded"
      hide scrollbar "progress"
      PutStatus "Done"
      enable btn "Download..."
      enable btn "Upload..."
      unload pUrl
      break
   end switch
end UpdateStatus

I added scripts to the Upload and Download buttons that simply call these handlers to initiate each respective transfer:

--
-- UploadFile
--
-- Called from the Upload button to select a local
-- file and start uploading it to the server
--
on UploadFile
   answer file "Select a file to upload:"
   if it is empty then exit to top
   put it into tSource
   --
   set the itemdel to "/"
   put ServerDirectoryUrl()&last item of tSource into tDest
   --
   libUrlSetStatusCallback "UpdateStatus", long name of me
   libUrlFtpUploadFile tSource, tDest, "UpdateStatus"
   disable btn "Download..."
   disable btn "Upload..."
end UploadFile

--
-- DownloadFile
--
-- Called from the Download button to start downloading
-- the selected file
--
on DownloadFile pFile
   ask file "Save this file to:" with pFile
   if it is empty then exit to top
   put it into tDest
   --
   put ServerDirectoryUrl()&pFile into tSource
   --
   libUrlSetStatusCallback "UpdateStatus", long name of me
   libUrlDownloadToFile tSource, tDest, "UpdateStatus"
   disable btn "Download..."
   disable btn "Upload..."
end DownloadFile

I had a little extra time so I added a useful flourish: I placed an animated GIF version of the flag over the image at the top of the window, and modified the UpdateConnectionUI handler to show the GIF when connected and hide it when disconnected. It was a small touch, but the animated element helped visually reinforce when the connection was "live".

After a little debugging to address a few typos, it seemed to be working well, so now it was time to build and test the standalone application.

3:30PM One of Revolution's greatest strengths is its broad support for deploying on multiple operating systems. With engines available for Mac Classic, OS X, all Win32 systems, and most flavors of UNIX and Linux it covers nearly every modern desktop system.

Standalones are built with Revolution's Distribution Builder (Figure 4), a utility accessed from the File menu, that lets you assign the application's file name and version info, assign icons, etc.. Once you set it up, you can save the settings for future use.

You just click the Build Distribution button and it does the rest, embedding the appropriate engine into a copy of your stack file to create the standalone. For OS X it constructs the bundle and writes the info.plist file for you as well.

Figure 5 shows the completed application in action.

4:00 PM I compressed the application with SuffIt and emailed it to the client. He ran a few tests and was delighted with its simplicity. I left work early.

Conclusion

4GLs may not be the best choice for every job, especially computationally intensive tasks like writing device drivers or rendering 3D. But, when time-to-market is a concern a good 4GL like Revolution can be hard to beat. How many other systems let you build a custom FTP tool in under 200 lines of code?

If you want to check out the source for yourself you can download it from:

http://www.fourthworld.com/mactech/

Since the application requires your server info you'll need Revolution to set up the Admin window and build the application. You can download the free trial version of Revolution from Runtime Revolution Ltd.'s site:

http://www.runrev.com/

Richard Gaskin is president of Fourth World Media Corporation, a Los Angeles-based consultancy specializing in multi-platform software development. With 15 years' experience, Richard has delivered dozens of applications for small businesses and Fortune 500 companies on Mac OS, Windows, UNIX, and the World Wide Web. http://www.fourthworld.com