TweetFollow Us on Twitter

Data Conversion
Volume Number:4
Issue Number:5
Column Tag:Basic School

Quick & Dirty Data Conversion

By Dave Kelly, MacTutor Editorial Board


Over the years, one of the most beneficial uses for Basic is the ability to quickly write programs to import/export data to and from data bases. This also provides a “quick and dirty” way to convert data or suppress unwanted characters. I’d like to explain/complain about the process and some of the problems that I have encountered.

My first major data conversion came a couple of years ago when I converted 30,000 names in a mail list from my Apple //e to the Macintosh. Yes, this was quite a feat, mostly because of the bottle neck which the dinky 143K floppy drives allowed on the Apple //. And you thought the Mac 400K drives were small! Anyway, on the Apple //e I was using DB MASTER, a now orphaned product, with some small support still available from a consulting firm in Northern California. There may be some of you that are familiar with DB MASTER (sometimes it was referred to as DB MONSTER). DB MASTER was great for mail lists and still would be today except for it’s slow speed (that’s partly the Apple //s fault).

The process went something like this: First the data needed to be converted from DB MASTER file format to ASCII so that I could direct connect the Mac and the //e together. DB MASTER utilities includes an export utility which would convert files to either DIF or a “One Field per Line” format. The utilities converted my mail list from my Sider hard disk to about 50 143K floppy disks (UGH!). Next the ASCII needed to be converted from Apple // ASCII to ”the rest of the world” ASCII format. ASCII EXPRESS telecommunications software provides a conversion utility for doing this (If they didn’t I’d have to use my own Basic program to convert it myself. Next each disk was converted via direct null modem connection to the Mac. Whew!

To make a long story (and many hours) short, the mail list was now on the Mac, but somehow during the process some control characters were intermixed in the files. Omnis 3 would not read the files with these control characters so it was BASIC to the rescue. There are 128 standard ASCII characters of which the first 31 characters are control characters, traditionally used for teletype terminals. The problem was easily solved by writing a Basic routine to read in the file, filter out the bad characters and rewrite the file. Compile the program to speed it up!

Control Character Filter
‘ ©MacTutor, 1988
‘ By Dave Kelly

OPEN “Filter Output” FOR OUTPUT AS #2
    IF ASC(Char$)>=32 THEN PRINT #2,Char$;
    IF Char$=CHR$(13) THEN PRINT #2,Char$;

That’s ok for simple filtering. Things may need to be more complex if the original file contains unexpected characters in it. Recently, I came across the need to convert data stored in a FileMaker Plus file in order to do some simple, but unusual calculations. As drafting is done for test equipment the progress is logged into a FileMaker Plus file. Each drawing number is entered with its own charge number and the date is entered indicating the start or completion of stages of the drawing process. Occasionally the boss (who is always right because he is the boss) wants a drawing status report according to charge number (there are multiple drawings per charge number). FileMaker Plus will give the total number of drawings but only after searching for each of the charge numbers and dates individually. BASIC to the rescue.

The Drawing Count program was written to count progress of each of the charge numbers. A file is opened for input and another file for output. Each field was previously exported to a BASIC formatted file and is read by the Basic program in the same order which it was written. The charge number is saved, but each of the date fields is replaced by a flag (0 or 1) which indicated if there was anything in the date field. These 0s and 1s are added together to get the totals after the data is sorted. The sort routine could have been beefed up but, for a “quick and dirty” program, it does the job. The program was compiled with the MS Basic compiler which provided adequate speed results.

If you are wondering why ZBasic wasn’t used here, so am I. Actually, I ran into a small problem which appears to be a bug with ZBasic 4.01 while reading the FileMaker Plus exported text files. Although ZBasic’s INPUT# statement is supposed to read data from the file until a carriage return, <COMMA>, End-Of-File, or 255 characters are encountered, when ZBasic saw a comma followed by a carriage return, the null between the comma and carriage return was skipped and the next field (from the next record) was read in. The data looks like this:


The null field after item3 was skipped and ITEM4 is read in. This is how data is formatted when dumped in comma delimited format as with FileMaker Plus. Sorry, Zedcor.

IF filename$=”” THEN END
OPEN “I”,#1,filename$,,vol%
INPUT #1,A$,B$,C$,D$
PRINT @(1,10)A$,B$,C$,D$

Another file conversion problem to deal with involves the conversion of text which has been dumped from text fields. FileMaker Plus, for example, inserts an end of line mark at the end of each line when it exports data to a file. When the data is read into another program which doesn’t use the end of line character to mark the end of lines (because the text is word wrapped and reformattable). If you read the FileMaker text field into Double Helix II for example, the end of line character shows up in the text field as an undefined character (a box). The filter method explained above can find these characters if you know what character it is.

Fortunately life is much easier for converting data these days than it was even five years ago. Now I use file formats like DIF and SYLK whenever I can when converting between programs that support both formats. The transfer goes a lot less painfully when the format is well defined as the DIF format is. For those not familiar with DIF, it is called the “Data Interchange Format” and was originally designed to by Software Arts (are they still around?) to transfer data from Visicalc (the father of all spreadsheet programs). The file is completely ASCII but has header information added to help maintain data integrity. (This saved me a few times). This means that the file will undoubtedly be larger than without DIF formatting so be sure that you only use it for file transport rather than data storage especially if your disk free space is critical. SYLK is Microsoft’s format. Most Microsoft products and a few data base programs such as Omnis 3 and FileMaker Plus support SYLK format. The best way to figure out how these work is to analyze some sample data from a program that exports in one of these formats.

‘ Drawing Count Program
‘ Converts Drawing Log Data created by FileMaker Plus
‘ and computes total number of drawings under each date
‘ By Dave Kelly
‘ ©1988

DIM Charge$(1000),Date1(1000),Date2(1000),Date3(1000), Date4(1000)
DIM tCharge$(20),tDate1(20),tDate2(20),tDate3(20),tDate4(20)
IF filename$=”” THEN END
OPEN “I”,1,filename$
OPEN “O”,2,”Drawing Count.DATA”
PRINT “Now Reading in Records...”
    CALL MOVETO (10,50)
    PRINT count
    IF LEN(Charge$(count))>0 THEN
        IF RIGHT$(Charge$(count),1)=”)” THEN
        END IF
    END IF
    IF LEN(i$)>0 THEN Date1(count)=1 ELSE Date1(count)=0
    IF LEN(A$)>0 THEN Date2(count)=1 ELSE Date2(count)=0
    IF LEN(C$)>0 THEN Date3(count)=1 ELSE Date3(count)=0
    IF LEN(S$)>0 THEN Date4(count)=1 ELSE Date4(count)=0
PRINT “Now Sorting Data...”
CALL Sort(count,Charge$(),Date1(),Date2(),Date3(),Date4())
FOR j=1 TO count
IF tCharge$(k)<>Charge$(j) THEN k=k+1
PRINT “Now Writing to File...”
PRINT#2,”Charge#    Date1    Date2    Date3    Date4"
FOR i=1 TO k
PRINT #2,tCharge$(i),tDate1(i),tDate2(i),tDate3(i),tDate4(i)
PRINT tCharge$(i),tDate1(i),tDate2(i),tDate3(i),tDate4(i)
PRINT”Click to continue...”

SUB Sort(count,Charge$(),Date1(),Date2(),Date3(),Date4()) STATIC
    WHILE flips
        FOR i=2 TO count
            IF Charge$(i-1)> Charge$(i) THEN
                SWAP Charge$(i-1),Charge$(i)
                SWAP Date1(i-1),Date1(i)
                SWAP Date2(i-1),Date2(i)
                SWAP Date3(i-1),Date3(i)
                SWAP Date4(i-1),Date4(i)
            END IF


The new update disk for HyperCard version 1.1 contains a read me stack with three new buttons pertaining to importing and exporting data. If you don’t have the new update you can get it from your Apple dealer. Version 1.1 is supposed to fix some bugs that “bugged” some of us.

The read me stack gives directions for using the new buttons, but I’ll try to explain them a little bit here. To try them out, create a new stack with a few fields in it such as the one I created; see fig. 1.

Fig 1. Hyper Dialog

HyperCard Import/Export

The arrow keys were added from the Button Ideas stack to make it easy to check out the data, but you will need to remember the command 1,2,3,4 keys that move you from card to card or you may use the arrow (or option-arrow) keys to move from field to field.

Fig. 2 Export Dialog

The most useful of the buttons is the Export Data button. It allows you to select the background fields you wish to export and the delimiters separating the fields and records. Most database programs will read the tab delimited records (with carriage return between each record). NO, the export button will not export card fields, only background fields. That is understandable since the output needs to be consistent.

I found that the best way to see how the buttons work is to try them out. Clicking on the Export Data (or Export Text) button will produce this dialog which will let you select the background field you desire to output. (See figure 2)

Selecting the Fields for Export

After selecting the All button or after selecting which of the background fields you want to export, you specify the file name to export to. A dialog comes up to ask you for the name of the text file to export to:

Fig. 3 Editable Dialog

Selecting a Filename for Export

There is an XCMD around someplace for calling the Standard File Dialog which would be better to substitute for this dialog. The problem with XCMD routines for general use is that not everyone has them. It would be best to set up your own custom library of XCMDs to use in your own stacks.

I ran into no difficulty getting my data to export to a file. You can examine your data with a word processor to see how the text file looks when the data has been exported. I like using MS Word for examining this kind of file because you can view all the tabs and carriage returns using the Show ¶ menu item.

The Import Button works well too, except that you can’t import data into existing background fields. It creates the background fields on the fly. Perhaps someone out there could create a button script that will allow you to specify which field you would like to be the first, second and so on field to be read in. AT LEAST, now these buttons are included with HyperCard. At first, it may appear that the import didn’t work, but if you read the instructions in the read me stack, it explains that the new background fields are created one on top of another. To see the fields you will have to peel them apart. (see figure below)

Imported Fields

Ideally, the importing will be done first thing before the stack layout is designed if possible. Importing/Exporting of text with HyperCard has been one of the weak links. With all the interest in HyperCard these days, there should be more XCMD routines and tools we can use to improve the usefulness of HyperCard.


Community Search:
MacTech Search:

Software Updates via MacUpdate

Nisus Writer Pro 2.1.2 - Multilingual wo...
Nisus Writer Pro is a powerful multilingual word processor, similar to its entry level products, but brings new features such as table of contents, indexing, bookmarks, widow and orphan control,... Read more
calibre 2.40.0 - Complete e-book library...
Calibre is a complete e-book library manager. Organize your collection, convert your books to multiple formats, and sync with all of your devices. Let Calibre be your multi-tasking digital librarian... Read more
Vivaldi - An advanced browser...
Vivaldi is a browser for our friends. In 1994, two programmers started working on a web browser. Our idea was to make a really fast browser, capable of running on limited hardware, keeping in mind... Read more
OmniPlan 3.0 - Robust project management...
With OmniPlan, you can create logical, manageable project plans with Gantt charts, schedules, summaries, milestones, and critical paths. Break down the tasks needed to make your project a success,... Read more
Yummy FTP 1.11 - FTP/SFTP/FTPS client fo...
Yummy FTP is an FTP + SFTP + FTPS file transfer client which focuses on speed, reliability and productivity. Whether you need to transfer a few files or a few thousand, schedule automatic backups, or... Read more
Tweetbot 2.1 - Popular Twitter client. (...
Tweetbot is a full-featured OS X Twitter client with a lot of personality. Whether it's the meticulously-crafted interface, sounds and animation, or features like multiple timelines and column views... Read more
MacPilot 8.0 - Enable over 1,200 hidden...
MacPilot gives you the power of UNIX and the simplicity of Macintosh, which means a phenomenal amount of untapped power in your hands! Use MacPilot to unlock over 1,200 features, and access them all... Read more
Typinator 6.7 - Speedy and reliable text...
Typinator turbo-charges your typing productivity. Type a little. Typinator does the rest. We've all faced projects that require repetitive typing tasks. With Typinator, you can store commonly used... Read more
Adobe Lightroom 6.2 - Import, develop, a...
Adobe Lightroom is available as part of Adobe Creative Cloud for as little as $9.99/month bundled with Photoshop CC as part of the photography package. Lightroom 6 is also available for purchase as a... Read more
ForeverSave 2.1.4 - Universal auto-save...
ForeverSave auto-saves all documents you're working on while simultaneously doing backup versioning in the background. Lost data can be quickly restored at any time. Losing data, caused by... Read more

Sago Mini Babies (Education)
Sago Mini Babies 1.0 Device: iOS Universal Category: Education Price: $2.99, Version: 1.0 (iTunes) Description: Introducing the Sago Mini babies. Boys and girls love caring for these adorable characters. Feed Robin her favorite mush... | Read more »
PAUSE - Relaxation at your fingertip (H...
PAUSE - Relaxation at your fingertip 1.1 Device: iOS iPhone Category: Healthcare & Fitness Price: $1.99, Version: 1.1 (iTunes) Description: | Read more »
Super Sharp (Games)
Super Sharp 1.1 Device: iOS Universal Category: Games Price: $1.99, Version: 1.1 (iTunes) Description: Your finger has never been so sharp! Cut with skill to complete the 120 ingenious physics levels of Super Sharp and become a cut... | Read more »
Assembly - Graphic design for everyone...
Assembly - Graphic design for everyone 1.0 Device: iOS Universal Category: Photography Price: $2.99, Version: 1.0 (iTunes) Description: Assembly is the easiest most powerful design tool on the App Store. Create anything you can... | Read more »
Dub Dash (Games)
Dub Dash 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: ARE YOU READY FOR THE ULTIMATE CHALLENGE? UNIQUE SYMBIOSIS OF MUSIC AND GRAPHICS | Read more »
Leave Me Alone (Games)
Leave Me Alone 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: 33% off launch sale!!! Somewhere between the 1980s and 1990s there exists a world that never was. A world of skatepunks,... | Read more »
YAMGUN (Games)
YAMGUN 1.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0 (iTunes) Description: The invasion has begun! Protect the walls of the citadel against waves of enemies! But watch out, you will soon run out of ammo...... | Read more »
Chesh (Games)
Chesh 1.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0 (iTunes) Description: It’s like chess, only not at all. ***40% off for a limited time to celebrate our launch!*** Chesh is a game of skill, strategy, luck,... | Read more »
Dust: An Elysian Tail (Games)
Dust: An Elysian Tail 1.051 Device: iOS Universal Category: Games Price: $5.99, Version: 1.051 (iTunes) Description: *Dust: An Elysian Tail is a visually rich experience and thus requires devices with 1GB RAM**Please note that... | Read more »
Royal Bounty HD (Games)
Royal Bounty HD 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: New World Computing Approved "Hi Guys! looks good so far! keep up the good work. I worked on HoMM 3 and 4 creating all of the... | Read more »

Price Scanner via

Apple’s Education discount saves up to $300 o...
Purchase a new Mac or iPad using Apple’s Education Store and take up to $300 off MSRP. All teachers, students, and staff of any educational institution qualify for the discount. Shipping is free, and... Read more
Save up to $350 with Apple refurbished iMacs
Apple has Certified Refurbished iMacs available for up to $350 off the cost of new models. Apple’s one-year warranty is standard, and shipping is free: - 27″ 3.5GHz 5K iMac – $1949 $350 off MSRP - 27... Read more
Mac Pros on sale for up to $300 off MSRP
B&H Photo has Mac Pros on sale for up to $300 off MSRP. Shipping is free, and B&H charges sales tax in NY only: - 3.7GHz 4-core Mac Pro: $2818.99, $181 off MSRP - 3.5GHz 6-core Mac Pro: $3699... Read more
5K iMacs on sale for up to $150 off MSRP, fre...
B&H Photo has the 27″ 3.3GHz 5K iMac on sale for $1899.99 including free shipping plus NY tax only. Their price is $100 off MSRP. They have the 27″ 3.5GHz 5K iMac on sale for $2149, $150 off MSRP... Read more
Twelve South Redesigns BookArc For Today’s Sm...
Twelve South has announced a redesigned version of their very first product, BookArc for MacBook. Tailored specifically for the newest generation of MacBooks, BookArc holds the new, smaller Apple... Read more
Phone 6s Tips & Tricks – Tips Book For iP...
Poole, United Kingdom based Tap Guides Ltd. has announced the release and immediate availability of iPhone 6s Tips & Tricks, an in-depth eBook available in the iBookstore that’s priced just $2.99... Read more
Apple refurbished 2014 13-inch Retina MacBook...
Apple has Certified Refurbished 2014 13″ Retina MacBook Pros available for up to $400 off original MSRP, starting at $979. An Apple one-year warranty is included with each model, and shipping is free... Read more
13-inch 2.5GHz MacBook Pro on sale for $994,...
Best Buy has the 13″ 2.5GHz MacBook Pro available for $994.99 on their online store. Choose free shipping or free instant local store pickup (if available). Their price is $105 off MSRP. Price valid... Read more
Is The iPad Pro Really A Serious Laptop Repla...
Probably not, at least for productive professionals and other power users. Steve Jobs declared that we’d entered the “post-PC Era” with the advent of the original iPad in 2010, a phrase we don’t hear... Read more
Wednesday Deal: 13-inch Retina MacBook Pros f...
Adorama has 13″ Retina MacBook Pros on sale for up to $130 off MSRP. Shipping is free, and Adorama charges sales tax for NY & NJ residents only: - 13″ 2.7GHz/128GB Retina MacBook Pro: $1199.99 $... Read more

Jobs Board

*Apple* Retail Online, Customer Experience R...
**Job Summary** Apple Retail's Online Store sells Apple products to customers around the world. In addition to selling Apple products with unique services such Read more
Frameworks Engineer, *Apple* Watch - Apple...
**Job Summary** Join the team that is shaping the future of software development for Apple Watch! As a software engineer on the Apple Watch Frameworks team you will Read more
Senior Manager, Global Direct Marketing *App...
**Job Summary** Apple Retail is looking for an experienced Direct Marketing Leader to join its Marketing team. This position will take a leadership role in creating Read more
*Apple* Online Store Expansion - Apple (Unit...
**Job Summary** The Online Apple Store is seeking a person to lead its expansion into new countries. Based in Cupertino, CA, this person will develop and maintain an Read more
*Apple* Retail - Multiple Customer Support P...
Job Description:Customer SupportSpecialist - Retail Customer Service and SalesTransform Apple Store visitors into loyal Apple customers. When customers enter the Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.