TweetFollow Us on Twitter

Still More Perl

Volume Number: 19 (2003)
Issue Number: 1
Column Tag: Section 7

Still More Perl

Munging Mail and Media...

by Rich Morin

Perl's "whipitupitude" is legendary. This column looks at a couple of small scripts I've recently been "whipping up", showing how Perl can work in and around more formal OSX tools. One script, fmmf, Finds Monster Mail Files; I use it to keep track of mailing list (and other) mail files which may be getting out of hand. The other script, cfwc.d, is a daemon (background process) which helps me operate an experimental webcam.

Finding MOnster Mail Files

I'm on quite a few mailing lists and I don't always get to the associated mailboxes regularly to keep them under control. I'm also trying to track the efficacy of my spam filtering system (based on SpamAssassin and Eudora), which drops suspected spam into one of several mailboxes, depending on its numeric spam rating, etc. I have written a short script which helps me keep on top of these issues.

The mainline code, below, is quite simple. Using finddepth, from the File::Find module (available on the CPAN; cpan.perl.org), it performs a depth-first examination of my email folder. The callback function, wanted, is invoked for each node (e.g., file, directory) in the tree. Using the lists produced by this traversal, the remaining code prints out the results for spam and miscellaneous email, sorting each list in a case-insensitive manner.

#!/usr/bin/env perl
#
# fmmf - find monster mail files
#
# Written by Rich Morin, CFCL, 2002.11
use File::Find;
$monster = 2000000;
{
  $eu = '/Users/rdm/Mail/Eudora Folder';
  finddepth(\&wanted, "$eu/Mail Folder");
  for $line (sort {lc($a) cmp lc($b)} (@spam)) {
    print $line;
  }
  print "\n";
  for $line (sort {lc($a) cmp lc($b)} (@misc)) {
    print $line;
  }
}

The tricky parts of this script, such as they are, lie in the "wanted" callback function. As it traverses the tree, finddepth changes the "current directory" and sets $_ to the relative name of the node. This makes it easy to skip over items that aren't files and Eudora's "table of contents" (*.toc) files.

sub wanted {
  return unless (-f $_);
  return if ($_ =~ m|\.toc$|);

For the next part, however, we need the "full path name" of the node. Getting this from a handy helper method, we can strip off the first part of the path and test the remainder in assorted ways. Perl's regular expressions are very useful for this sort of name handling.

  $path = $File::Find::name;
  $path =~ s|^.*/Eudora Folder/Mail Folder/||;
  return if ($path =~ m|_Inactive/ Save/|);

After picking up the size of the file (in bytes), the script opens each mailbox in the "spam" area and counts the number of "From: headers (i.e., messages). Eudora uses carriage returns (rather than the conventional BSD newlines) for line termination, but setting Perl's $/ (input record separator) variable handles that quite easily. The strings containing the formatted output are pushed into a list, for use by the mainline code.

  $size = -s $_;
  if ($path =~ m|!Spam|) {
    open(MBOX, $_) or die "can't open mailbox($_)";
    $/ = "\r";
    $fcnt = 0;
    while (defined($line = <MBOX>)) {
      $fcnt++ if ($line =~ m|^From:|) ;
    } 
    close(MBOX);
    push(@spam, sprintf("%-35s  %9d  %4d\n",
      $path, $size, $fcnt));
    return;
  }

The code for miscellaneous mailboxes is comparatively simple. After ensuring that the mailbox is large enough to qualify as a "monster", it formats and saves the output lines. Perl's "x" operator comes in handy for creating a "quick and dirty" histogram.

  return if ($size < $monster);
  $isiz = int($size/$monster);
  push(@misc, sprintf("%-35s  %9d  %s\n",
    $path, $size, '*' x $isiz));
}

This sort of "personalized" script is quite common in BSD circles. Clearly, it isn't suitable for use by others, as is, but it is short and simple enough that it can easily be customized to meet the needs of different users. Here is some sample output, from my own system:

!Spam/?? Junk (Eudora)                9041     5
!Spam/?? Junk (SA 1)                 39192     6
!Spam/?? Junk (SA 2)                 11467     2
!Spam/?? Junk (SA 3)                420538    60
_Lists/DocBook                     3231686  *
_Lists/FreeBSD/FreeBSD-Ports       6431902  ***
_Lists/FreeBSD/FreeBSD-Questions   2666962  *
...

A WebCam Daemon

I recently started playing with an iBOT, a FireWire-based camera made by Orange Micro

(www.orangemicro.com). My initial goal was to create a simple "security camera" app that would display a set of recent images on a web page.

After downloading the OSX driver for the iBOT, I started looking around for image capture software. One package, EvoCam (www.evological.com), captures images, based on elapsed time and/or software-based motion detection. It can also upload the image files (via FTP) to a web server and/or save numbered copies on the local disk.

Unfortunately, this wasn't exactly what I wanted. The FTP upload feature simply refreshed the same file; turning this into a time history would be tricky. The numbered image files would do, however, if I could get them over to the web server. All told, it was a good start on what I wanted. All I needed to do was create a little plumbing...

The first part of the plumbing had to do with getting the files from my desktop Mac onto the (FreeBSD-based) local web server. FreeBSD provides NFS, but getting OSX to mount the provided volumes can be quite a trial. Fortunately, Marcel Bresink's NFS Manager (www.bresink.de/osx/NFSManager.html) eases the pain considerably.

Once I got the files sifting into a directory on the web server, I merely had to rename them (for convenience) and build up a web page to display a selected subset. The following script, while still a "work in progress", accomplishes these tasks quite handily.

#!/usr/bin/env perl
#
# cfwc.d - Canta Forda WebCam Daemon
#
# Written by Rich Morin, CFCL, 2002.11
$imgs = '/.../iBOT';   # adjust to taste...
$html = '/.../cfwc';   # adjust to taste...
{
  for (;;) {

As mentioned above, EvoCam generates a unique name (e.g., 123456789.jpg) for each image file. In writing these to the NFS-mounted FreeBSD machine, OSX also generates a companion file (e.g., ._123456789.jpg) for the resource fork. The code below creates a new name for the image file, based on the file's modification time, and discards the companion file.

    # Clean out incoming directory.
    opendir(IN, "$imgs/incoming")
      or die "can't open $imgs/incoming";
    @in = grep(!/^\./, readdir(IN));
    chomp(@in);
    closedir(IN);
    for $in (sort(@in)) {
      @stat = stat("$imgs/incoming/$in");
      $mtime = $stat[9];
      ($sec, $min, $hour, $mday, $mon, $year,
       $wday, $yday, $isdst) = localtime($mtime);
      $out = sprintf("%d.%02d%02d.%02d%02d%02d.jpg",
        $year+1900, $mon+1, $mday, $hour, $min, $sec);
      rename("$imgs/incoming/$in",
             "$imgs/i.queue/$out");
      unlink("$imgs/incoming/._$in"); 
    }

Perl's approach to reading directories is rather messy, but it isn't all that difficult. The code below gets a list of filenames, discarding any that don't match the desired format, and sorts them. Because the names were crafted with this in mind, the list is now in chronological order.

    # Get list of images to display.
    opendir(IN, "$imgs/i.queue")
      or die "can't open $imgs/i.queue";
    @in  = sort(grep(/^\d{4}\.\d{4}\.\d{6}\.jpg$/,
                     readdir(IN)));
    chomp(@in);
    closedir(IN);

Using Perl's "slice" syntax, we grab the last (i.e., most recent) nine file names.

    @show = @in[-9 .. -1];

Now we start generating a web page. The META tag tells the user's browser to refresh the page every 15 seconds. I am rather compulsive about formatting the HTML; the web browser doesn't care, but it sure makes debugging less painful for humans!

    # Make up a new web page.
    open(OUT, ">$html/index.temp")
      or die "can't open index.temp";
    print OUT <<EOT;
<HTML>
  <HEAD>
    <META HTTP-EQUIV="Refresh" content="15">
    <TITLE>Canta Forda WebCam</TITLE>
  </HEAD>
  <BODY>
    <TABLE>
EOT

The code below generates a 3x3 table of images, each followed by a centered label. I could have used the file names (e.g., 2002.1129.2039.jpg) as labels, but that would have been a bit ugly. Why not parse the names and reformat the values into a more readable format?

Note the multi-line regular expression that is used to break up the file name. When REs get long and complex, breaking them up in this manner can make them much easier to follow.

    $cnt = 0;
    for ($i=0; $i<9; $i+=3) {
      print OUT "      <TR>\n";
      for ($j=0; $j<3; $j++) {
        print OUT "        <TD>\n";
        $k = $i + $j;
        $tmp1 = $show[$k];
        $tmp1 =~
          m|^(\d{4})\.            # (YYYY).
             (\d\d)(\d\d)\.       # (MM)(DD).
             (\d\d)(\d\d)(\d\d)\. # (HH)(MM)(SS).
             jpg$|x;              # jpg
        $tmp2 = sprintf("%s/%s/%s at %s:%s:%s",
                        $1, $2, $3, $4, $5, $6);
        print OUT "          <CENTER>\n";
        print OUT "            ",
                  "<IMG SRC=\"iq/$tmp1\"><BR>\n";
        print OUT "            $tmp2\n";
        print OUT "          </CENTER>\n";
        $cnt++;
        print OUT "        </TD>\n";
      }
      print OUT "      </TR>\n";
    }

Finally, we push out the last of the HTML, close the file and (Oh, yes!) move it into place for Apache to find. Then, after a second's repose, we go back up and do the whole exercise again.

    print OUT <<EOT;
    </TABLE>
  </BODY>
<HTML>
EOT
    close(OUT);
    rename("$html/index.temp",
           "$html/index.html");
    sleep(1);
  } 
}

Lessons Learned

As we all know, the Mac and BSD universes aren't a perfect fit. Perl is a very good "glue language", however, allowing us to deal smoothly with issues such as line termination, extra (e.g., resource fork) files, etc.

Similarly, there are a wealth of useful apps which can perform small tasks, fill in gaps between operating systems, and generally make our lives easier. If a $20 shareware package can save me hours of frustration, the purchase decision is a no-brainer.

Unfortunately, some issues are still difficult to resolve. For instance, although it's easy to scan a Eudora mail file for header lines, editing Eudora mailboxes would be far trickier. Aside from file locking problems, there is the small issue of the (binary, undocumented) format of the TOC files. In short, choose your challenges carefully...


Rich Morin has been using computers since 1970, Unix since 1983, and Mac-based Unix since 1986 (when he helped Apple create A/UX 1.0). When he isn't writing this column, Rich runs Prime Time Freeware (www.ptf.com), a publisher of books and CD-ROMs for the Free and Open Source software community. Feel free to write to Rich at rdm@ptf.com.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Together 3.6.1 - Store and organize all...
Together helps you organize your Mac, giving you the ability to store, edit and preview your files in a single clean, uncluttered interface. Features Smart storage. With simple drag-and-drop... Read more
Cloud 4.1.1 - File sharing from your men...
Cloud is simple file sharing for the Mac. Drag a file from your Mac to the CloudApp icon in the menubar and we take care of the rest. A link to the file will automatically be copied to your clipboard... Read more
OmniFocus 2.7.1 - GTD task manager with...
OmniFocus helps you manage your tasks the way that you want, freeing you to focus your attention on the things that matter to you most. Capturing tasks and ideas is always a keyboard shortcut away in... Read more
CleanApp 5.1.1 - Application deinstaller...
CleanApp is an application deinstaller and archiver.... Your hard drive gets fuller day by day, but do you know why? CleanApp 5 provides you with insights how to reclaim disk space. There are... Read more
ForkLift 3.0 Beta 2 - Powerful file mana...
ForkLift is a powerful file manager and ferociously fast FTP client clothed in a clean and versatile UI that offers the combination of absolute simplicity and raw power expected from a well-executed... Read more
Sublime Text 3126 - Sophisticated text e...
Sublime Text is a sophisticated text editor for code, markup, and prose. You'll love the slick user interface, extraordinary features, and amazing performance. Features Goto Anything. Use Goto... Read more
1Password 6.3.3 - Powerful password mana...
1Password is a password manager that uniquely brings you both security and convenience. It is the only program that provides anti-phishing protection and goes beyond password management by adding Web... Read more
WhatsApp 0.2.1880 - Desktop client for W...
WhatsApp is the desktop client for WhatsApp Messenger, a cross-platform mobile messaging app which allows you to exchange messages without having to pay for SMS. WhatsApp Messenger is available for... Read more
NeoFinder 6.9.3 - Catalog your external...
NeoFinder (formerly CDFinder) rapidly organizes your data, either on external or internal disks, or any other volumes. It catalogs all your data, so you stay in control of your data archive or disk... Read more
Amadeus Pro 2.3.1 - Multitrack sound rec...
Amadeus Pro lets you use your Mac computer for any audio-related task, such as live audio recording, digitizing tapes and records, converting between a variety of sound formats, etc. Thanks to its... Read more

Our 5 Favorite iMessage Sticker Packs
At long last, iMessage joins the ranks of messaging apps the likes of LINE and Whatsapp, adding an impressive collection of stickers. They’re a great way to add a little something extra to your daily conversations. [Read more] | Read more »
How to get past Vulture Island's tr...
Vulture Island is a colorful and quirky mish-mash of platforming and puzzles. It’s creative and fresh, but sometimes the game can throw a curveball at you, leaving you stuck as to how you should progress. These tips will help you explore smoothly... | Read more »
The new Clash of Kings is just for Weste...
If you’ve played the original Clash of Kings, you’ll probably recognise the city building, alliance forging and strategic battles in Clash of Kings: The West. What sets this version apart is that it’s tailor made for a Western audience and the... | Read more »
Frost - Survival card game (Games)
Frost - Survival card game 1.12.1 Device: iOS Universal Category: Games Price: $3.99, Version: 1.12.1 (iTunes) Description: *Warning: the game will work on iPhone 5C and above and iPad Pro / 4. Other devices are not supported* | Read more »
How to build and care for your team in D...
Before you hit the trail and become a dog sledding legend, there’s actually a fair bit of prep work to be done. In Dog Sled Saga, you’re not only racing, you’re also building and caring for a team of furry friends. There’s a lot to consider—... | Read more »
How to win every race in Dog Sled Saga
If I had to guess, I’d say Dog Sled Saga is the most adorable racing game on the App Store right now. It’s a dog sled racing sim full of adorable, loyal puppies. Just look at those fluffy little tails wagging. Behind that cute, pixelated facade is... | Read more »
Let the war games commence in Gunship Ba...
Buzz Lightyear famously said, “This isn’t flying, this is falling – with style!” In the case of Gunship Battle: Second War, though, this really is flying - with style! The flight simulator app from Joycity puts you in control of 20 faithfully... | Read more »
How to get a high score in Fired Up
Fired Up is Noodlecake Games’ high score chasing, firefighting adventure. You take control of a wayward firefighter who propels himself up the side of a highrise with blasts of water. Sound silly? It is. It’s also pretty difficult. You can’t... | Read more »
NBA 2K17 (Games)
NBA 2K17 1.0 Device: iOS iPhone Category: Games Price: $7.99, Version: 1.0 (iTunes) Description: Following the record-breaking launch of NBA 2K16, the NBA 2K franchise continues to stake its claim as the most authentic sports video... | Read more »
Dog Sled Saga (Games)
Dog Sled Saga 1.0.1 Device: iOS Universal Category: Games Price: $3.99, Version: 1.0.1 (iTunes) Description: A game by Dan + Lisa As a rookie musher, foster a dogsledding team whose skills will grow if they're treated right. Week by... | Read more »

Price Scanner via MacPrices.net

Toughbook Celebrates 20 Years of Ruggedized M...
Panasonic System Communications Company of North America, Division of Panasonic Corporation of North America (Panasonic) today celebrates the 20th anniversary of its industry-leading Toughbook mobile... Read more
12-inch 1.1GHz Gray Retina MacBook on sale fo...
B&H Photo has the 2016 12″ 1.1GHz Gray Retina MacBook on sale for $1199.99 including free shipping plus NY sales tax only. Their price is $100 off MSRP. Read more
13-inch 2.5GHz MacBook Pro (Apple refurbished...
Apple has Certified Refurbished 13″ 2.5GHz MacBook Pros available for $829, or $270 off the cost of new models. Apple’s one-year warranty is standard, and shipping is free: - 13″ 2.5GHz MacBook Pros... Read more
Save 30% on Camera Guard’s Secure Protection...
To celebrate the release of macOS Sierra, Miami-based security solutions company, ProtectStar has announced a special 30% discount on Camera Guard Professional for Mac 2016. This innovative security... Read more
DVDFab Special Deal – Get a 1-Year Free Licen...
Beijing, China based specialist in the field of DVD, Blu-ray and video backup solutions, Fengtao Software has launched its Autumn Special Deals 2016, giving a 1-year free license of a randomly picked... Read more
21-inch iMacs on sale for up to $120 off MSRP
B&H Photo has 21″ iMacs on sale for up to $120 off MSRP including free shipping plus NY sales tax only: - 21″ 3.1GHz iMac 4K: $1379 $120 off MSRP - 21″ 2.8GHz iMac: $1199.99 $100 off MSRP - 21″ 1... Read more
13-inch 2.7GHz/256GB Retina MacBook Pro on sa...
Amazon.com has the 13″ 2.7GHz/256GB Retina Apple MacBook Pro on sale for $151 off MSRP including free shipping: - 13″ 2.7GHz/256GB Retina MacBook Pro (sku MF840LL/A): $1348 $151 off MSRP Read more
Apple TVs on sale for up to $50 off MSRP
Best Buy has 32GB and 64GB Apple TVs on sale for $40-$50 off MSRP on their online store. Choose free shipping or free local store pickup (if available). Sale prices for online orders only, in-store... Read more
Apple refurbished 13-inch Retina MacBook Pros...
Apple has Certified Refurbished 13″ Retina MacBook Pros available for up to $270 off the cost of new models. An Apple one-year warranty is included with each model, and shipping is free: - 13″ 2.7GHz... Read more
Duplicate Sweeper Free On Mac App Store For O...
To celebrate the launch of Apple’s latest macOS Sierra, Stafford, United Kingdom based Wide Angle Software has announced that its duplicate file finder software, Duplicate Sweeper, is now available... Read more

Jobs Board

Sr. *Apple* Mac Engineer - Net2Source Inc....
…staffing, training and technology. We have following position open with our client. Sr. Apple Mac Engineer6+ Months CTH Start date : 19th Sept Travelling Job If Read more
*Apple* Retail - Multiple Positions-Norfolk,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
Restaurant Manager (Neighborhood Captain) - A...
…in every aspect of daily operation. WHY YOU'LL LIKE IT: You'll be the Big Apple . You'll solve problems. You'll get to show your ability to handle the stress and Read more
Lead *Apple* Solutions Consultant - Apple (...
# Lead Apple Solutions Consultant Job Number: 51829230 Detroit, Michigan, United States Posted: Sep. 19, 2016 Weekly Hours: 40.00 **Job Summary** The Lead ASC is an Read more
US- *Apple* Store Leader Program - Apple (Un...
…Summary Learn and grow as you explore the art of leadership at the Apple Store. You'll master our retail business inside and out through training, hands-on Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.