TweetFollow Us on Twitter

Still More Perl

Volume Number: 19 (2003)
Issue Number: 1
Column Tag: Section 7

Still More Perl

Munging Mail and Media...

by Rich Morin

Perl's "whipitupitude" is legendary. This column looks at a couple of small scripts I've recently been "whipping up", showing how Perl can work in and around more formal OSX tools. One script, fmmf, Finds Monster Mail Files; I use it to keep track of mailing list (and other) mail files which may be getting out of hand. The other script, cfwc.d, is a daemon (background process) which helps me operate an experimental webcam.

Finding MOnster Mail Files

I'm on quite a few mailing lists and I don't always get to the associated mailboxes regularly to keep them under control. I'm also trying to track the efficacy of my spam filtering system (based on SpamAssassin and Eudora), which drops suspected spam into one of several mailboxes, depending on its numeric spam rating, etc. I have written a short script which helps me keep on top of these issues.

The mainline code, below, is quite simple. Using finddepth, from the File::Find module (available on the CPAN; cpan.perl.org), it performs a depth-first examination of my email folder. The callback function, wanted, is invoked for each node (e.g., file, directory) in the tree. Using the lists produced by this traversal, the remaining code prints out the results for spam and miscellaneous email, sorting each list in a case-insensitive manner.

#!/usr/bin/env perl
#
# fmmf - find monster mail files
#
# Written by Rich Morin, CFCL, 2002.11
use File::Find;
$monster = 2000000;
{
  $eu = '/Users/rdm/Mail/Eudora Folder';
  finddepth(\&wanted, "$eu/Mail Folder");
  for $line (sort {lc($a) cmp lc($b)} (@spam)) {
    print $line;
  }
  print "\n";
  for $line (sort {lc($a) cmp lc($b)} (@misc)) {
    print $line;
  }
}

The tricky parts of this script, such as they are, lie in the "wanted" callback function. As it traverses the tree, finddepth changes the "current directory" and sets $_ to the relative name of the node. This makes it easy to skip over items that aren't files and Eudora's "table of contents" (*.toc) files.

sub wanted {
  return unless (-f $_);
  return if ($_ =~ m|\.toc$|);

For the next part, however, we need the "full path name" of the node. Getting this from a handy helper method, we can strip off the first part of the path and test the remainder in assorted ways. Perl's regular expressions are very useful for this sort of name handling.

  $path = $File::Find::name;
  $path =~ s|^.*/Eudora Folder/Mail Folder/||;
  return if ($path =~ m|_Inactive/ Save/|);

After picking up the size of the file (in bytes), the script opens each mailbox in the "spam" area and counts the number of "From: headers (i.e., messages). Eudora uses carriage returns (rather than the conventional BSD newlines) for line termination, but setting Perl's $/ (input record separator) variable handles that quite easily. The strings containing the formatted output are pushed into a list, for use by the mainline code.

  $size = -s $_;
  if ($path =~ m|!Spam|) {
    open(MBOX, $_) or die "can't open mailbox($_)";
    $/ = "\r";
    $fcnt = 0;
    while (defined($line = <MBOX>)) {
      $fcnt++ if ($line =~ m|^From:|) ;
    } 
    close(MBOX);
    push(@spam, sprintf("%-35s  %9d  %4d\n",
      $path, $size, $fcnt));
    return;
  }

The code for miscellaneous mailboxes is comparatively simple. After ensuring that the mailbox is large enough to qualify as a "monster", it formats and saves the output lines. Perl's "x" operator comes in handy for creating a "quick and dirty" histogram.

  return if ($size < $monster);
  $isiz = int($size/$monster);
  push(@misc, sprintf("%-35s  %9d  %s\n",
    $path, $size, '*' x $isiz));
}

This sort of "personalized" script is quite common in BSD circles. Clearly, it isn't suitable for use by others, as is, but it is short and simple enough that it can easily be customized to meet the needs of different users. Here is some sample output, from my own system:

!Spam/?? Junk (Eudora)                9041     5
!Spam/?? Junk (SA 1)                 39192     6
!Spam/?? Junk (SA 2)                 11467     2
!Spam/?? Junk (SA 3)                420538    60
_Lists/DocBook                     3231686  *
_Lists/FreeBSD/FreeBSD-Ports       6431902  ***
_Lists/FreeBSD/FreeBSD-Questions   2666962  *
...

A WebCam Daemon

I recently started playing with an iBOT, a FireWire-based camera made by Orange Micro

(www.orangemicro.com). My initial goal was to create a simple "security camera" app that would display a set of recent images on a web page.

After downloading the OSX driver for the iBOT, I started looking around for image capture software. One package, EvoCam (www.evological.com), captures images, based on elapsed time and/or software-based motion detection. It can also upload the image files (via FTP) to a web server and/or save numbered copies on the local disk.

Unfortunately, this wasn't exactly what I wanted. The FTP upload feature simply refreshed the same file; turning this into a time history would be tricky. The numbered image files would do, however, if I could get them over to the web server. All told, it was a good start on what I wanted. All I needed to do was create a little plumbing...

The first part of the plumbing had to do with getting the files from my desktop Mac onto the (FreeBSD-based) local web server. FreeBSD provides NFS, but getting OSX to mount the provided volumes can be quite a trial. Fortunately, Marcel Bresink's NFS Manager (www.bresink.de/osx/NFSManager.html) eases the pain considerably.

Once I got the files sifting into a directory on the web server, I merely had to rename them (for convenience) and build up a web page to display a selected subset. The following script, while still a "work in progress", accomplishes these tasks quite handily.

#!/usr/bin/env perl
#
# cfwc.d - Canta Forda WebCam Daemon
#
# Written by Rich Morin, CFCL, 2002.11
$imgs = '/.../iBOT';   # adjust to taste...
$html = '/.../cfwc';   # adjust to taste...
{
  for (;;) {

As mentioned above, EvoCam generates a unique name (e.g., 123456789.jpg) for each image file. In writing these to the NFS-mounted FreeBSD machine, OSX also generates a companion file (e.g., ._123456789.jpg) for the resource fork. The code below creates a new name for the image file, based on the file's modification time, and discards the companion file.

    # Clean out incoming directory.
    opendir(IN, "$imgs/incoming")
      or die "can't open $imgs/incoming";
    @in = grep(!/^\./, readdir(IN));
    chomp(@in);
    closedir(IN);
    for $in (sort(@in)) {
      @stat = stat("$imgs/incoming/$in");
      $mtime = $stat[9];
      ($sec, $min, $hour, $mday, $mon, $year,
       $wday, $yday, $isdst) = localtime($mtime);
      $out = sprintf("%d.%02d%02d.%02d%02d%02d.jpg",
        $year+1900, $mon+1, $mday, $hour, $min, $sec);
      rename("$imgs/incoming/$in",
             "$imgs/i.queue/$out");
      unlink("$imgs/incoming/._$in"); 
    }

Perl's approach to reading directories is rather messy, but it isn't all that difficult. The code below gets a list of filenames, discarding any that don't match the desired format, and sorts them. Because the names were crafted with this in mind, the list is now in chronological order.

    # Get list of images to display.
    opendir(IN, "$imgs/i.queue")
      or die "can't open $imgs/i.queue";
    @in  = sort(grep(/^\d{4}\.\d{4}\.\d{6}\.jpg$/,
                     readdir(IN)));
    chomp(@in);
    closedir(IN);

Using Perl's "slice" syntax, we grab the last (i.e., most recent) nine file names.

    @show = @in[-9 .. -1];

Now we start generating a web page. The META tag tells the user's browser to refresh the page every 15 seconds. I am rather compulsive about formatting the HTML; the web browser doesn't care, but it sure makes debugging less painful for humans!

    # Make up a new web page.
    open(OUT, ">$html/index.temp")
      or die "can't open index.temp";
    print OUT <<EOT;
<HTML>
  <HEAD>
    <META HTTP-EQUIV="Refresh" content="15">
    <TITLE>Canta Forda WebCam</TITLE>
  </HEAD>
  <BODY>
    <TABLE>
EOT

The code below generates a 3x3 table of images, each followed by a centered label. I could have used the file names (e.g., 2002.1129.2039.jpg) as labels, but that would have been a bit ugly. Why not parse the names and reformat the values into a more readable format?

Note the multi-line regular expression that is used to break up the file name. When REs get long and complex, breaking them up in this manner can make them much easier to follow.

    $cnt = 0;
    for ($i=0; $i<9; $i+=3) {
      print OUT "      <TR>\n";
      for ($j=0; $j<3; $j++) {
        print OUT "        <TD>\n";
        $k = $i + $j;
        $tmp1 = $show[$k];
        $tmp1 =~
          m|^(\d{4})\.            # (YYYY).
             (\d\d)(\d\d)\.       # (MM)(DD).
             (\d\d)(\d\d)(\d\d)\. # (HH)(MM)(SS).
             jpg$|x;              # jpg
        $tmp2 = sprintf("%s/%s/%s at %s:%s:%s",
                        $1, $2, $3, $4, $5, $6);
        print OUT "          <CENTER>\n";
        print OUT "            ",
                  "<IMG SRC=\"iq/$tmp1\"><BR>\n";
        print OUT "            $tmp2\n";
        print OUT "          </CENTER>\n";
        $cnt++;
        print OUT "        </TD>\n";
      }
      print OUT "      </TR>\n";
    }

Finally, we push out the last of the HTML, close the file and (Oh, yes!) move it into place for Apache to find. Then, after a second's repose, we go back up and do the whole exercise again.

    print OUT <<EOT;
    </TABLE>
  </BODY>
<HTML>
EOT
    close(OUT);
    rename("$html/index.temp",
           "$html/index.html");
    sleep(1);
  } 
}

Lessons Learned

As we all know, the Mac and BSD universes aren't a perfect fit. Perl is a very good "glue language", however, allowing us to deal smoothly with issues such as line termination, extra (e.g., resource fork) files, etc.

Similarly, there are a wealth of useful apps which can perform small tasks, fill in gaps between operating systems, and generally make our lives easier. If a $20 shareware package can save me hours of frustration, the purchase decision is a no-brainer.

Unfortunately, some issues are still difficult to resolve. For instance, although it's easy to scan a Eudora mail file for header lines, editing Eudora mailboxes would be far trickier. Aside from file locking problems, there is the small issue of the (binary, undocumented) format of the TOC files. In short, choose your challenges carefully...


Rich Morin has been using computers since 1970, Unix since 1983, and Mac-based Unix since 1986 (when he helped Apple create A/UX 1.0). When he isn't writing this column, Rich runs Prime Time Freeware (www.ptf.com), a publisher of books and CD-ROMs for the Free and Open Source software community. Feel free to write to Rich at rdm@ptf.com.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

FotoMagico 5.4.1 - Powerful slideshow cr...
FotoMagico lets you create professional slideshows from your photos and music with just a few, simple mouse clicks. It sports a very clean and intuitive yet powerful user interface. High image... Read more
Espresso 3.0 - Powerful HTML, XML, CSS,...
Espresso is back. For people who make delightful, innovative, and fast websites -- in an app to match. Espresso helps you write, code, design, build, and publish with flair and efficiency.... Read more
Spotify 1.0.51.693. - Stream music, crea...
Spotify is a streaming music service that gives you on-demand access to millions of songs. Whether you like driving rock, silky R&B, or grandiose classical music, Spotify's massive catalogue puts... Read more
Gutenprint 5.2.12 - Quality drivers for...
Gutenprint is a suite of printer drivers that may be used with most common UNIX print spooling systems, including CUPS, lpr, LPRng, or others. Gutenprint currently supports over 2000 printer models.... Read more
Firefox 52.0.2 - Fast, safe Web browser.
Firefox offers a fast, safe Web browsing experience. Browse quickly, securely, and effortlessly. With its industry-leading features, Firefox is the choice of Web development professionals and casual... Read more
Google Chrome 57.0.2987.133 - Modern and...
Google Chrome is a Web browser by Google, created to be a modern platform for Web pages and applications. It utilizes very fast loading of Web pages and has a V8 engine, which is a custom built... Read more
RapidWeaver 7.3.3 - Create template-base...
RapidWeaver is a next-generation Web design application to help you easily create professional-looking Web sites in minutes. No knowledge of complex code is required, RapidWeaver will take care of... Read more
Chromium 57.0.2987.133 - Fast and stable...
Chromium is an open-source browser project that aims to build a safer, faster, and more stable way for all Internet users to experience the web. Version 27.0.2987.133: Note: This update has no Flash... Read more
Lyn 1.8.8 - Lightweight image browser an...
Lyn is a fast, lightweight image browser and viewer designed for photographers, graphic artists, and Web designers. Featuring an extremely versatile and aesthetically pleasing interface, it delivers... Read more
Adobe Animate CC 2017 16.2.0 - Advanced...
Animate CC 2017 is available as part of Adobe Creative Cloud for as little as $19.99/month (or $9.99/month if you're a previous Flash Professional customer). Animate CC 2017 (was Flash CC) lets you... Read more

Dynasty Warriors: Unleashed arrives on m...
No one does mindless hack and slash quite like a Musou game. What's more fun than single-handedly chopping your way through hundreds of enemies? Now, for the first time ever, that unrivaled experience goes mobile with Dynasty Warriors: Unleashed.... | Read more »
Crash of Cars beginner's guide
Crash of Cars is a fun new entry in the realm of multiplayer beat 'em ups. It's basically bumper cars off the rails -- players drive around in vehicles of their choosing -- anything from a mini-van to the weinermobile in all of its glory --... | Read more »
Dynasty Blades new update introduces a n...
Sharpen your weapons -- Dynasty Blades is back with new and improved hack n’ slash stylings. The Romance of the Three Kingdoms-inspired action MMORPG introduces a bunch of fun new features in its latest update. For the uninitiated, Dynasty Blades... | Read more »
Meganoid(2017) (Games)
Meganoid(2017) 1.0 Device: iOS Universal Category: Games Price: $3.99, Version: 1.0 (iTunes) Description: LAUNCH DISCOUNT 20% UNTIL APRIL 2nd! Support, tip and tricks: http://www.orangepixel.net/forum/ Subscribe to our newsletter... | Read more »
Telltale's Guardians of the Galaxy...
Telltale will be releasing their rendition of Guardians of the Galaxy later this month. The first episode, Tangled Up in Blue, features familiar faces including Star-Lord, Groot, Rocket, Gamora, and Drax. If the first episode's title is any... | Read more »
Royal Dungeon (Games)
Royal Dungeon 1.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0 (iTunes) Description: The king and his queen are trapped in their castle which suddenly turned out as a very dangerous place. The goal is to escape... | Read more »
Tom Clancy's ShadowBreak is a real-...
Ubisoft is treating Tom Clancy fans to the series' very first mobile-exclusive game in ShadowBreak, a real-time, multiplayer shooter in which players snipe at enemies in fast-paced tactics-driven combat. [Read more] | Read more »
Power Rangers: Legacy Wars beginner...
Rita Repulsa is back, but this time she's invading your mobile phone in Power Rangers: Legacy Wars. What looks to be a straightforward beat 'em up is actually a tough-as-nails multiplayer strategy game that requires some deft tactical maneuvering.... | Read more »
Hearthstone celebrates the upcoming Jour...
Hearthstone gets a new expansion, Journey to Un'Goro, in a little over a week, and they'll be welcoming the Year of the Mammoth, the next season, at the same time. There's a lot to be excited about, so Blizzard is celebrating in kind. Players will... | Read more »
4 smart and stylish puzzle games like Ty...
TypeShift launched a little over a week ago, offering some puzzling new challenges for word nerds equipped with an iOS device. Created by Zach Gage, the mind behind Spelltower, TypeShift boasts, like its predecessor, a sleak design and some very... | Read more »

Price Scanner via MacPrices.net

Deal alert! 13-inch 2.0GHz MacBook Pros now f...
B&H Photo has lowered their price on 13″ Apple 2.0GHz MacBook Pros to $225 off MSRP through midnight tonight (3/30/17). Shipping is free, and B&H charges NY sales tax only: - 13″ 2.0GHz... Read more
1.4GHz Mac mini on sale for $399, $100 off MS...
B&H Photo has the 1.4GHz Mac mini on sale for $100 off MSRP including free shipping plus NY sales tax only: - 1.4GHz Mac mini: $399 $100 off MSRP Sale ends on March 31st. Read more
13-inch 128GB MacBook Air on sale for $849, s...
B&H Photo has lowered their price on the 13″ 1.6GHz/128GB MacBook Air to $849, or $150 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 1.6GHz/128GB MacBook Air (MMGF2LL/A... Read more
Is Apple Planning An iPhone Based Modular Doc...
Today’s more powerful and larger-screened smartphones and phablets are becoming the default anchor computing device for more and more users computing devices, but even a five or six inch panel is not... Read more
Razer Launches New Razer Blade Pro World’s Fi...
Razer, the gaming and high performance hardware specialists, have announced the new Razer Blade Pro laptop — the first laptop to be qualified for THX Mobile Certification, an accreditation reserved... Read more
Gro CRM’s Apple Small Business Mac And iOS CR...
Gro Software, developers of the Mac CRM software for small business and enterprise, are included in FinancesOnline 2017 CRM Rising Stars and Great User Experience lists by business software review... Read more
Deal alert! 15-inch and 13-inch MacBook Pros...
B&H Photo has the new 2016 15″ and 13″ Apple MacBook Pros in stock today and on sale for up to $200 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 15″ 2.7GHz Touch Bar... Read more
Save up to $420 on a new MacBook Pro with App...
Apple is offering Certified Refurbished 2016 15″ and 13″ MacBook Pros, including some Touch Bar models, for up to $420 off original MSRP. An Apple one-year warranty is included with each model, and... Read more
12-inch 1.2GHz Retina MacBooks on sale for $1...
B&H has 12″ 1.2GHz Retina MacBooks on sale for up to $200 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 12″ 1.2GHz Space Gray Retina MacBook: $1449 $150 off MSRP - 12″ 1.... Read more
Is A New 10.5-inch iPad Still Coming In April...
There was no sign or mention of a long-rumored and much anticipated 10.5-inch iPad Pro in Apple’s product announcements last week. The exciting iPad news was release of an upgraded iPad Air with a... Read more

Jobs Board

Fulltime aan de slag als shopmanager in een h...
Ben jij helemaal gek van Apple -producten en vind je het helemaal super om fulltime shopmanager te zijn in een jonge en hippe elektronicazaak? Wil jij werken in Read more
Fulltime aan de slag als shopmanager in een h...
Ben jij helemaal gek van Apple -producten en vind je het helemaal super om fulltime shopmanager te zijn in een jonge en hippe elektronicazaak? Wil jij werken in Read more
Fulltime aan de slag als shopmanager in een h...
Ben jij helemaal gek van Apple -producten en vind je het helemaal super om fulltime shopmanager te zijn in een jonge en hippe elektronicazaak? Wil jij werken in Read more
Desktop Analyst - *Apple* Products - Montef...
…technology to improve patient care. JOB RESPONSIBILITIES: Provide day-to-day support for Apple Hardware and Software in the environment based on the team's support Read more
*Apple* Mobile Master - Best Buy (United Sta...
**493168BR** **Job Title:** Apple Mobile Master **Location Number:** 000827-Denton-Store **Job Description:** **What does a Best Buy Apple Mobile Master do?** At Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.