Volume Number: 23 (2007)
Issue Number: 04
Column Tag: Geek Toys

Lego NXT on the Mac

Lego's next generation Mindstorm kits play nice with the Mac.

By Rich Warren

I admit it. I have had a long-term love/hate affair with the Lego Mindstorm line. I love the idea of building robots from preformed plastic bricks, but the actual kits always failed to live up to my expectations. Sure, the Robotic Invention System seemed cool when I first opened the box, but I soon found the limited selection of motors and sensors quite...um...limiting.

Of course, the software side was even worse. Lego's graphical programming environment only ran on Windows. Sure, you could find third party programming tools that offered partial Mac support, but these frequently lagged behind their Windows or Linux counterparts. Programming the Mindstorm brick from the Mac often devolved into an exercise in frustration.

So, when Lego announced the new Mindstorm NXT, I greeted the news with a fair bit of cynicism. At least, that was my intention, but part of me still wanted to believe. I grew curious, and peeked--just peeked--at the press releases. The more I read, the more intrigued I became.

The Mindstorm NXT offers several significant improvements over earlier models. It provides a more-interesting range of motors and sensors straight out of the box. Also, Lego's software now runs under OS X; you can even purchase the NXT kit at your local Apple store. Finally, Lego has open sourced their firmware. By providing software, hardware and Bluetooth developer kits, they give the community unprecedented access to the brick's inner workings. Overall, the new Lego Mindstorm NXT is a promising platform for anyone interested in hobby robotics, and it is especially nice for Mac lovers.

The brick comes with four input ports (one doubles as an expansion slot for a still-hypothetical sensor hub) and three output ports. The nice-but-small LCD can display black and white text and graphics, while the brick's loudspeaker plays a variety of NXT sound files. Unfortunately, the brick can only play .rso files. However, you can find Mac-friendly utilities that convert .wav and MIDI files at http://bricxcc.sourceforge.net/utilities.html.

You connect your computer to the brick using either a USB cable or a Bluetooth wireless connection. The wireless connection is very cool. The brick can link with a computer, with other bricks or even with Bluetooth phones or PDAs. The Mindstorm community has already produced a number of PDA and cell phone remote controls. I suspect we will see interesting Bluetooth sensors in the future.

The kit comes with three motors. This is the NXT's biggest limitation, since (unlike sensors), with only three ports, this looks like a hard-and-fast limit. However, including a third motor in the basic kit feels like big improvement over the RIS 2.0, which only came with two. The new servomotors are bigger than the old motors, but they include a built-in rotation sensor. This gives you a lot of fine control over the robot's drive train.

The kit also comes with four sensors: a touch sensor, a light sensor, a sound sensor and an ultrasonic range finder. The touch and light sensor are bigger than, but otherwise similar to, their RIS 2.0 equivalents.

The sound sensor can detect the volume of the ambient sound in either adjusted or standard decibels. The dBA setting focuses on sounds within human hearing, while the dB setting includes sounds too high or too low for our ears. In either case, it can distinguish volumes up to about 90 dB, which is roughly equal to a typical lawnmower.

The Ultrasonic Sensor uses sonar to determine the range to objects. It can detect items from 0 to 255 cm (a little over 8 feet 4 inches), with an accuracy of 3 cm. However, this accuracy depends a lot on the object you are sensing. Ultrasonic sensors work best when approaching hard, flat objects straight on. If you approach the wall at too steep an angle, a lot of the sound may reflect away from you. For similar reasons, the sensor struggles with round, soft or thin objects. Around my apartment, the robot keeps running into my beanbag (probably because of the roundish shape and the soft cover), and it cannot see the screen door leading to our balcony. Other than that, it does a reasonably good job.

Finally, no discussion of hardware would be complete without mentioning the third party expansions. Several companies offer new, high-quality sensors: everything from magnetic compasses to DYI prototype boards. Even Lego's getting in on the hardware-hacking game. You can buy legacy cables on their online store, letting you connect old RIS 2.0 motors and sensors to the new, NXT brick.

Overall the NXT seems well designed with an eye towards future expansion. Sure, it still has its limits, but it is much more promising than any of the earlier Mindstorm products.

Lego's Mindstorm software lets you program and manage the files on your brick. Programs are written using a drag-and-drop graphical programming language, referred to as NXT-G. Powered by National Instruments' LabVIEW, NXT-G represents a huge improvement over the graphical programming tool used by the RIS 2.0. Great for simple projects or quick prototypes, it still becomes frustrating when you try to create larger or more complicated programs.

If you want even more power, you can use the NXT toolkit for LabVIEW, but LabVIEW is not cheap. They offer a free, academic version to Mindstorm owners--but the free version only runs on Windows. If you have a spare $1,199 lying around, you can purchase the Mac version. I don't. So let's talk about something else.

If graphical programming is not your cup of tea, you can find a growing number of third-party tools scattered across the web. I will only mention the Mac-friendly ones in this article. This includes: LeJOS NXJ and iCommand from the LeJOS team (http://lejos.sourceforge.net/), Next Byte Codes (NBC) and Not eXactly C (NXC) from the people at Bricx Command Center (http://bricxcc.sourceforge.net/nbc/),and finally Matt Zukowski and Tony Burser's ruby-nxt library (http://rubyforge.org/projects/ruby-nxt/).

These third-party projects can be divided into three main camps: software that runs on a computer and sends commands to the brick over Bluetooth (iCommand and ruby-nxt), languages that compile to the same bytecode used by the existing NXT firmware (NBC and NXC), and systems that completely replace the brick's firmware (LeJOS NXJ). I will briefly examine each of these options, but be warned: these projects are still green, and most of them are undergoing frantic development. Anything I say now will undoubtedly be outdated by the time this article prints.

Of course, there has to be a catch somewhere. As I am writing this, the Lego software is not available as a universal binary. It only runs under Rosetta, and this creates some problems.

First, Rosetta does not recognize the Bluetooth adapter on my MacBook Pro. So, I can only communicate with my brick using the USB cable. While that definitely knocks the software down a few pegs on the coolness-O-meter, it's not (by itself) a deal breaker. Unfortunately, the Lego software also seems slow and buggy. Dropping in a series of simple commands works fine, but don't even think about using data wires. Don't get me wrong; the software works great on my wife's PowerMac. The Windows version also runs smooth-as-silk under Parallels. But if you have to use the Lego software on an Intel Mac, save often and don't try anything too complicated.

Still, all hope's not lost. If you have an Intel Mac, you can manually configure the Bluetooth connection (see http://juju.org/articles/2006/10/22/bluetooth-serial-port-to-nxt-in-osx for good instructions). This won't help you with Lego's software, but it does let you use iCommand and ruby-nxt. I also recommend checking out the NXTBrowser for uploading and managing files over a Bluetooth connection. (http://web.mac.com/carstenm/iWeb/Lego/NXT/NXT.html).

The rest of this article focuses on programming the tribot robot. If you followed the instructions when you opened the NXT box, it is the first robot you built. It's a differentially steered, three-wheeled robot with snapping mandibles--you can't miss it. If, like me, you've lost the original instructions, you can find a digital version in the Mindstorm software's Robo Center.

We will only use the ultrasonic sensor in these samples, but go ahead and make the whole contraption--after all, snapping mandibles are just too cool!

Launch the Mindstorm NXT application. In the Start New Program text box, type wanderer and click Go.

This gives you a blank grid. You can create your program by dragging blocks from the toolbar. Let's start by dragging a loop block and dropping it onto the sequence beam as shown.

This is our program's main loop. We want it to run forever, which should be the default. To double-check, select the loop block, then look at the control settings below the workspace. Select Forever from the drop-down options.

Now drop a switch block inside the loop block. This time we will want to change the settings. Control should say Sensor. Change the Sensor setting to Ultrasonic Sensor and set the Distance to < 25 inches. You might want to play around with the distance setting, but 25 inches seemed good for running around my apartment.

This is the skeleton of the world's simplest behavior-based robot. Behavior-based robotics (also called reactive robotics) builds a robot from the bottom up. Professor Robert Brooks pioneered this technique at MIT in the 1980's.

Instead of worrying about complicated data processing and planning, behavior based robotics focuses on immediate reactions to the robot's environment. These robots tend to be simple but very robust. They effortlessly handle unexpected changes in their environment. And, often, surprisingly complex behavior can emerge from the interaction of otherwise simple rules.

Given the relatively limited sensors and on-board processing power, behavior based approaches seem the perfect fit for Mindstorm projects.

Here, we are building a simple rules-based architecture. Each rule has a prerequisite, if that prerequisite is met, the rule fires. In our case, we have two behaviors: avoid obstacles and explore. Since we only have two options, we can express them in a single if...else... rule. If we are closer than 25 inches, avoid the obstacle, else explore.

To actually program this, drop a move block into the top and bottom of the switch. If our robot senses an object closer than 25 inches, it will execute the top beam (indicated by the flower icon). If it is further than 25 inches, it will execute the bottom (note the mountain icon--get it? Flower for close. Mountain for far away.).

Select the top move block. Set the Power to 25, the Duration to Unlimited and slide the Steering control all the way to the right, as shown.

Select the bottom move block. Set the Duration to Unlimited. Leave all the other settings alone.

Normally, a move instruction will cause the program to block until the movement finishes. When you set the block's duration to unlimited, it starts the motors, and then continues onto the next command on the sequence beam. In our case, it will just keep looping around. On each iteration it checks the ultrasonic sensor, then executes the appropriate block. If something's in our way, we turn to the right. Otherwise, we drive straight ahead.

So far this is a very simple example. NXT-G programs can become a lot more complicated. Most blocks have a pullout drawer on the bottom-left corner. Opening this drawer exposes a series of input-output ports. You can connect blocks using data wires, allowing you to modify the block's settings at runtime. While the exact details are beyond the scope of this article, I will include a slightly more complicated example with the online source code.

NXT-G's biggest limitation is that it quickly becomes unwieldy. I find scrolling around the desktop quite awkward, especially when the project no longer fits on a single screen. You can compress common sequences into custom blocks. This gives you some control over the size of your program, but it does not allow real functional decomposition. As far as I can tell, there's no way to pass non-global values into these custom blocks.

So, if you're like me, you'll quickly reach the limits of what NXT-G has to offer. Or, at least, what you're willing to put up with. So, let's look at the third party options.

Next Byte Codes (NBC) is an assembly-like language for the default brick firmware. It gives you fine-grain, low-level control over the brick. You can write and compile programs, and then download them to the brick, much like you would from NXT-G.

Not eXactly C (NXC) is a c-like language built on top of NBC. In many ways NXC is modeled after NQC, which was a c-like programming language for the old RIS 2.0 Mindstorm kits.

Both languages provide powerful, on-Brick programming capabilities that go well beyond NXT-G. Personally, I prefer the high-level, NXC approach, but to each their own.

Here is our sample program in NXC:

NXC sample code

The following code shows our sample program written in NXC. The robot turns to the right whenever the ultrasonic sensor detects an obstacle closer than 25 inches. Otherwise, it drives straight.

#include "NXCDefs.h"
// Very simple implementation, everything is placed in the main task.
task main(){
   short ultra;
   
   SetSensorLowspeed(S4);
   
   while(true){
      ClearScreen();
      ultra = SensorUS(S4);
      
      // range converted manually.
      // 63 cm is approximately 25 inches.
      if (ultra > 63){
         TextOut(0, LCD_LINE7, "Forward");
         NumOut(0, LCD_LINE8, ultra);
         OnFwdSync(OUT_BC, 75, 0);
      } else {
          TextOut(0, LCD_LINE7, "Turn!");
         NumOut(0, LCD_LINE8, ultra);      
         OnFwdSync(OUT_BC, 25, -100);
      }
      
      // If we try to check the ultrasonic sensor too often, 
      // it will give us incorrect values.
      Wait(20);
   }
}

You compile this using the nbc application from the command line.

nbc -O=wander.rxe wander.nxc

Now, upload wander.rxe to the NXT brick. You can either use the Lego Mindstorm's software, or try NXTBrowser. Once it's uploaded, you run the program on the brick like any other NXT-G program.

Our program is so simple, it should be easy to translate it into NBC, right? Right?

I'll leave that project for the overly ambitious. However, if you just want to see the bytecode, you can use the NBC compiler to decompile an existing *.rxe file.

nbc -x wander.rxe

You can save this file, edit it and recompile to your heart's content. You can also decompile NXT-G programs. Unfortunately, the Mindstorm software does not save compiled binaries. This leaves you with two choices. You can download the binary from the NXT brick to your computer, or you can use a third-party extension that saves NXT-G binaries to disk (http://forums.nxtasy.org/index.php?s=053735c91df88266e477eda9b28b06ca&showtopic=531&st=0&p=4477& - entry4477).

Note: while the NXT-G and NXC programs are functionally identical, the NXT-G binary is 5 to 6 times larger than the NCX version. Yet another reason to use third-party tools.

Version    Lines    Bytes
NXT-G      1448     27185
NXC         222      4793

Now we move to the realm of the strange. The ruby-nxt library lets you program your robot using Ruby. However, these programs run on your computer, not on the brick. Ruby-nxt sends commands to the brick using the Bluetooth connection. The brick simply executes these commands.

If you've read my earlier articles, you know I'm a big Ruby fanboy. So, I was quite excited when I found this project.

This telerobotic approach has several advantages. The robot's program has full-access to your computer. This gives you a lot more memory and processing power. You can even build a GUI for the program using RubyCocoa (http://rubycocoa.sourceforge.net/doc/).

There are, however, a few downsides. When your robot wanders too far away, you completely lose control. Also, killing the program on your computer often leaves the robot executing it's last command, which tends to send it spinning over the horizon. Finally, you might notice a slight delay between the moment your computer issues a command and the moment your robot begins executing it. This can become a serious issue when trying to perform precise maneuvers.

Also, like many of the third party tools, this project still has several rough edges--mostly in the documentation, or relative lack thereof. The installation comes with a few sample programs, and a reasonably documented API. But, sometimes it's hard to see how all the pieces fit together, especially since ruby-nxt provides multiple options for doing the same thing. For our simple sample, I will use the commands API, a high-level abstraction which largely mimics NXT-G blocks

Here's a ruby-nxt version of our wandering robot:

ruby-nxt sample code

This is our sample program written in Ruby using the ruby-nxt library. Just like the other samples, it turns to the right whenever the ultrasonic sensor detects an obstacle closer than 25 inches. Otherwise it drives straight.

require 'rubygems'
require 'nxt_comm'
def do_sonar
  sensor = Commands::UltrasonicSensor.new(@nxt)
  sensor.mode = :inches
  
  return sensor.distance
end
def forward
  b = Commands::Motor.new(@nxt)
  b.port = :b
  b.power = 75
  b.direction = :forward
  b.duration = :unlimited
  
  c = Commands::Motor.new(@nxt)
  c.port = :c
  c.power = 75
  c.direction = :forward
  c.duration = :unlimited
  
  b.start
  c.start
end
def turn
  b = Commands::Motor.new(@nxt)
  b.port = :b
  b.power = 25
  b.direction = :forward
  b.duration = :unlimited
  
  c = Commands::Motor.new(@nxt)
  c.port = :c
  c.power = 25
  c.direction = :backward
  c.duration = :unlimited
  
  b.start
  c.start
end
def shutdown
  b = Commands::Motor.new(@nxt)
  b.port = :b
    
  c = Commands::Motor.new(@nxt)
  c.port = :c
  
  b.stop
  c.stop
end
@nxt = NXTComm.new('/dev/tty.RikiBot-DevB-1')
@running = true
runloop = Thread.new {
  while @running do
    distance = do_sonar
    if (distance < 25) 
      turn  
    else
      forward
    end
  end
  
  shutdown
}
puts "Press the return key to stop."
gets
@running = false
runloop.join
@nxt.close

This code assumes that you loaded ruby-nxt using rubygems. If you loaded it manually, you may need to edit the require lines.

You can run this from the command line as shown below

ruby wander.rb

There are a couple of important points here. I could not use the synchronized move commands. Whenever the robot encountered an obstacle, it would turn away as expected. But, when it started to drive forward again, the synchronization always kicked in, and the robot snapped back around to face the blockade.

Instead, I gave each motor a separate command. Even at the best of times, the synchronized motors don't drive in a truly straight line. So, with the motors running separately, expect to see distinct list to one side.

Additionally, the Bluetooth delay sometimes causes trouble. Still, I recommend ruby-nxt (and it's soul-sister, iCommand) for any telerobotic project (that's just a fancy way of saying computer-based remote controls). I'll include a ruby-based remote in the sample code.

Most of this code was developed using old-fashioned trial and error while staring at the API. I tested several other approaches, but this was the only one that worked satisfactorily. Again, as the project matures, things might even out a bit.

Similar in spirit to ruby-nxt, iCommand allows you to build and run Java programs on your computer. These programs also send low-level commands to the NXT brick over a Bluetooth connection.

As I am writing, iCommand only communicates with the original NXT firmware. However, future versions should also communicate with NXJ (see below).

Currently, iCommand excels at computer-to-brick communication. Telerobotic programming seems overly difficult at this point. A lot of the API does not work as advertised. I spent a lot of time massaging this simple sample into an acceptable shape. And, as you can see, the resulting source code is a bit on the verbose side.

Still, iCommand offers an intriguing set of very-high-level robotic commands. These could greatly simplify telerobotic programs, once the library matures.

Here's our sample program in Java:

iCommand sample code

This shows our sample program written in Java using iCommand 0.5. Just like the other samples, it turns to the right whenever the ultrasonic sensor detects an obstacle closer than 25 inches. Otherwise it drives straight.

import icommand.platform.nxt.*;
import icommand.nxtcomm.*;
import java.io.*;
public class Wander {
   
   private boolean running;
   
   public Wander(){
      
      System.out.println("Wandering...");
      
      running = true;
      NXTCommand.setVerify(true);
      
      Thread runloop = new Thread(){
         public void run(){
            
            Ultrasonic sensor = new Ultrasonic(Sensor.S4);
            int distance;
            
            while(isRunning()){
               distance = sensor.getDistance();
               // Manual conversion of distances
               // 63 cm is approximately 25 cm.
               if (distance < 63) {
                  Motor.B.setSpeed(25);
                  Motor.C.setSpeed(25);
                  Motor.B.forward();
                  Motor.C.backward();
               }
               else {
                  Motor.B.setSpeed(75);
                  Motor.C.setSpeed(75);
                  Motor.B.forward();
                  Motor.C.forward();
               }
            }
         }
      }; // end thread
      
      runloop.start();
      System.out.println("\nPress the return key to quit.");
      
      try{
         
         BufferedReader br = 
            new BufferedReader(
               new InputStreamReader(System.in));
         
         br.readLine();
         
      } catch(Exception e){ 
         e.printStackTrace();
      }
      
      running = false;
      
      try{
         runloop.join();   
      } catch(Exception e){
         e.printStackTrace();
      }
      
      stopRun();
   }
   
   public boolean isRunning(){
      return running;
   }
   
   public void stopRun(){
      System.out.println("Shutting Down!");
      Motor.B.stop();
      Motor.C.stop();
      NXTCommand.close();
   }
   
   public static void main(String[] args){
      
      new Wander();
   }
}

You compile iCommand programs just like a normal Java:

javac Wander.java

Then run it:

java Wander

Both iCommand and ruby-nxt make excellent prototype languages. Trust me, it's a lot easier to see data printed out on your computer screen, than try to spot messages on the NXT while it zips away at top speed.

Ruby-nxt and iCommand also provide the full power of a general-use programming language, with a large number of useful libraries at your fingertips. Imagine building your Robot on Rails, or crafting interesting Swing GUI for a telerobotic project. These tools let your imagination run wild. And the Bluetooth connection really opens doors--at least until your robot wanders out of range...

As far as I can tell, this is the first firmware replacement written for the NXT brick. NXJ allows you to run Java programs on the brick itself. As I am writing this, the LeJOS crew just released their initial 0.1 offering, and it's not quite ready for prime time. Currently, there's no sound, Bluetooth or ultrasonic sensor support, but like all the projects mentioned in this article, it's undergoing rapid development.

The LeJOS crew built an excellent firmware replacement for the old RIS 2.0 system, including a built-in robotics API for behavior-based development. Looking at the new API, they clearly plan to follow a similar approach here.

NXJ promises to be a powerful replacement for the default Lego firmware. I look forward to playing with it once it matures, but I'm going to wait a little while before I make that plunge.

We are always faced with tough engineering decisions--even in hobby robotics. Sometimes I need the accurate motor control and fast reactions that I get by running my code on the brick itself. Other times I need access to more memory, to greater computational abilities or a nice GUI interface. The great thing is, with a Bluetooth wireless connection, you can have your cake and eat it too.

Part of the program can run on the brick. It can gather data, and then send that data back to your computer via the Bluetooth connection. Your computer then churns through this data, doing all the heavy computations. Once done, the computer sends new commands back to the brick, perhaps setting new goals or otherwise changing the robot's behavior.

This is where the real power lies. And I suspect, as the third-party tools mature, splitting a program between the computer and the brick will become easier and easier.

Well, that's the end of our lightning tour of the Mindstorm NXT. I was pleasantly surprised with the breadth of third-party tools currently available. Of course, our PC counterparts have a few additional choices, but not as many as you might guess.

All of these projects are still young and a bit rough around the edges. NXC seems to provide the solidest implementation, but the documentation remains rather light. Still, I highly recommend NXC as an NXT-G replacement.

Compared with the old Mindstorm kits, the NXT offers a much wider range of features and capabilities. There's more than enough to keep me busy for quite some time. And, as third-party tools and hardware mature, the possibilities will continue to grow.

I can hardly wait.

Rich Warren lives in Honolulu, Hawaii with his wife Mika and daughter Haruko. He is a freelance writer, full-time software engineer and part-time Graduate student at the University of Hawaii in Manoa. When not playing on the beach with his daughter, he can be found writing, studying, or doing research--all on his MacBook Pro. You can reach Rich at rikiwarren@mac.com.