While working long days on GeoSport, a second year teamwork project in autumn and winter 2007, some third year students at the department of Computer Science were equally busy with their teamwork project (more about that in Dutch on the blog of my first year programming professor). It involved a Lego Mindstorms robot that had to be capable to recognise and find its way to a certain spot when it was put in a room with some obstacles in it.

When eventually not choosing computer science (in favor of mechanics as my minor – my major is electrical engineering), I thought I should try to do a project like this on my own, to improve on my programming skills, to finally do something useful with my Mindstorms set and for the challenge of doing it, of course. I changed the goals a bit, and I set out to make a robot that can autonomously map a room, using the Mindstorms ultrasonic sensor. I named it RoboDaGama. I also decided to do the entire project in English, to practise that as well.

1. Robot design

The basic requirements of my robot were:

• A stable base, for this I put the NXT on its back in the middle of the robot. I chose two steerable wheels and two small wheels at the back. This made turning fairly simple and the two wheels don’t greatly influence the movement I want.
• I wanted the ultrasonic sensor to be in the middle, above the NXT. I also wanted it to be able to turn independent of the robot as a whole (for scanning accuracy), which meant I had to use a third motor. This also makes the robot look beter and makes it easy to deal with the scan results. But it also meant I had to make quite a complicated gear structure to be able to move the camera around.
• A bumper for the touch sensor, so accidental brushing against the touch sensor won’t set the trigger. Making a bumping element that touched the sensor when wanted turned out to be quite hard, but it eventually worked, and it performs admirably.
• Easy to remove the NXT (the smart part of Mindstorms, the backbone of the entire robot) from the robot. This involved not actually fixing the NXT into the robot, but laying it into a kind of bed. This was no problem. The NXT doesn’t move but can easily be taken away after releasing the connections to the sensors.
• Easy to replace the batteries, without having to remove the sensor connections. At first, I was planning to remove the NXT to replace the batteries, but it turned out I could actually just turn the robot around, remove one bar and there they were.
• As the ultrasonic sensor is situated above the NXT, there had to be enough space to get to the buttons and read the screen. Screen readability turns out to be a bit of a problem sometimes, but reaching the buttons isn’t.
• Of course, the robot had to be built with the Mindstorms basic set, plus some old Lego Technic parts.

I made the robot during pauses from learning for my January exams. It was really creatively rewarding and addictive, mostly searching for ways to make my ultrasonic sensor turn or to construct the bumper. I admit, most of it was actually done in one (long) evening and the following day, just after New Year’s Day. But I guess it was better for my studying to finish it at once (and my rates turned out to be good, so no big deal).

This is what my robot looks like.

2. Software

2.1 Looking into it

The next step was to check out all available software programs to program my robot to do what I wanted. As that was to produce and display a map on the computer, a computer interface was needed too. After surfing around and testing some things, I decided to use the iCommand library to connect to the NXT over Bluetooth, and to control the robot entirely via the computer. The robot has no autonomy of itself (without the computer). This made the whole project a lot easier. Plus, I could program everything using Java and Eclipse, things I already knew the basics about. I also found out I would need to make ‘virtual robot’, which didn’t require a connection to the actual NXT for testing.

2.2 First test programs

I started with an iCommand example program, with which you could control the robot with your keyboard. My first steps were then to make a small interface, and to implement the ultrasonic and touch sensors. After that, I extended it with a Java AWT canvas for displaying the scanning results. It took quite a while before everything worked as it had to, mainly because I had no experience with the canvas.

Then, I tested the accuracy of the ultrasonic sensor (which was better than expected for straight surfaces, but bad for round or irregular shapes) and the performance of the buffer and touch sensor. And I don’t think our cat has forgotten my little robot yet (they met while I was testing in the living room).

2.3 Basic feature implementation

After the basics worked fine, I began to implement and improve a list of basic features:

• xml import and export, which later became loading and saving
• map screenshot exporting
• visualizing the track of the robot and its position, moving forwards and backwards
• turning left and right and displaying the rotation of the robot, this was somewhat trickier, but with some basics about rotation matrices I got it done. I used the Java Matrix Package JAMA for this.
• the virtual robot mentioned above. For scanning it initially used randomly generated scan results, but it was hard to get realistic results, so I made a second version, in which you have to provide an xml file of the coordinates of the room the virtual robot is in. The scanning now came down to calculating a what distance the robot was from the various walls of the virtual room. I later saw this really gave me taste of what would come later, as the program is all about room mapping of course.

2.4 A usable program

By now, my program was more like a collection of various buttons to do all sorts of things. I spent a lot of time at this point to redesign the underlying structure as well as the user interface:

• adding of, amongst others, Room and Track classes
• complete UI redesign, using Java Swing, and ZoneLayout, an easy to learn and use layout manager
• mouse control of moving and scanning (that was only keyboard-based up until now)
• each room has a name and you can rename it
• choose previous tracks from room, so you can continue with a track after quitting the program
• zooming and panning of the canvas, and center on robot. This cost me a lot of time, but it really was a feature that I felt was needed to have a good program.

2.5 Left pieces and difficult parts

At this point, the basic design program was basically finished. Only the last, but certainly not least, pieces of the puzzle had to be fit in.

• April 1st, 2008: With RoboDaGama version 0.5, I just finished the wall recognising algorithm, which calculates where the walls are from the scanned points. This was certainly one of the most difficult algorithms I have had to make, and I spent quite a lot of time on this. The first method I tried used least squares solutions to find the best walls, but that turned out not to be very accurate. I then programmed an in se simpler method, and after a lot of tweaking and testing it worked.

To do:

• the artificial intelligence algorithm of the robot
• keeping track of the ultrasonic sensor position, and situation aware scanning
• a user interface for calibrating the robot and the program
• testing of interface in every possible situation (including abuse cases)
  1. Download the last version of the software (RoboDaGama-0.5.zip, 280 kb). External libraries and example rooms are included.
  2. Execute the RoboDaGama-0.x.jar file. To do this, you need to have a Java Runtime Environment installed, if this is the case just double clicking the jar file should do (if it doesn’t, a Google search on ‘execute jar file’ or something like that should provide solutions).
  3. Open a room (for example the included bigroom1.xml) and click ‘New track’ (this is not really necessary, the program just makes a new one anyway when you connect without have selected a previous track).
  4. Connect to ‘Virtual robot’, trying to Connect to NXT probably won’t work…
  5. You can then try the manual control: click ‘Manual control’ and try moving, rotating, scanning, zooming, centering on robot, saving your track…
  • make my own robot
  • do part of the ‘thinking’ on robot itself
  • buy a Mindsensors NXTCam and use it to create a 360 degrees model of the room, complete with Image Recognition software

3. User Guide

If you want to try the virtual robot for yourself (for the real one, you’ll need Mindstorms), you follow these steps (screenshots are coming someday). This should work on Mac OS X (the OS which I designed it on), but also on Windows (tested on XP) and Linux versions. Please let me know if you’ve successfully made it work on your platform.

4. CAD

I tried to make a CAD model of the robot I made. I got started with Lego Digital Designer, but I couldn’t model some things from the robot so I didn’t continue.

5. Future possibilities

After all of the things in 2.5 are finished, I’ll consider RoboDaGama as done. That doesn’t mean there are no possibilities to extend this, of course. Only a few that I’ve already thought about:

If you have any suggestions or questions, please don’t hesitate to send me an email.