Face detection with Nokia N900, using ROS and OpenCV

This post is about a small recipe to perform face detection using Nokia N900 phone. It's based on ROS and OpenCV and shows how these components are mixed together and configure. See this OpenCV wiki page about face detection to understand how it works behind the scene.

First of all, ROS needs to be installed on N900. I've built several ROS packages, including latest offical release, code name "C turtle". I assume you know how to ssh to your N900 and gain root access.

Disclaimer

Careful while tinkering with your N900, you may "brick" it (if you don't know what I mean, close this page). And I wouldn't be responsible if this would occur, or if anything would turn into something bad. You've been warned.

Setting up ros-n900 source

I've created a Google Code project dedicated to N900 ports and package developments. This project can be reached at http://code.google.com/p/ros-n900/. In download section, you'll find deb packages. You directly download them to your N900, or configure a new APT source pointing to this project:

$ echo "deb http://ros-n900.googlecode.com/files /" >> /etc/apt/source.list

$ apt-get update

Now install ROS with apt-get:

$ apt-get install ros-cturtle-base

This will install ROS on /opt partition (usually 2GB ext3 space), leaving rootfs untouched. ROS uses ~500MB to install. You can also install it on a (fast) SD-card, formatted using ext3 filesystem (don't use FAT32). You'd then need to create a symlink /opt/ros-cturtle-base pointing to your SD-card.

Accessing N900 webcams under ROS

brown-ros-pkg project hosts gscam, a very nice ROS packages used to access camera with GStreamer. Since N900 webcams are recognized as V4L2 devices, it's easy to setup a gstreamer pipeline. First install dependencies. On N900:

$ apt-get install gstreamer-tools

This example shows how to send videos to PC host using UDP. Device /dev/video0 is back camera (big one, high resolution), /dev/video1 is the front one (small resolution).

# On N900, assuming 192.168.2.14 is PC's IP address (usually true if following usbnet howto tutorial)

$ gst-launch v4l2src device=/dev/video1 ! videoscale ! video/x-raw-yuv,width=320,height=240 ! ffmpegcolorspace ! smokeenc ! udpsink host=192.168.2.14 port=1234



# On PC computer:

$ gst-launch-0.10 udpsrc port=1234 ! smokedec ! autovideosink

Here we are! A small window appears broadcasting N900 videos. You may even see your face. Install some more dependencies to build ROS package. On N900:

$ apt-get install libgstreamer0.10-dev

Now go to ROS stacks directory.

$ roscd

$ cd ../stacks

Install gscam package. Follow instructions here, download gscam archive in download section or install it from sources.

$ svn co -r 682 http://brown-ros-pkg.googlecode.com/svn/trunk/unstable/gscam gscam

Once installed, build it using rosmake:

$ roscd gscam

$ rosmake -i

(requires quite a lot of time, be patient...)

gscam requires environment variable GSCAM_CONFIG to be set. It stores gstreamer pipeline definition. I had lots of troubles running the correct pipeline, and finally got help from ros-users list. The trick is to convert YUV format (the only N900 cams seem to output) into RGB.

$ export GSCAM_CONFIG="v4l2src device=/dev/video0 ! videoscale ! video/x-raw-yuv,width=320,height=240 ! ffmpegcolorspace ! video/x-raw-rgb ! identity name=ros ! fakesink"

You can manually check it's working without ROS:

$ gst-launch-0.10 $GSCAM_CONFIG 

If it pauses, that's ok. Now run it using gscam node. It requires to run within "bin" directory.

$ roscd gscam

$ cd bin

$ rosrun gscam

At this time, gscam should say it's "processing..." (of course, a reachable roscore should run somewhere, for instance on PC host). Now back on PC, install n900-cam package.

$ roscd

$ cd ../stacks

$ svn co http://ros-n900.googlecode.com/svn/trunk/src/n900-cam

$ roscd n900-cam

$ rosmake -i

Now run n900-cam testcam.py node. It'll retrieve images from image_topic subscription, add a circle and display the result in a window (from ROS tutorials).

$ rosrun n900-cam testcam.py

So far so good. Let's face detect ourself ! This is closed to previous example, except now images are submitted to OpenCV for face detection. Code is coming from OpenCV samples and is glued here to work with a ROS node.

$ rosrun n900-cam facedetect.py

If you can see yourself with a red square around your face, that's good news. If not, either you're not human, either something is wrong with running configuration...