Saturday 7 February 2009

...and now for the annoying bit - Accelerometers

After hours of testing and gazing hypnotically at a spreadsheet application, one hand on the mouse, the other clutching my Wii Remote, I've come to the conclusion that it's not possible to measure displacement using data from the Wii Remote's accelerometers.

I have searched through forums associated with non-gaming-exclusive applications of the Wiimote and found that the general opinion of members is that it's not possible (http://www.wiili.org/forum/tracking-distance-and-3d-position-without-a-sensor-bar-t3763.html). Another search under general applications of accelerometers (not associated in any way with the Wiimote) revealed algorithms that take accelerometer data and several other determinable variables to calculate displacement.

Technically I 'should' be able to swipe the Wiimote 1 metre to the right and collect the data output from the accelerometers. Then I could integrate these readings to retrieve the velocity at a given time, and therefore the displacement (or distance). The final result should have some resemblance to 1 metre.

I performed the same gesture a number of times, however the displacement calculations returned wildly different results. A slight deviation in the results could be acceptable however when moving the Wiimote over the distance of 30 centimetres I returned results between 4 and 40 centimetres.

At this point, it should be noted that the Wiimote doesn't actually output values that can be interpreted as acceleration. The values it returns are actually measured in g's (g-force) which is the amount of acceleration in relation to the Earth's gravity. 1g is roughly equivalent of 9.8m/s/s or 9.8ms-2 so the data from the Wiimote can be converted to real acceleration data. (http://en.wikipedia.org/wiki/G-force)

In the experiments I conducted using the Wiimote, the accelerometer data was captured at anything from 100 times per second to as little as 10 times per second. Therefore, 9.8ms-2 has to be divided by the number of samples per second.

All the calculations are perfectly sound on paper, however the source of the data – the accelerometer – isn't reliable enough to produce consistent, predictable results.

So, why not?

Well, first, I had a look at the specification of the Wiimote's accelerometer – an Analog Devices ADXL330 (http://www.analog.com/en/mems-and-sensors/imems-accelerometers/adxl330/products/product.html). The g-force readings of the accelerometer are accurate to +/-3g. If the Wiimote is shaken hard or swung sufficiently fast then readings outside +/-3g can easily be produced. Any readings outside the specification of accelerometer should be disregarded, however, in some cases, this would mean completely disregarding an entire set of data!

Next, I found that the Wiimote can be held at a certain orientation and provided the user has a steady enough hand, they can move the Wiimote through the air without altering the output of any of the axis on the accelerometers.

Before writing off the Wiimote's capabilities to measure displacement using its accelerometer data, it should be noted that a programmer and car-enthusiast (http://forums.vwvortex.com/zerothread?id=3362586) successfully strapped his Wiimote to his car and managed to measure his 0-60mph time, and therefore distance traveled. It should be noted that perhaps periods of smooth acceleration can be accurately recorded with the Wiimote. This is because the g-force readings from the accelerometer would be constantly increasing (or decreasing). If one was to plot the readings on a graph of acceleration against time, they would see a smooth line. If one was to plot the readings of a rapid hand gesture on a similar graph, the line would resemble more that of a 'heart-rate monitor'. It's obvious that the former readings would be more accurate.

The problems I have experienced don't appear to just be the preserve of programmers using the Wiimote for non Nintendo Wii software.

It also appears that more observant gamers have noticed the shortcomings of the Wiimote as well. The 'real-life' movement that was promised by Nintendo with the release of the Wii console doesn't match the now-apparent limitations of the system.

(http://www.gamasutra.com/view/feature/1494/wheres_the_wiimote_using_kalman_.php)

Using the Tennis game in Wii Sports as an example, more specifically the way the player serves the ball to the opponent, one can see that there are many ways of playing the game. The method of serving in Wii Sports Tennis is to hold the Wiimote down by your side, raise it up quickly and bring it down again. A player who may have played tennis before or at least knows what a tennis serve looks like may try to serve the ball using a real-life serving technique. From personal experience, I've seen adults try to serve by throwing a non-existent ball into the air, and then try to hit it using the Wiimote. Annoyingly for them, using a real-life technique in Wii Sports Tennis doesn't actually work. The serve must be carried out using the rather childlike method described above.

Other gestures like those required in the bowling game in Wii Sports appear to be more realistic in terms of the motion the player makes with the Wiimote. However, this is purely due to the fact that the real-life bowling action is within the capabilities of the Wiimote's accelerometers. Compare this to where the action had to be altered by the game's developers for the tennis game to a gesture that the Wiimote can understand.

In terms of the gaming aspect of the Wiimote's technical ability, it seems suitable to simulate some types of motion that occur on very definite axis lines – i.e. in straight lines rather than a diagonal swing across all axes. For other, more complex gestures, there is a certain amount of difference between the motion that the game's developers tell the player to make (on-screen instructions) and the action motion that the game picks up (the code running the game). For example, a game may tell the player to perform an elaborate swishing gesture with the character's 'sword', but in fact, the game's code doesn't care how elegant or 'swishy' the gesture is. It will only concern itself with a very small portion of the movement to produce a distinct gesture.

Possibly the most irritating aspect of the Wiimote for consumers is that it isn't the all-singing all-dancing control device that Nintendo promised. All the people who go and buy the various sword fighting games out there (Red Steel etc) and games from the Star Wars franchise must have a sour taste in their mouths when they realise that the Wiimote isn't quite the sword or Lightsaber they first thought. This is due, once more, to the Wiimote's inability to sense it's position in space – it's not possible using accelerometers.

Nintendo are shortly to release a device called Wii Motion Plus that bolts onto the Wiimote (where the Nunchuck is usually connected) that contains a gyroscope. This device will enable game developers much more scope when programming the physical aspect of their games.

"Nintendo's upcoming Wii MotionPlus accessory for the revolutionary Wii Remote controller again redefines game control, by more quickly and accurately reflecting motions in a 3-D space."

Hopefully, this will see less use of the IR Sensor Bar to detect distance from the TV and more use of the Wiimote on its own.

While the purpose of this article wasn't to moan able the shortcomings and downfalls of the Wiimote, I feel that some of the text is necessary due to the fact that I myself came to the Wiimote thinking that so much more was possible! However, it's not all bad – there are ways around most of the problems I'm encountering thus far.

Now, for the purpose of this investigation, I shall suspend the research into calculating displacement with the Wiimote. The main reason for this is that it doesn't appear to be possible to obtain accurate readings from the types of gestures I wish to record and recognise.

2 comments:

Asit Jain said...

We too learned it the hard way 4 years back when working on a similar project. The drift/sensitivity/noise/bias will not let the readings come anywhere closer to what is expected.

But its not impossible. Having Gyros will help a lot, and additional sensors like magnetometer will increase the precision. You need to use something like a Kalman filter to fuse the data. Some additional constrains like object will come to a rest will help.

BTW, check out http://www.freescale.com/files/sensors/doc/app_note/AN3397.pdf

Michael said...

Hi Dab, thanks for the pointers there, especially the link to the article, it's very interesting!