LEDs, Camera, Tilt!

Whilst all the tilting is happening with the 3d model on the computer monitor, the Wiimote is also providing feedback to the user. Every one of the 9 positions (8 tilts and 1 'idle' position) has a different light sequence. I'm particularly happy with the way the LEDs zoom left and right when the Wiimote is held in 'idle' position - slightly reminiscent of a certain car...

Have a look at the video (apologies for the shoddy focus of my camera!)
There is a high(er!) quality version of the video here.

The picture below shows how the LEDs will behave when the Wiimote is tilted in different directions - hopefully clearer to understand than the video!
(note: the glowing circles denote LEDs that flash)

The LED sequences are merely there, at the moment, to provide the user with extra feedback as to their positioning of the Wiimote.

(it also looks rather good as well!)

Wii Gesture Project - version 2

...one step further to having some gesture recognition!

Have a look at the video.

(Once again, if the video quality is too iffy for you, go here)

You will see that the application records the user's gestures in groups of 8. The movements, each direction represented by a number, are stored in an array (or 'list' as Lingo would have us call them!). At the end of every group of 8, the array is condensed into a single string. This forms the basis of a Levenshtein Distance comparison string - I'll explain this in another post, but it's a core part of the application nevertheless!)

For example, the application may output:
movementList = [1, 2, 1, 2, 3, 5, 1, 5]

...therefore...

movementListLev = "12123515"

We take each number...
1=up, 2=up-right, 3=right, 4=down-right, 5=down, 6=down-left, 7=left & 8=up-left

So we know from 'movementListLev' that the user has moved the Wii Remote up, up-right, up, up-right, right, down, up & down.

There is a problem with this method of data collection and that is that a complete gesture must fall perfectly within each group of 8 movements. The next stage in the development will involve creating the array/list so that newest movement numbers push out the oldest numbers, that way the 8 most recent movements are ALWAYS shown...

Wii Gesture Project - version 1

Here is the first iteration of an application that will hopefully be able to recognise a gesture made with a Nintendo Wii Remote. So far, the program can mimic the rotational movement of the Wiimote and therefore, can determine which direction the Wiimote is pointing.

Watch the video above (if you haven't already!) to see how it works.

(If you don't like the Blogspot video quality, have a look at a higher resolution version here)

What you'll probably notice is that the movement is quite stuttery, so even when the Wiimote is in a stationary position there may still be some jitters. This is due to the fact that the 3D model of the Wiimote takes it's rotation properties directly from the ever-changing data of the real Wiimote's accelerometers. Perhaps the next thing to do is add a method that 'rounds' the accelerometer data to a certain (lesser) degree of accuracy - although less accurate just for the visual appearance as opposed to the inner workings of the application.

The work continues...

Movement purely on accelerometers!

Finally back on track after months of mucking about...!

Today, I altered my Wiimote program in Director to make a 3D object move around based on the data gathered from the Wii Remote's accelerometers. The control is actually pretty accurate. So far, you have to hold the remote in a fairly 'wooden' fashion. By this I mean that you can't swish the remote left or right as one might if playing a real Wii game (Wii Sports Tennis comes to mind here).

So, after some faffing with the Lingo in Director I have the following degree of control over my 3D ball...

note: the 'deadzone' for movement is attained when you hold the remote straight up in the air.

Move left: swing the remote to the left
Move right: swing the remote to the right
Move down: pitch the remote forwards (the end of the remote is moved in an arc away from your body)
Move up: pitch the remote backwards (the end of the remote is moved towards your body, the amount of movement is limited for this action)

At the moment, the movement of the ball isn't directly related to the accelerometer data in terms of the actual numbers... So if the accelerometer hints that the Wii Remote is being swung to the left then the ball is moved left by a finite amount.

That's all for the moment, however everything is well on the way to more exciting things!

How the axes work on the Wii Remote

If you move the Wii Remote along each of these axes then the values from the accelerometer along each axis will vary.

So moving the remote rapidly from side to side will cause change in the 'x' axis.

The sensitivity of the accelerometer in the remote (and indeed the Wii Nunchuck - they're the same spec) generally means that it's very difficult move along one axis at a time.

Now it's time to work out how to get this information recorded in Director!

Squeezii Schmiizii Diizii "Wiigee"

Ohhhhh, yes, the titles can and shall get worse than that...

In my quest for all things interactive and Wii, I stumbled apon this rather nice gesture-recognition library based in Java called Wiigee. It's being developed by several German university students and from the demo video it looks amazing.

I compiled all the code using Eclipse but, unfortunately, my Bluetooth stack ('Blue Soleil') and Bluetooth dongle aren't compatible with the requirements of the Java Bluetooth library, 'Bluecove' which requires 'L2CAP'.

To save the waffle, check out the following video on the Wiigee site:

http://www.wiigee.org/demovideo/demovideo.html

Links

Wiigee site: http://www.wiigee.org/index.html
L2CAP definition: http://en.wikipedia.org/wiki/Bluetooth_protocols#L2CAP_.28Logical_Link_Control_and_Adaptation_Protocol.29
Bluecove site: http://code.google.com/p/bluecove/
Eclipse site: http://www.eclipse.org/
Blue Soleil site: http://www.bluesoleil.com/

Using the Wii Remote with Director

Proof Of Concept time folks!

Using an example from my module for 3D graphics using Director and the example code for the Wii Remote Director Xtra, I've managed to demonstrate how it is indeed possible for the Wii Remote to change variables in Macromedia Director MX (note: the Xtra only seems to work for MX, those with the new Adobe Director probably won't get anything working, tough :P).

Obviously, this isn't exactly the first time anyone's done this before (yeh, obviously!) but this stage of my research moreorless defines the direction in which my project is likely to take - for the programming portion of it at least.

The screenshot below shows what I came up with... It's the quickest and dirtiest method, but I just needed to see for myself that it's possible to manipulate 3D graphics, albeit on a basic level, using the Wii Remote.

I set the program up to make the cube rotate about the x-axis and the sphere rotate about the y-axis. In terms of the Wii Remote this means that pointing it up or down will make the cube move and tilting it from side to side will make the sphere spin from side to side.

The program is very responsive with no delay between making the movement with the remote and seeing the result of the movement reflected on the screen.

When I placed the remote on a flat surface and flicked it to make it rock, even these small rocking movements were registered. All in all, this test was a complete success! The next test will be to import a terrain/model from 3dsMAX that's slightly more complicated, just to see if the smoothness of this test can be maintained.

One thing to note for all this - this is just taking the numbers from the Wii Remote Xtra and using them without performing any calculations. I also didn't have much regard for HOW the 3D models were moved/rotated, I just used the first method that came to hand, therefore the test doesn't respond as expected sometimes...!

The next task will be to have the 3D models moving in exact relation to the way the Wii remote is pitched or rolled. i.e. the changes aren't relative, they should be absolute! Another task will be to officially begin my research into specialised user-interfaces and methods of control. This will all hark back to my original ideas that involved the use of webcams - who knows, this method of control may crop up again in my project!