Phase I:: Basic IR Tracking

At first we had to collect all the information about the wii-mote device and its capabilities, before implementing the IR Tracking module. After that we began to search for ready made libraries that deals with the wii-mote device.

1- Background Information:

Wiibrew is wonderful source of information on this device, to know its details.






But for this project we only needed certain information which are,


- Connecting to wii:
In order to interface with the wiimote, it must first be placed in discoverable mode by either pressing the 1 and 2 buttons at the same time, or by pressing the red sync button under the battery cover. Once in this mode, the Wii Remote can be queried by the Bluetooth HID driver on the host.

- Wii Reports:
This block includes an enumeration of reports that the device understands. A report can be thought of similar to a network port assigned to a particular service. An "Input" report is sent by the Wii Remote to the host. An "Output" report is sent by the host to the Wii Remote. The Wii Remote has a number of different data reporting modes. Each of these modes combines certain Core data features with data from external peripherals, and sends it to the host through one of the report IDs.

- Wii IR Camera:
We use the IR Camera inside, which has an object tracking processor (IR Blob Tracking).
It's capable of tracking up to 4 moving objects, and these data are the only data available to the host. The IR camera has an effective field of view is about 33 degrees horizontally and 23 degrees vertically (as measured on one unit).

2- Implementation Details:

We used Brian Peek's WiimoteLib ,
and its wrapper  BobCat Library

and inside the Bobcat wrapper we used the first layer ( Code Layers )
because it had 2 main features that we needed
a) it performs smoothing
b) it keeps track of points to keep point indexes consistent as they move.

First Phase

as we started to work, we made a plan to which we shall stick for the the rest of our work...
we divided our project into modules which is much easier for implementation and maintenance.
And we focused in first phase on:
1- Basic IR Tracking
2- Image Warping
3- Automatic Calibration

We'll discuss each one of those modules in other articles, 

but for now we'll discuss some of the problems that we faced during implementation

 
Problems faced:

The projected image wasn't correctly aligned with size of the of the display ...so figured that we should manually calibrate the wii to projector before projecting.

another problem is that sometimes the image freezes when moving it , this's because wii loses track of some of the Ir Leds ....so we need to track the lost Leds.

Also the image sometimes turned upside down this was because of point identification, which means identifying the LED as the top left, top right, ...
and the solution was either use motion modeling ,data grid, or optical flow...
we thought we shall start with the data grid solution and see the results.

Related Work

We were mainly inspired by Johnny Lee's Thesis 2008, projector based Tracking
http://johnnylee.net/academic/p104-lee.pdf

Those are some videos for his work





And this is Johnny Lee's Blog, for more information about him
http://procrastineering.blogspot.com/2008_06_01_archive.html


And of course we were really inspired by Sixth Sense next Gen by Pranav Mistry that was posted on Nov 2009



We actually had those ideas on mind even before watching the video and after watching it we were really inspired to move on with our project, as he had everything that we wanted to do except one thing which was to support multiple projected display, instead of working on only one display

Also another project that we felt is similar to our work especially in the part of projecting dual image,
is Microsoft Second Light, designated to launch in 2011

This is a video about it

Wii-Display, Idea "Technical Point of View"

Tools:

1- Projector of any kind




2- Wii-mote (mounted over projector)

3- IR LEDs (they will be mounted on the frame)

 







4- IR pen

Software: 

- Our System Architecture is divided to 4 main modules: 

1- IR Tracking: (Based on WiimoteLib)

Operation:

- Calibration (Automatic/ Manual)

- Tracking Loss Strategies

- Point Identification

Output: 2D co-ordinates of 4 IR LEDs

2- Image Warping: (Based on OpenCV, Emgu wrapper)

Input: Co-ordinates of IR LEDs

Operation:

- Computing Homography Matrix

- Dual Image projection

Output: Warped image to be projected

3- Orientation Sensitivity:

Input: Co-ordinates of 4 IR LEDs

Operation:

- Computing Orientation direction (angle)

- Handling Orientation Change event

Output: Desired action according to designed purpose.

4- Controlling mouse: (Based on Win API functions)

Input: Co-ordinate of IR pen.

Output: mouse actions

Wii - Display "the big idea"

as we were searching for an idea for our GP , we wanted to do something new and useful .
when we (actually "menna siam" did) hit johnny lee's idea "projector based location discovery and tracking " we were amazed by it, so we decided to go for it.

The main idea of the wii - Display:
there are some obstacles concerning the usual display:
1- The display on a typical device whether a mobile or a monitor remains fixed in size and rigid thus forcing constraints on the interaction with it.
2- those devices are composed of highly sophisticated hardware.

so, the solution is by combining Motion Tracking  +  Image Projection,

                                   










We'll construct a simple frame with IR LEDs attached to its corners.
This frame will act as a movable projected display.

Our Project aims to Creating a flexible display using any kind of surfaces,Creating a low cost display with the simplest hardware.

as we were thinking about applications for this project , we thought that it can be used as :
1- Wii-Book: where it can be used inside classrooms as an interactive book
2- Map Viewer: Use the display to project different information about the map behind the display, like projecting the zooming of the part of the map behind the display.

3- eBook reader : inside libraries.
4- Tablet PC : The display can be used as Low cost tablet PC by using interactivity feature.
and so many others....
so , it's the time for work and trials.

Our GP Journey

Our journey to this GP was really weird. At first, people were saying this is a very difficult project you'll never be able to do this stuff. Then, ironically we found it's not impossible at all, to the extent that some people said that it's so easy. Actually there were others who told us that it's not useful.

But, then we decided we're all gonna pray este5ara and see what happens. We entered to Dr Haytham and told him about our project, and he was so enthusiastic and he liked it and told us to go on with it. And yeah that was a relief, but yet we had another problem which application to use for the project.

Which application to use that would really make the project worth the effort. We've searched & thought a lot. “Application”, we all really felt that this word was haunting us. Whenever we go people say wow what a great project, but what's its application anyway? Yeah those words really irritated us, but they were right. And for some time we felt skeptic about the project whether to continue with it or change it totally. But, we didn't because it was the one idea that we all loved, and we all wanted to participate in it for different reasons. And we took the decision to continue with it and squeeze our minds for any application. What we've settled upon at last, was 2 applications actually, one was the GIS viewer and the other was the wii-Book, they'll be explained later in another post.

At the end, we selected the project each with her own reasons. Mohga loved the idea of learning new stuff. Menna Hisham was fascinated with the wii-mote device and she liked that the idea had some hardware that we had to built with our own hands. As for Menna siam really wanted to do sth that's new and useful especially for the teaching field, she always had that dream of changing the teaching system to the better cause it's the one thing that can change the whole nation

Our Team

We're 3 senior students at the faculty of computer and information sciences Ainshams university, computer science department. The 3 of us share the passion & love of programming and learning new stuff.

Who we are:

Menna Hisham:is the top speed coder we've ever seen msa.

Very very committed to her work and very punctual if you run late on her you're dead meat for sure :D.

Also she's a tech maniac she loves any hardware device that's related even remotely to technology, of course she has the number one device that all people should get an ipod touch.

Mohga Nabil: is a real pro, we've always pictured her as CEO of Microsoft Redmond:D. she Loves to learn new stuff each day, the most hardworking girl we've seen. And she's an artist, she doesn't just write some chunk of code and that's it, she always put a solid design before coding, honestly when we read her code we feel that it's a master piece :D msa.

Menna Siam:is our GEEk :D. all our friends and her family know that about her.

she first fell in love with coding when she was at school, and her love for it grows year by year. she wants to be a computer scientist not just coder. she loves to feel that she's a mathematician, understanding all this hard math & probability stuff and actually using them in programming just fascinates her.

Our Technical Skills:

Menna Hisham: she's a very talented .Net developer. And She's skilled in searching about any kind of topic and has a relatively good knowledge about wii-mote which will help us in the project.

Mohga Nabil: She is maniac to algorithms and coding. has good knowledge about web application development, especially using PHP and MySql. She had an internship program at Specialized Solutions (2009).

Menna siam : She is a very talented reseracher/scientist . Has a good knowledge about computer graphics .She had an internship program at Nile University as a research assistant (2009).

Common Programming Languages: C++, C#, Prolog, Assembly

Common Courses: Algorithms, Data Structures, Object-Oriented Programming, Design Patterns, Computer Graphics, Computer Architecture, System Analysis and Design, Windows Programming, Artificial Intelligence, and Simulation and Modeling.