Of Multitouch and HCI

A buzz word running around at the moment is multitouch. The two big giants are throwing it around like anything. Apples iPhone and iPod Touch uses a multitouch display, Microsoft Surface uses a HUGE multitouch display. But what does it all mean?

The iPhones multitouch display goes as far as you can have two points of contact on it at any one time, this is how they achieve the 'pinch' zooming function. However, Surface can take in as many points of contact as you want, true multitouch.

I'll go into how it works another day, I want to talk about possible applications today. Focusing on two major areas of possible use. Business meetings and lectures. Lets take a typical meeting, six people sat round a table all from different departments. One more person taking the minutes of the meeting. Each person has an ID card tagged with RFID which the drop onto the table infront of them. The table glows into life. Where once it looked like a black glass surface now it turns into several displays, one for each person.

On their display shows a Home area that contains all their documents, a presentation window that allows information to be displayed to everyone in the meeting in real time. A notes page that can be hand written to with automatic conversion to text. Now imagine that all this can be interacted with with multiple touches of your fingers. Everything works with everything else. So you can drag the current showing slide from the presentation window straight to your notes. Or drag a keyword to a search box that automatically Google's it for you.

To head towards a perspective display, under all these main windows is a virtual, top down view of the table. Around its edge it shows who is at the meeting, exactly where they are sat in real life. Want to share a file with the entire table? Pick up the file from your area, tap the table to bring it to the top and drag the file to the middle of the table, let go and it automatically appears on everyones screen. Only want it to one person? Just simple drag it to the person you want to see it and it will appear on their screen.

Looking at another use, lectures. You don't have to worry about that the lecture is in America and you're in England. The lectures being streamed across the internet. Video of the speaker appears on your display, along with their presentation. Again you have access to files, etc. Even relevant E-books that can be searched through. Find something interesting on a web page thats relevant? Select it, and drag it on to the virtual lecture theater. Or type out a question and send it straight to the speaker. It doesn't matter where anyone is, they all appear in the virtual theater as if you are all in the same room.

Touch displays could be taken in any direction. Alot of the technology talked about here isn't just multitouch, but also networking. But this networking is something I think we should look at doing, why does the speaker need to be in the same room at a lecture? Bournemouth University are going to be doing their first live streaming of a lecture from one campus to another soon. If every person in the two theaters had their own display you could just have the video feed of the speaker upfront with their presentation or notes on the screen in front of you. But its the interaction with this system where multitouch comes into play. For example, placing your finger on a file to select and hold it then using your other hand to tap on the table of lecture theater to bring it to the surface so you can drag the file to who you want to show it to.

However HCI comes into it, what sort of hand gestures should we use? Already we've seen the pinching motion on the iPhone to zoom in and out of pictures. Of flicking your finger up the display to scroll down the contact list. Jeff Han shows a couple of other gestures that his team have come up with when using World Wind. For example placing two fingers from the same hand horizontal on the screen and then using the other hand to rotate the view around that axis created by the your first hand. These are important things to consider if one is looking to develop a multitouch system. What gestures are most natural? What gestures are harder to do than others? For example to delete or close a window place all five digits of a hand on the screen then bring them all together. It's not a regularly used gesture so you wouldn't accidentally close or delete something yet, it symbolizes scrunching something up in your hand and throwing it away. Symbolism, I believe, is something else to consider when looking at hand gestures. What motions do we do in real life and how can we relate them to this interaction with a display?

Of Staff and Illness

Well I'm feeling under the weather at the moment. Got a nasty cold! Claire (my partner) has it as well! Mwahahahaha! Had the day off yesterday because I felt like death-on-a-stick and slept for most of the day!

Anyways, back at work today and Andy and I are trying to wrap up the staff assignments, yes, they are still going on. I sent an email out asking the staff to check what units they can see and to let me know of any problems. Not five minutes after I sent it I get a knock on my door. In this man comes;

"Hi I'm Nick from down the hall"
Wonderful I'm thinking, "Hi, what can I do for you?"
"The email you just sent out, can you pull it up?"
Great, whats the problem with my email? I spell checked it... it was short and to the point... so I pull up the email. He starts to read it to me as if I'm stupid...
"Staff assignments... what assignments?"
"Bt assignments I just mean assigning you to the units you'll be teaching this year."
"Oh, well assignments are things we give to the students ((no shit sherlock...)) so thats a bit confusing."
"Ok, sorry."
Reading the next line he asks where he is suppost to check where his units are.
"In blackboard..." Where they have been for the past two years and will continue to be until they eventually get rid of the bloody system! Where else numbnuts?!
"Well thats not very clear is it?"
"No, I suppose it's not. Very sorry, I'll send out a follow up email to clear up the confusion."
"Thanks! That would be great" I'm sure it would you dim witted idiot. You're suppost to teach people for a degree but you can't put two and two together and think to check what units you can see in the place you have seen them for two years?

Makes my cold worse this stuff... man this is the bitchyest post yet! Anyway... I'm off to try and install Vista Ultimate now... should be fun considering it's going on a box that already has Ubuntu on...

Of Flash and Virtualness

As mentioned previously, I'm currently developing a virtual sports center in Flash. Yesterday I got a quick and dirty mock up done. Doesn't look pretty and has hardly any functionality in it for the time being, but as a quick demo to people I think it will work fine. If people like the idea then I'm going to scrap it and start over. Need to try and get access to a 3D program however so I can render some scenes for it. Shame I can't do it all in Flash so it's vectored, would make life a little easier. Still, I should be able to get some nice images done for the project!

As for 3D, I've been taking a look at Swift 3D. It's a 3D rendering program for Flash application. But some of the 3D models, etc that have been done in it are outstanding! It outputs in both vector and raster and is also capable of photo-realistic images. So with any luck I'll be able to get hold of a copy and produce some great 3D content for the VSC.

After another mini brainstroming session with Andy we decided that, at some point, we'd like to end up with a game-esque style interface. A main window showing you what room you are in, a minimap to allow quick navigation (while still be able to navigate in the main window). An info box giving a brief description of room and finally a box that displays the information you click on in the main window. So for example, you could click on a notice board in the main window and it would display the info contained in the notice board just below in a little box.

We are also going to look at controlling the content via XML documents that are imported into Flash at runtime. This means that the entire VSC can be kept up-to-date without editing the Flash at all. Hopefully we will be able to put a PHP backend somewhere to allow the creation/updating of the XML documents so that anyone can add new/edit information.

All in all it sounds like a great project to work on and something to write my dissertation on when that comes about!

Of Bluesky Syndrome

Today Andy and I have been discussing a project we are going to be working. This being a virtual sports center. We are currently looking into how we can present it. I've been doing research into 3D rendering within Flash, as we think Flash will be the best way forward for an interface base.

As Flash doesn't natively support 3D I have been looking at ways of achieving 3D in Flash. It is possible to fake 3D within Flash with the use of maths, and this would be a basic and ideal way of presenting what we want to do without overloading the browser and bandwidth. An excellent tutorial on it can be found here.


However, an open source program I found looks even better, in fact it can produce some amazing 3D work, all rendered in real time. It's called PaperVision3D.

Their blog can be found here. It has loads of information about it as it's being developed. It produces better and better quality visuals as the project progresses. It even has interactivity within it on a 3D scale, demo here.

However, although all this is amazing stuff I don't think we will be going 3D for now, but a simple 2D interface will be used.

Apart from the cool stuff we've been looking at we are still getting ready for the start of term. Still assigning staff to their units, etc. Also been working on the placement website for the SM (Services Management) school. Thats almost complete, just waiting on content really now. Everything is getting set for moving into the new term! I'll have to start practicing the presentation for the BlackBoard inductions we have to do as well soon!!

Of Steam and Skittles

Well this weekend was great. The Great Dorset Steam Fair was visited. Some of you may laugh at the thought of going to a Steam Fair but you shouldn't knock it until you've tried it! Cider was consumed, as were burgers. Then large and possibly dangerous rides were partaken in. All in all Saturday was a great night out. It's all a touch on the expsensive side but the people I went with and I had a bloody great time, definatly revisiting next year. Should be a big one as it will be the 40th GDSF!

Last night was the first skittles match of this years season. After winning every single match last year and moving up a section in the league, my team felt the pressure a little. But we pulled through with a 51 up win! Lets hope that continues through this season, maybe we can go up another section!! For those interested I'll keep you up to date on the wins (and hopefully no losses).

Finally on todays quick update, Widgets again! Haha. I have been seeing about pulling information down from a webpage though a widget recently. Basically we are looking at using a widget to pull down emails from an MS Exchange Server. Not sure if it's possible, going to have to talk to a few people! But we shall see... The problem is that the page I'm pointing it to is Web Outlook which could cause a problem. So I'm going to ask whether there is another source to point it to. Again, I'll keep you up todate. I would show you some widget designs, but I've not gotten very far on them since my Vista install has decided to be really freaking slow and thus stopping me from using Photoshop properlly. Also doesn't help that I don't have a mouse...