Interfaces, metaphors & multi-touch

that demo with multi-touch was warcraft 3, not WoW.

I don't play either of them.

 

this was mentioned in the article

however, that video has been around for a while, and if u think about it in practical terms it just seems like a gimmick

 

good article, I saw that multi-touch screen video before and I had been wondering about it, along with other new imput devices and how usefull they would be in my dabblings with cad and such.

the thing is with complexity you can get more utility. The simple analogy would be say a musical instrument, I played tuba for many years and all the student models tend to have 3 keys, while performance instruments tend to be 5-6 keys. With every increase in key count the imputs become more complex but the utility goes up. This doesn't mean the more complex an interface automatically the faster or better utilized it just has the propensity and it looses it's level of intuition. Likewise if you take programs like say sketch up and inventor, Sketch up on the surface is more intuitive and straight forward but lacks on the utility side. Once you start memorizing and learning the complexity of invetor's tool's you can do such things as find internal volume of objects make moving parts etc. this holds true for all sorts of software from cad and mediea creation programs, down to GUi interface menu's where old school people prefer hot key's and CLI and your grandma wants to just hit the giant blue E.

I think we are at a crossroads, professionals and gamers want more utility, and the average user wants more intuition. They aren't nessicarily opposing factions, I'm particularily interested in that multi touch screen and accompaning Gui. In say photo shop you could have more intuitive handstrokes and gesture summon various tools and use the simplicity to pan and zoom a photo while you work on it, instead of a current system of obscure key strokes (ala ctrl+a thru ctrl+z having a different function). Likewise getting grandma to poke the blue E or sort through her photo's doesn't sound that hard on this interface.

It will be interesting to see how this all pans out, below the instant mind readers and such (anyone remember those demo's that read the nerve impulses in your hand to choose left or right?) I could see the minority report level, but more like a virtual cube where you could gesture in true 3d, that way say rotating a model and zooming would be as analogous as having the item in your hand.

 

at first, that might be true, but then further in the future we could end up with a minority report style interface

something like where you control where atoms are, then you can make a UI wherever you want

 

I thought I should say something about a recent display I've been exposed to. I dont' know the product name, but it's made by Jester Tech, an Ottawa company. It projects onto the back of a piece of glass and you touch the front of the glass to control it. It has two cameras with mirrors and special lenses that peer out over the front of the glass. Using some trigonometry, it can determine where you're trying to push on the glass. The current system can detect up to 10 presses, however sometimes they'll overlapp, and things get complicated. It's not perfect, but it's really cool, and is cuirrently being used to give interactive storefront displays.

Great article btw. I had a Spaceball back in the day, but it used an ADB connection (ADB = Apple Desktop Bus).

 

I don't think the issue is the interface, but how it is generally used. The desktop itself is too disorganized by nature and is self-prone to cluttering. Adding any clever interfacing is not the solution, but simple automatic cross-content associative sorting would be a big help (grouping like items into related menu systems).

I stopped using the desktop altogether and simply added some task-specific folders to my start menu.
Everything that involves Net activities is under a specific menu which then has some more specific child folders.
Everything that involves Content Creation is under a "Create" menu (child folders for 3D/CAD, Graphics, Documents, etc.)
Everything involving pure Entertainment is under a "Play" menu (child folders for Media, Games, Etc.)

Then files are sorted by topic.
I convinced my sister to do the same and she came back two days later to thank me for the tip because it saved her so much time. If such systems could be self-governing and transparent it would make a far more significant difference than any other gimmicky system.

Touch systems are actually utterly useless in CAD applications because movements are too fluid and not terrible precise due to the fact that you're using a 2D display surface to interact with a 3D object. However, touch systems will be seeing much wider usage within the Military for Command & Control applications on very large table-sized display interfaces.

The preferred tool for most CAD work is still a mouse.
For secondary navigation or multiplane control, spaceball type interfaces are preferred. For sculpturing, ray-trace-linking pen systems are used similarly to a pen tablet, but in open air.

Unfortunately most of the interface "innovations" as proposed for daily tasks would actually be terrible time wasters and would have little or no effect on the productivity of the user. Barehand touch system for example are very unergonomic for most computer tasks and they do not offer fine control.

Pen-on-screen systems present workspace configuration issues and are mostly used for freehand painting, but have limited usability for anything outside of that venue.

There are very transparent reasons behind the persistence of the common usage of the mouse and keyboard. Other interfacing systems are not as adaptable and don't offer such a direct interface that's also easy to learn.
It's takes far longer to verbally explain or draw a diagram of what you want your computer to do then it is to simply click a few times and type a few keys.

Photoshop actually has a very common software issue: static menu systems.
It would be far more intuitive of a product if any given task could be given a toolbar button or simply be assigned to a preferred keystroke. There are too many default keystrokes, no options for making common tasks more prominent in the interface, and no means of gearing the software to the tasks you perform the most. This is a problem for many titles and it makes them more daunting than they need to be for the end user.

 

you have a point there

there is a reason people don't use touch pad laptops for gaming

and the accuracy issue is a very good point indeed

Generally, i think the biggest killer for this will be price, touchscreens are VERY expensive

that guy was demonstrating on a 30" touchscreen - imagine the price, and his space seemed very limited

its like buying a dell 30" screen, and touch pad interface too

In fact his screen was rear projected, but a projector costs somewhere around £500 (for a cheap one) and the bulbs are also ludicrous

 

Article=

The multitouch system is almost exactly what I've been waiting for for years.

The "touch" portion makes it intuitive but it's the "multi" part of the system that could make it HUGELY adaptable and powerful.

Use rear projection instead of overhead then add something like the (sadly, now out of business) FingerWorks Gesture system. This would create a tool that could scale in complexity without loosing the simple interface for noobs - metarinka can have his 6 tuba keys but a noob could still play it using only the three they know. Captain Slug could use his fingers to do more than a simple pen-on-screen system ever could - and perhaps as much as a current 3D controller can (though I think the niche market for those will remain). This could be like a one button Mac mouse OR like a ten button mouse with endless button combinations and gestures!!! (sorry, I'm excited)

I agree with yahooadam: "...i think the biggest killer for this will be price..." Once that gets solved mice will become a niche product like joysticks seem to be now. The only place I can see them holding out is in first person shooters where a 3cm (or less) mouse motion gets you all the way across a 20" screen.

[OT]

@Captain Slug: lol, I do something very similar with my Windows progs but some Linux distros already do this from the start (just installed Gimp in Mepis & there it was under "Graphics" when it installed )

@Reaper_Unreal: got any links for that system you saw? Sounds cool

[/OT]

 

Well, color me any colour you like! I missed that part completely. The only drawback I see is that you can't see the filename or description of the content of the document. Otherwise it looks pretty practical.

Touchscreens do have 1 mayor drawback imho. You need to touch the screen. Fingerprints all over my screen, bleh. I have two kids (aged 1 and 2) fingering my monitor at home and it leaves staines like you wouldn't believe.

 

I'm glad I don't have to pronounce that name out loud Welcome to the forums.

From your comments I'm guessing you have an LCD screen. Sadly most LCDs don't have a nice, easy to clean, top layer when compared to the glass on the old CRTs. I have, however, seen some very robust touch screens (in public kiosks) that look like they should be easy to clean and fairly stain-resistant.

 

Seconded.

 

you mean forthed?

(sorry, but how many times in a 2 page thread do we need to hear that wow is not warcraft3?)

 

I use 2 12" touchscreen 1024x768 LCD panels for about 6 years. They are hanging above my synths in the studio i have. I use it to control cubase sx faders and vsti instrument controls. Not all is controllable with my vingers, because some things are to small to react on my vingertips. But i can only dream of using all my vingers to control the faders. I also wish there where some cheap 19" or 20" touchscreen panels. So i can have bigger controls on the screen.

Oh! and before anyone asks how i can control both touchscreens. I use 2 computers

 

Nice article and I'd like to add a little extra nito the mix. A friend of mine has developed a touch screen interface for use in games. Originally thought of whilst we were freezing outside whilst having a smoke (no smoking in the house!), he put the spark of an idea into action and came up with 'Touch-Buddy'

Although it was originally designed for Lock On Modern Air Combat as a pseudo 'glass cockpit', it's now being developed further to be used in more games. At the moment we're trying to get a copy of the keyset for up and coming Armed Assault so that we can get together a profile for that too.

The best bit is, its entirely customisable so you decide which buttons you want and where you want them to go.

Forget all those nasty shortcuts and key combinations you have to remember for games or video and sound editing applications, just create your own profile and off you go.

The only downside is you have to get your own touch screen!

The site isnt much for now, but theres plenty of activity in the forums:

http://www.touch-buddy.com/
http://www.touch-buddy.com/forums/

 

That is very cool Indigo. All you need now is a few links to touchscreen lcd manufacturers or get in contact with them. It is a great argument to sell some small touchscreen displays...

 

What happened to the glove controllers??

Were they not used for those early 3D game thingies (where you were more or less strapped in a round ring with some giant headgear on). Anyway the idea is also shown in Jonny nemonic (I think), gloves detecting both hand movement (like the Wii controller) and finger movement would provide a powerful tool I think. Of course this would mean that you would have to put on the glove instead of just putting your hand on a mouse/screen/ball. Could be cool!!

 

The gestural interface used in Minority Report is controled by gloves. They take the movment of your hands and interpret them into intstructions.

Also Jeffs multi touch innovation uses his own method of recognizing the touch commands. He calls it FTIR (Frustrated Total Internal Reflection). Some modders who have worked with edge lit acrylic will know exactly what he is talking about.

 

Nice discussion, thanks; no WoW != Warcraft3.

 

First of all: very nice article.

Voice recognition and Touchscreens at first glance seem to be the way to go but there are some severe constrains that most people don't think about:

Touchscreens
Touchscreens are great for the types of apllications shown in the Multitouch demonstartions but fail miserably at most of today's common desktop applications. And there is one further issue: fingerprints! Imagine the fingerprints you will leave all over your 10,000.- Euro 30" Touchscreen. Better keep a large supply of those extra expensive TFT-wipes around if you don't want to see your new Photoshop masterpiece full of funny moire patterns.
Touchscreens fail at performing the number one computer task - text input. A good keyboard has a strictly meassured distance of key-travel and "pressure point" for a reason: because it's faster and more healthy to type that way than on a flat surface.
And if you simply add a standard keyboard to you Touchscreen workspace you will get a whole new set of ergonomic problems because you can't place both within optimal reach.

Voice recognition
I read an article when Opera got voice recognition (I don't remember on which site) that IMHO contained the death sentence for voice recognition: The author asked the reader to imagine voice recognition in an office environment. It wasn't about the noise levels connected with everyone talking to their computer.... Do you really want every of your colleagues to know that you are surfing to a nudie site? Again? Do you really want to hear your supervisor e-mail your work assessment to your boss?

I don't think that either Touchscreens or voice recognitions are valid input methods for today's most used applications, so I'll stick with my keyboard and mouse until something better arrives.

 

@wraith: I totally agree about voice control (though it will / does have a niche market) (- though, when I first saw Star Trek (late 70s) I wondered why Kirk gave commands then others input them when the ship's computer should have been able to understand & execute those orders)

However, I don't think that multi-touch will (or is intended to) replace keyboards any more than mice have. As for the rest: Apparently Philips thinks touch-screens are tough enough for games