Is 10/GUI the Future Replacement of the Mouse and Keyboard?

Posted to Miscellaneous  |  Nathan Yau

10/GUI, a prototype interface by R. Clayton Miller, is a new way of interacting with the computer. We’re all familiar with touch screens, but what Miller proposes is separating the touch from the screen and bringing your hands back to where you normally would use a keyboard and a mouse.

Here, watch the demo, and it’ll be much clearer than my explanation:

Whether this is a viable solution to the next generation of interaction, I have no idea. Some have commented that window layout could be an issue as the line-up doesn’t work for all situations, but I think it’d just be a matter of getting used to things.

I mean, I used to think the Mac was impossible when I was forcibly switched over from PC during my first year of grad school, but now I love it.

One thing’s for sure though. Miller has great taste in online material.

[Thanks, William and Till]


  • I see a lot to like about this. The way it handles menus is clever, and I fully agree the windowing metaphor is outdated.

    However, the “con10uum” concept doesn’t seem ideal to me – I’d rather see a touch-optimized tiling window manager, ala dwm or IonWM for Linux – moving past arbitrarily-placed 2D windows doesn’t have to mean giving up the second dimension.

    Also, I saw this hinted at in the explanation but the demonstration didn’t show any minimizing windows – that would make this whole cataloguing system much nicer, I think.

  • I love the concept (and the FD shout-out :).
    The organization of the windows seems practical & intuitive (we already have application taskbars that display information linearly). The underlying thought driving the change from windows moveable in two dimensions to a linear model seems to be that allowing users more freedom to organize their desktops actually allows them to create inefficient workspaces. I’m a huge fan of interfaces that help a user stay out of her/his own way. That said, I’m not convinced that this model is a solution that would serve the majority of users.
    The multitouch is even more interesting (and really, a distinct concept from the window organization, even though they are presented together here). I prefer, by far, the macbookpro touch pad to a mouse, though I find the button on the MBP touch pad endlessly irritating and unnecessary. I don’t get why people think they need tactile feedback to have an effective experience with their devices. Along that line of thought, the only part of the demo vid I question is, why include a keyboard in addition to the multitouch pad (see 8:04)? Why assume a multitouch replaces only a mouse? Why not set up the multitouch pad to convert to a keyboard mode with a specific touch command? Then “keys” could display backlit on the multitouch pad or display translucently on screen. If displayed on screen the user can have visual feedback regarding finger placement (via the crosshairs). If displayed on the pad, the multitouch could have tiny bumps to orient the index fingers or, to avoid physical interruptions (however tiny) during navigation mode, the multitouch could have actual hot spots (warmed just enough that most people would be able to tell when their index fingers were properly oriented for typing).

    Regardless, I want one! Who’s going to build this thing?

  • In truth, the modest mouse is not as narrow a device as its single on-screen pointer might convey. Right click, scroll wheels, and mouse + keyboard combinations have greatly extend its expressiveness.

    And let’s not forget the most powerful multi-touch device: the keyboard alone. As any power vi/vim or emacs user can attest, it’s hard to compete with the speed and efficiency of keystroke-driven user interaction; but the learning curve is steep.

    In any case, it’s exciting to see novel work done in the UI space. And a nice shout-out to FD in the browser demo to boot.

  • That’s cool, but I’m still waiting for this Minority Report style Wii remote powered interface.

  • Really cool overall. Touch is amazing for interface. My 2-year-old can get around just fine in my iPod touch, but has trouble with a mouse on a computer.

    Don’t like the con10uum – I like more freedom than that (such as having many things open, but all hidden but one window).

    Sucks for people who are missing fingers.

  • In this concept, you’d still have to twist your wrists 90 degrees from their natural position, which leads to pain in the long run ( –> an accordion is healthier to play than a piano). A vertical mouse doesn’t have that problem, I can guarantee (have been using one for years).

  • Joe, not sure that’s necessarily true. For instance, on my Macbook with multi-touch, my hand rests on the touchpad at a natural angle, and all the one, two, three, and four finger gestures seem to work just fine.

    Mike, what you say is true, but it could also be argued that a mouse isn’t of much use to a person with no hands. That doesn’t mean the concept’s bad, it just means that, as with any control metaphor, it isn’t going to work for absolutely everybody.

  • floormaster squeeze October 15, 2009 at 3:00 pm

    I really like this idea. I do not like the names.

  • Dane, to find out what exactly is the natural angle, just stand up, hold nothing in your hands and relax.
    You’ll see that the palms of your hands face your thighs and are “hang down” parallel to your legs. Now angle your lower arms 90 degrees forward like you do when you’re working at a desk, but don’t change the rest, keep relaxed.
    Your hands will now positioned like you’re about to shake somebody’s hands (the arms aren’t but that’s irrelevant).
    When you use a (true) vertical ergonomic mouse, your hands will keep this natural relaxed position.
    In order to have them rest on your Macbook (or any other notebook) you have to twist your hands 90 degrees inward. That’s not all: the next move is to angle your lower arms towards the Macbook, because the keyboard area isn’t as wide as your body and it has all keys next to each other. Ideally there’d be two areas of keys, one for the left hand, one for the right with a keyless area in between and each in line with your shoulders.
    Also the flatness of any notebook keyboard is a negative factor.
    Besides the vertical mouse I mentioned there exists a keyboard that through its layout deals with all the mentioned shortcomings. I’ve been using this keyboard for years too, but since I don’t want to be mistaken for a spammer, just search yourself for the brand and type of both mouse and keyboard.

    • Joe, I see what you’re saying, you’re referring to their vertical orientation. I’ve never found using a flat typewriter surface uncomfortable, but I definitely see what you mean. I suppose a pitched, tee-pee-shaped interface would be the most natural of all in that case, wouldn’t it? Though then we’d probably want something to support our wrists so we don’t have to hold our hands up all the time. Hmm…tricky. Also, at odds with hand position when using the keyboard to type. Ultimately a flat touch surface still be the best bet, as long as the hands can approach the interface from natural horizontal angles.

  • Dane, with the -indeed somewhat teepee shaped- keyboard I’m using, the hands don’t have to held up all the time: they rest on the keyboard, while the fingers reach down in the two not flat but hollow(!) shaped key-areas that also take the difference in length of each human finger into account.
    In my case the mouse and keyboard I use were bought out of necessity. I used to think there’s nothing wrong with using a common mouse and keyboard and the way you have to position your arms and hands, until my body told me otherwise.

    I think it’s just a pity that a project like the one shown here, disregards so many of the scientific ergonomic developments.

    I really don’t feel like I’m using special medical devices designed only for the half-crippled; instead anybody could or ideally should benefit, if only to be able to work more hours.

  • like it alot, but the (think someone mentioned it more detail) key pad should be halfed – still above but angled down/around the top edges of the touchpad.. and id want buttons on the edges of the pad ..and keys that arent type oriented. plus im sure theres use for stylists too. building one of those in (pull cord?) shouldnt be too hard – mostly for drawing use.

  • The hardware part of the touch interface is pretty much the same thing as the Fingerworks TouchStream. The software part is kind of neat, but it doesn’t seem to incorporate any “gestures” besides dragging and scaling. The TouchStream also had rotation, dragging in particular directions triggering different actions based on the direction and which fingers are down, and so on. Probably 10/GUI has that, but just didn’t mention it in the video.

    The main downside to keyboarding on the TouchStream was that most people couldn’t get past 25 WPM on it. I could personally do around 50, but on regular keyboards I can do 80.

    BTW, the TouchStream technology was bought by Apple and used for the iPhone.

    “Though then we’d probably want something to support our wrists so we don’t have to hold our hands up all the time.”

    According to ergonomics best practice, you shouldn’t be resting anything on the (regular) keyboard when you’re typing.You wrists should be suspended just above the keyboard. This is the same as playing the piano. Typists and pianists can do either for many hours a day. The only thing it puts stress on is your biceps, to hold your forearms up, and your wrists, to hold your hands straight. Neither presents an RSI risk when done without tension.

  • Oh, and the TouchStream had bumps on the home row for your fingers. When you typed on it, your hands would gradually drift in random directions, and when you got far enough off you’d start pressing the wrong “keys”, and you’d have to periodically rest your hands for a moment to find the bumps again. They had to actually handle the input from the touch sensors specially in the firmware, because the bumps distorted the input.


Jobs Charted by State and Salary

Jobs and pay can vary a lot depending on where you live, based on 2013 data from the Bureau of Labor Statistics. Here’s an interactive to look.

A Day in the Life of Americans

I wanted to see how daily patterns emerge at the individual level and how a person’s entire day plays out. So I simulated 1,000 of them.

The Best Data Visualization Projects of 2014

It’s always tough to pick my favorite visualization projects. Nevertheless, I gave it a go.

19 Maps That Will Blow Your Mind and Change the Way You See the World. Top All-time. You Won’t Believe Your Eyes. Watch.

Many lists of maps promise to change the way you see the world, but this one actually does.