What’s a Natural User Interface and Why Should I Care?

In the near future the majority of user experiences will be NUIs.

I was at a user experience presentation about augmented reality. At the end of an interesting talk, as audience members were gathering belongings and checking phones, the person sitting beside me turns and says, “I don’t know. I prefer my GUI” while gesturing at his phone. I suppressed my design nerdiness to correct him, but in my head I was exclaiming “Technically, that’s not a GUI, it’s a NUI!”

I know the gist of what he was saying was, “I like having a visual interface.” That’s ok. People use the term GUI — or graphical user interface — loosely to mean any visual interface screen. It’s convenient and we do need a way to refer to the visual interface screen of an otherwise multimodal interface like a smartphone. But if designers and developers don’t fully understand the differences between a GUI and a NUI — a natural user interface — the quality of any user experience that is not a standard GUI (desktop/mouse configuration) can suffer.

If you use NUI principles to design a touch interface you will get much better results than if you default to GUI principles. So what’s the difference?

NUI vs GUI

So what exactly is a NUI? A natural User Interface is almost any interface that goes beyond a standard GUI. It’s a concept that appears simple at first glance, but then defies a simple, one-sentence definition. While a command line interface (CLI) and a GUI can each be described concisely (as in “the user types commands to the computer in the form of text” and “the user directly manipulates graphical representations on a computer screen with a pointing device,” respectively) a NUI is not so straightforward. Because it is not limited to particular input and output technology as CLIs and GUIs are, a NUI has exponentially more possibilities for realization and thus is more difficult to define. In fact, NUIs are not defined by the input and output technology but instead by the user experience. To me, that seems like progress for the user experience field!

Many people end up latching onto one-line definitions of a NUI anyway, regardless of the need for more explanation. Here are a few popular definitions: “it’s a gesture interface,” “it has multimodal interaction” or “it mimics real-world interaction.” These are not false statements about any possible example of a NUI, but they in no way indicate the range of possibilities that a NUI can encompass.

So, what exactly is a NUI?

If I had to choose a one-line definition for a NUI, I like this one from Daniel Wigdor: Content is the interface.¹ Although it is an ambiguous statement, at the same time it is completely to the point. The statement proposes a solution to the primary problem CLIs and GUIs have, which is that the user must operate controls (menus, buttons, etc.) to influence something (document, dataset, etc.) rather than interact directly with the thing itself.

It suggests removing a layer of the interface to more directly and transparently access the content. It also suggests that if the content is what the user is interacting with directly than the content will potentially have behaviors that clue the user into how to interact with it.

I find Wigdor’s definition useful as a most basic tenet while I’m designing for a NUI. Asking the question, “is the user interacting with the content as directly as possible?” helps me evaluate each aspect of potential interactions and modify as needed to be more in keeping with the idea of natural interaction. It also helps me break my knee-jerk GUI habit of using buttons or menus, which is so ingrained, as an automatic solution to any visual interface requirements.

If you are confused by how a user can interact more directly with content on a screen, here’s a simple example of replacing button controls with a gesture. Imagine that you have a group of images with “next” and “previous” buttons to browse through them. Replace the “next” and “previous” buttons with a swipe or drag gesture. Although the buttons were easier to use with a mouse, swiping directly on the images rather than using button controls is easier for the hand. It’s a more natural interaction.

An Even Stronger Definition

If you’re like many designers, you’re probably still struggling to understand exactly what constitutes a NUI. Here are two more in-depth (still imperfect, but good) definitions:

• A NUI is a type of user interface that is designed to feel as natural as possible to the user. The goal of a NUI is to create seamless interaction between the human and machine, making the interface itself seem to disappear. ²

• A NUI is an emerging paradigm shift in man-machine interaction of computer interfaces to refer to a user interface that is effectively invisible, or becomes invisible with successive learned interactions, to its users. The word natural is used because most computer interfaces use artificial control devices whose operation has to be learned. A NUI relies on a user being able to carry out relatively natural motions, movements or gestures that they quickly discover control the computer application or manipulate the on-screen content. ³

Though these definitions are still rich with ambiguities, the concept of  a natural user interface is very useful in the evolution of UI/UX design. After all, the design goal is to keep making it easier for the user — and the advent of the NUI is a big step in that direction.

References

1. Microsoft Research Paper

http://research.microsoft.com/en-us/um/redmond/events/fs2010/presentations/wigdor_a_whole_nui_world_rfs_71210.pdf

2. Tech Term.com definition of a Natural User Interface (NUI)

http://www.techterms.com/definition/nui

3. Wiki definition of a Natural User Interface (NUI)

http://wiki.nuigroup.com/Natural_User_Interface#Publications