Reality-based interaction

One of the sessions I attended at this year’s CHI conference was called Post-WIMP (WIMP = Window, Icon, Menu, Pointer). It covered recent trends in user interface design and highlighted attempts to move beyond the typical WIMP concepts we generally use today. In one of the talks, the paper Reality-based interaction: a framework for post-WIMP interfaces by Robert J.K. Jacob et al. was presented which I found particularly interesting. Specifically, the authors developed a framework which allows for the comparison of today’s emerging post-WIMP interfaces according to several aspects, based on the underlying notion of a foundation of these interfaces in the physical, real world.

For the authors, up to four themes of the physical world can be reflected in these new user interfaces:

  • Na├»ve Physics
  • Body Awareness & Skills
  • Environment Awareness & Skills
  • Social Awareness & Skills


While Jacob et al. argue that building user interfaces around these innate human skills can have several positive benefits, such as reduced cognitive workload, they also stress that designers should ideally deviate from these concepts in explicit cases. An example they give is that of walking in a virtual world. The command for walking could then be augmented by a command for flying which should remain as analogous as possible to its realistic sibling. Unless some additional power (such as flying) was gained by a command, the authors state, a designer should always use the command closest to reality.

This rationale has become a central criteria for me when judging novel reality-based interfaces, many of which have been promoted (and hyped) in online videos over the last months. I will soon post a few examples of reality-based interfaces which I have recently discovered and put them into relation with the four themes above.