The rapid evolution of computer technology has made it possible to generate increasingly realistic virtual environments. However, despite the promise and hype of virtual reality that began in the 1980s, actual usage of the technology (outside of research settings) is limited primarily to derivatives of truly immersive VR, such as 3D video games. We suggest that one of the greatest barriers to widespread adoption of VR technology is the lack of intuitive interaction techniques available in virtual environments. True 3D manipulation is a 6 degree of freedom (6DOF) task, i.e. three independent axes of movement and three independent axes of rotation. Although we regularly perform such manipulations in reality with our hands, in a virtual environment, we are faced with a difficult choice of input devices: either use a familiar, but less immersive device, such as a mouse or use any of a number of unfamiliar and problematic 3D input devices. However, a mouse is a 2DOF device, providing movement in two directions only. Without support from software techniques that map 2D mouse movements into 6DOF operations, such a device is inadequate for manipulating objects in 3D.
One alternative is to use a higher DOF input device (e.g. 3D wands or gloves) which can provide high-precision 3D positioning and orientation. This is an attractive solution for immersive environments where the user is standing. Another possibility is to use a mouse with indirect manipulation techniques, such as 3D widgets, or control keys to toggle the axis of movement/rotation. However, both of these approaches present unique sets of problems. Full 6DOF input devices tend to be difficult to use and indirect manipulation techniques force the user to mentally decompose each manipulation into a series of steps. Both solutions require the user to think about the interface, rather than the task at hand.
A third approach is to use constraints, such as gravity, collision avoidance, etc. Building on this idea, we propose instead the use of "intelligent" 2DOF interaction techniques that automatically and intuitively map 2D mouse motions to 3D movement. One such technique, used for moving objects in the Sesame 3D conceptual modeling tool has been shown in experiments to be significantly faster and more accurate than both 6DOF input and 3D widget interaction techniques. This technique is an example of 3D direct manipulation, allowing users to "click and drag" objects in the scene. Dragging the object causes it to slide along the foremost visible surface behind it in the scene. This sliding motion behaves in a very predictable, natural way, and suggests that a mouse with suitably intelligent software support may be an ideal "3D" input device for certain VR and 3D applications, notably 3D modeling.
Based on this, and related research, we have composed a list of several guidelines for researchers developing immersive applications requiring 3D interaction. These guidelines are intended to aid designers in choosing methods of interaction, but are broad enough to encompass any type of 3D application requiring the accurate interactive positioning of objects in the environment. Current research suggests that techniques based on these guidelines may not only benefit mouse-based input, but input from higher-DOF devices as well.