Three-dimensional user interfaces (3D UIs) allow users to interact with virtual objects, environments, or information using direct 3D input in the physical and/or virtual space. With the advent of systems like the Nintendo Wii, 3D UIs have entered the mainstream. Thus, research on 3D UIs is more relevant than ever. In this article, the founders and organizers of the IEEE Symposium on 3D User Interfaces reflect on the current state-of-the-art in several key aspects of 3D UIs, and speculate on future research directions for 3D UIs as well.