The vision of always-on Mixed Reality interfaces that can be used in a continuous fashion for an entire day, depends on solving many difficult problems including display technology, comfort, computing power, batteries, localization, tracking, and spatial understanding. However, solving all those will not bring us to a truly useful experience unless we also solve the fundamental problem of how to effectively interact in Mixed Reality. I believe that the solution to the MR interaction problem requires that we combine the approaches from interaction design, perceptual science, and machine learning, to yield truly novel and effective MR input and interactions. Such interfaces will need to be adaptive to the user context, believable, and computational in nature. We are at the exciting point in the technology development curve where there are still few universally accepted standards for MR input, which leaves a ton of opportunities for both researchers and practitioners. more/less
Hrvoje Benko is a Director of Research Science at Facebook Reality Labs Research, working on novel interactions, devices and interfaces for Augmented and Virtual Reality applications. He currently leads a multi-disciplinary organization that includes scientists and engineers with expertise in HCI, computer vision, machine learning, AI, neuroscience, robotics and cognitive psychology. His interests span AR/VR, haptics, new input form factors and devices, as well as touch and freehand gestural input.
He is a world-renowned expert in Human-Computer Interaction (HCI). He has coauthored more than 60 scientific articles, 50 issued patents, and has served as the general chair (2014) and the program chair (2012) of the ACM User Interface Systems and Technology conference, the premiere technical conference in HCI. He is also an Associate Editor for the TOCHI Journal, the premiere journal in the HCI field. Prior to his current role, he was at Microsoft Research, where he had the privilege of working on several projects that were released as Microsoft products or as open-sourced projects, including Microsoft Touch Mouse, Windows Touch Visualizations, Microsoft Surface, and RoomAlive Toolkit. He received his Ph.D. in Computer Science from Columbia University in 2007 and his work has been featured in the mainstream media (including The New York Times and Forbes) and on popular technology blogs.
Our reliance on computing technologies for making decisions or sense-making, has undergone a transformation from occurring in well defined settings (such as on traditional PCs) to taking place in-situ, for supporting everyday activities. In-situ user interfaces, have emerged largely from improved mobile and mixed-reality technologies and rely on mid-air input as the basis for interaction. Often, designers of such, on-the-go, user interfaces place an emphasis on enhancing end-user performance. However, we argue that supporting end-user comfort is as critical if in-situ interfaces are to become commonplace among the general population. In this talk, I will present some of our work on various aspects of end-user comfort for in-situ interactions. I will present models for estimating arm fatigue induced by mid-air input, and showcase interactive systems that have been specifically designed to circumvent such fatigue. I will also discuss elements of social comfort and present a framework for including such factors in the design process of end-user interfaces. I end my presentation with a discussion of some of the open problems in this space. more/less
Pourang Irani is a Professor in the Department of Computer Science at the University of Manitoba and Canada Research Chair in Ubiquitous Analytics. His research is in the areas of Human-Computer Interaction and Information Visualization. More specifically, his work focuses on in-situ user interfaces, interactive methods and systems for giving end-users efficient access to information "anywhere" and "anytime". His interests in translating research outcomes into consumer ready technologies has motivated his work with industry. He has collaborated with industrial partners such as the Honda Research Institute, was a visiting scholar at Microsoft Research (Redmond), and serves as a scientific advisor to several organizations, including the Human-Machine Interaction lab at Huawei Canada.