The Guided Evaluation Method: An Easier Way to Empirically Estimate Trained User Performance for Unfamiliar Keyboard Layouts Article

Abstract:

To reliably identify if novel/unfamiliar keyboard layouts, like OPTI, can out-perform QWERTY, participants have to undergo lengthy training through longitudinal studies. To address this logistical bottleneck for creating trained users, a popular approach in the literature requires participants to type the same phrase repeatedly. To validate this easy-to-use method, we first performed a study where participants typed the same phrase 96 times for both OPTI and QWERTY. Results showed that this approach has the potential to estimate trained user performance, yet, accurate estimates still require training over several days and, therefore, do not eliminate the need for a longitudinal study. Thus, we propose the Guided Evaluation Method (GEM), a novel approach to quickly estimate the trained user performance with novices. To validate the GEM, we compared OPTI and QWERTY for touch-based typing. Our results reveal that in a matter of minutes, participants exhibited performance similar to an existing longitudinal study – OPTI outperforms QWERTY. Thus, GEM enables much faster estimation of trained user performance, which will support reigniting research on better text entry methods as researchers are no longer required to conduct a more logistically expensive longitudinal study.