Crossing Realities Has a Cost: Evaluating AR Second Screens for AI-Assisted Desktop Authoring Inproceedings

Abstract:

AI-assisted authoring requires users to frequently exchange user-generated and AI-generated content between a primary workspace and a Large Language Model (LLM) assistant. Such repetitive actions can fragment attention and compete for limited display space. While Augmented Reality (AR) has been proposed as a way to extend desktop workspaces with virtual displays, just adding a spatial display area does not address the repeated attention shifts, input-focus changes, and content-transfer steps required in LLM-assisted workflows. This creates a gap in understanding whether AR second screens support such workflows or introduce cross-reality transition costs that limit their effectiveness. To investigate this gap, we designed an ARSecondScreen system around a unified keyboard-and-mouse interaction model and evaluated it against SingleScreen and DualScreen desktop setups. Results show that DualScreen remained more efficient than ARSecondScreen, with significantly lower task completion time and switching cost. ARSecondScreen also produced higher task and LLM idle ratios, indicating re-engagement costs when users crossed between physical and virtual workspaces. At the same time, participants rated AR positively and preferred it significantly over SingleScreen, with no significant difference in preference compared to DualScreen. These findings show that spatially extending AI-assisted workflows into AR can be appealing, yet effective cross-reality interfaces should minimize transition costs rather than simply add a virtual display space.