Virtual Reality (VR) 3D tracking systems are susceptible to minor fluctuations in signal (jitter). In this study, we explored how different levels of jitter affect user performance for 3D pointing. We designed a Fitts' Law experiment investigating target positional jitter and cursor rotational jitter at 3 different depth distances. Performance was negatively affected when up to ± 0.5° rotational jitter was applied to the controller and up to ± 0.375 cm positional jitter was applied to the target. At 2.25 m distance, user performance did not improve with decreasing positional jitter or rotational jitter compared to the no jitter condition. Our results can inform the design of 3D user interfaces, controllers, and interaction techniques in VR. Specifically, we suggest a focus on counteracting controller rotational jitter as this would globally increase performance for ray-based selection tasks.