Previous work has shown that the mismatch between disparity and optical focus cues, i.e., the vergence and accommodation conflict (VAC), affects virtual hand selection in immersive systems. To investigate if the VAC also affects distal pointing with ray casting, we ran a user study with an ISO 9241:411 multidirectional selection task where participants selected 3D targets with three different VAC conditions, no VAC, i.e., targets placed roughly at 75 cm, which matches the focal plane of the VR headset, constant VAC, i.e., at 400 cm from the user, and varying VAC, where the depth distance of targets changed between 75 cm and 400 cm. According to our results, the varying VAC condition requires the most time and decreases the throughput performance of the participants. It also takes longer for users to select targets in the constant VAC condition than without the VAC. Our results show that in distal pointing placing objects at different depth planes has detrimental effect on the user performance.