Three.js From Zero · Article s13-11

R3F VR & WebXR

R3F VR & WebXR is Article s13-11 of Three.js From Zero, a MasterAllArts free interactive lesson for artists learning creative 3D on the web.

← Three.js From ZeroS13-11 · R3F Mastery

Season 13 · Article 11 · R3F Mastery

Take any R3F scene into VR with @react-three/xr — controllers, hand tracking, locomotion, teleport. Quest-ready in JSX. The walled garden of native VR cracked open by an open web standard.

Code-walkthrough article

Requires Quest 2/3/Pro or another WebXR-capable headset. Browser-based VR — no app install, no store.

Setup

npm install @react-three/xr

import { XR, createXRStore, VRButton, useXR } from '@react-three/xr';

const store = createXRStore();

<VRButton store={store} />
<Canvas>
  <XR store={store}>
    {/* your normal R3F scene goes here */}
  </XR>
</Canvas>

Three lines added to any R3F scene. <VRButton> is the "Enter VR" button. Once the user enters, R3F switches rendering to the XR session — stereoscopic, head-tracked, with controllers visible in-scene.

Hand tracking & controllers

import { Hands, Controllers } from '@react-three/xr';

<XR store={store}>
  <Controllers />            {/* renders the physical controllers as 3D models */}
  <Hands />                  {/* renders the user's hands when tracking is on */}
  <Scene />
</XR>

Quest 3 supports hand tracking out of the box — no controllers needed. <Hands /> renders the 25-joint hand skeleton with realistic shading. Pinching gestures are exposed as events.

Locomotion: teleport

import { TeleportTarget, useXR } from '@react-three/xr';

<TeleportTarget>
  <mesh receiveShadow rotation={[-Math.PI/2, 0, 0]}>
    <planeGeometry args={[20, 20]} />
    <meshStandardMaterial />
  </mesh>
</TeleportTarget>

Any mesh wrapped in <TeleportTarget> is a valid landing zone. The thumbstick + trigger pattern (thumbstick aim, trigger to confirm) is bound automatically. No code beyond the wrapper.

Pinch / grab events

function Grabbable() {
  const ref = useRef();
  return (
    <mesh ref={ref}
      onPointerOver={(e) => { /* hover */ }}
      onClick={(e) => { /* pinch confirmation */ }}
    >
      <boxGeometry />
      <meshStandardMaterial />
    </mesh>
  );
}

R3F's pointer event system extends naturally to VR — onClick fires on pinch / trigger, onPointerOver fires when ray hits the object. Same API in browser and VR.

Headset-aware effects

function Toggle() {
  const session = useXR(s => s.session);
  if (session) {
    return <Html>You're in VR!</Html>;
  }
  return <Html>Click "Enter VR" to enter.</Html>;
}

useXR exposes the active WebXR session. Use it to swap UI, disable mouse-only controls in VR, or render different geometry per mode.

The hard things made easy

  • Eye-tracked foveation (Vision Pro): auto-applied if the headset supports it.
  • Reprojection / time warp: WebXR handles, R3F gets it for free.
  • Refresh rate: useFrame fires at the headset's native rate (72/90/120Hz).
  • Reference space: defaults to local-floor (room-scale with origin at floor level), no setup needed.

Common first-time pitfalls

"VRButton doesn't appear." Browser doesn't support WebXR, or you're on https://localhost (WebXR requires HTTPS, except localhost which is allowed).
"Scene looks fine in browser, wrong scale in VR." You're working in arbitrary units. In VR, 1 = 1 meter. A "tall" human is 1.7 units. Build to scale.
"Stuck inside a mesh on entry." Default starting position is (0,0,0). Place your reference origin somewhere reasonable, or add a teleport-to-start when session begins.

Exercises

  1. VR-ify the game capstone. Take S13-10. Add <XR> wrapper. Use the right-hand trigger as the jump button via useXR events.
  2. Inspect a model. Place a glTF model at floor level. Walk around it in room-scale. Add a teleport floor so you can move beyond physical room bounds.
  3. Two-handed sculpting. Each controller controls a 3D cursor; pinch with both hands at once = stretch the cube between them. Bimanual 3D editing in 50 lines.