Mixing HTML and WebGL

Season 14 · Article 08 · Setup & Tooling

A Three.js scene with HTML overlays — tooltips on 3D objects, fullscreen menus, form inputs that read text. The combinations that look impossible but use four basic patterns underneath.

The four patterns

1. Overlay HTML — fixed/absolute positioned div over the canvas. Best for HUD, menus, score.
2. World-anchored DOM — div positioned via JS each frame to track a 3D point. Best for tooltips.
3. CSS3DRenderer — real DOM elements in 3D space, transformed via CSS3D. Best for embedded UI.
4. DOM as texture — render an HTML element to a canvas2D, use that as a Three.js texture. Best for "TV in scene."

Pattern 1: Overlay HTML

<div id="canvas-host" style="position: relative">
  <canvas>...</canvas>
  <div id="hud" style="position: absolute; top: 20px; left: 20px; color: white; pointer-events: none">
    Score: 0
  </div>
</div>

pointer-events: none on the overlay lets mouse events fall through to the canvas. Use selectively — buttons need to keep their pointer events.

Pattern 2: World-anchored DOM

const tooltip = document.getElementById('tooltip');
const vec = new THREE.Vector3();

function update() {
  vec.copy(targetMesh.position);
  vec.project(camera);     // converts world coords to NDC (-1 to 1)

  const x = (vec.x * 0.5 + 0.5) * canvas.clientWidth;
  const y = (vec.y * -0.5 + 0.5) * canvas.clientHeight;

  tooltip.style.transform = `translate(${x}px, ${y}px)`;
  tooltip.style.visibility = vec.z > 1 ? 'hidden' : 'visible';
}

vector.project(camera) is the magic — converts world coords to clip space (NDC). Map NDC to pixels and you have the screen position of any 3D point.

The vec.z > 1 check hides the tooltip when the target is behind the camera.

Pattern 3: CSS3DRenderer (real DOM in 3D)

import { CSS3DRenderer, CSS3DObject } from 'three/addons/renderers/CSS3DRenderer.js';

const cssRenderer = new CSS3DRenderer();
cssRenderer.setSize(w, h);
cssRenderer.domElement.style.position = 'absolute';
cssRenderer.domElement.style.top = '0';
host.appendChild(cssRenderer.domElement);

const el = document.createElement('div');
el.innerHTML = '<form>...</form>';
const obj = new CSS3DObject(el);
obj.position.set(0, 1, 0);
scene.add(obj);

// In loop: render both
renderer.render(scene, camera);
cssRenderer.render(scene, camera);

The HTML form actually works — typing, focus, submit, all native browser behavior. The trade-off: the form can't be occluded by 3D meshes (it renders on a separate layer). Workaround: a "pseudo-occlude" using opacity based on depth.

Pattern 4: DOM as texture

// Render an HTML element to canvas2D using html2canvas or similar
import html2canvas from 'html2canvas';

html2canvas(document.getElementById('source')).then((canvas) => {
  const tex = new THREE.CanvasTexture(canvas);
  material.map = tex;
  material.needsUpdate = true;
});

Heavy operation (~50ms). Use for static or rarely-updated UI. For real-time DOM-in-3D (typing visible inside a 3D screen), the techniques get exotic — usually CSS3DRenderer behind a depth mask.

The decision tree

• Do you need keyboard input? → CSS3DRenderer or Overlay.
• Does it need to be occluded by 3D? → World-anchored DOM (with depth-test in JS) or render to texture.
• Is it a HUD? → Overlay HTML.
• Is it a label on a 3D object? → World-anchored DOM.
• Is it a virtual screen / monitor in 3D? → DOM as texture.

Common first-time pitfalls

Exercises

1. Add a 3D label. A floating label above a mesh. Implement with world-anchored DOM. Hide it when target goes behind camera.
2. Settings menu overlay. A full overlay panel with sliders for camera speed, audio volume, etc. Open/close with a button.
3. Computer screen in a scene. A 3D monitor mesh with a CSS3DRenderer'd webpage on its face. Bonus: a form on the page that the user can type into.