The Hand

The idea came from the classic photo of blurry hands behind bluish glass: a hand that looks sharper the closer it gets to the glass. As a second stage, the glass had to behave like lycra fabric when the hand pushes on it.

The goal was to do it all in WebGPU with TSL, and for the hand not to be a recorded animation: the pose and the movements are computed every frame on a rig built by script in Blender.

The blur of each pixel depends on how far the hand is from the glass at that point.

• The hand is rendered with MRT into two outputs: the colour and the distance to the glass.
• A 6-level blur pyramid, like the one used for bloom, blurs both at once.
• In the composite, the distance is estimated by adding up every level weighted by coverage. That way the edge of the hand blurs the same as its inside and no halos show up.
• The blur level comes from the distance and is interpolated between the two nearest levels. Far away, the opacity also drops.

When the hand goes through the plane of the glass, the glass rises like fabric wrapping it.

• An orthographic camera renders the depth of the hand from the camera side: that says how far it goes through the glass at each point.
• Compute shaders on storage textures turn that depth into height. With jump flooding, each texel finds the contact point that lifts it the most, which gives round tents with ridges between the fingers.
• The height is smoothed and each texel chases it with a spring, so the fabric has inertia and bounce. The edges stay fixed, like a frame.
• In the glass material, the height displaces the vertices and the normal comes from finite differences.

The final mesh is the hand and forearm of the realistic male body from Human Base Meshes, simplified to about 15,000 vertices. The 17-bone rig is generated by Python scripts in Blender: they place the joints with slices of the mesh and anatomical proportions, compute the weights and correct them so the knuckles do not drag the palm and the wrist bends gradually.

Poses are stored as data (degrees per joint) and checked in Blender before they go into the scene. They follow rules taken from references: the fingers bend in a cascade, the last phalanx barely bends at rest and the pinky follows the ring finger, since they share tendons.

The interaction has three layers, each with its own range:

• Aim: only when the mouse is near the wrist. The forearm turns on its axis, and the wrist and the index knuckle finish the adjustment, with per-axis limits so the mesh does not deform.
• Reach: the closer to the index fingertip, the more the index stretches, and the others to a lesser degree.
• Idle: a soft sway per finger, which the index loses when it is about to point.

• Performance with the glass is still a problem. With the glass on, the frame rate drops a lot compared with the scene without it, and more so on high pixel-density screens. Measuring piece by piece, the biggest cost was the blur passes: I halved them, merging the two pyramids with MRT and using linear sampling, without changing the image, but it was not enough to fully solve it. Working at a lower resolution looked worse, so I dropped it, and skipping the fabric while the hand is not touching it made no difference.
• The hand is drawn three times per frame: for the frosting, for the fabric depth and in the scene.
• The fabric is not real physics: the shape is a geometric target with one spring per texel, with no waves travelling across it.
• Pointing upward falls short: the wrist starts flexed and the limits that prevent deformation do not let it get all the way.
• Only tested on Chromium with WebGPU.

• Hand mesh: realistic male body from Human Base Meshes (Blender Studio, CC0).
• Project structure and the fabric depth technique, from the Post-processing and Snow lessons of Three.js Journey's WebGPU & TSL course.
• The first version used an AI-generated model (Rodin), later replaced.