Tracking one head turned a flat screen into a window

A single demonstration, documented by its own creator

In December 2007, Johnny Chung Lee, then a graduate student, published a demonstration showing that a standard flat computer monitor could appear to have real depth if the software knew exactly where the viewer's head was and re-rendered the scene accordingly. His own account, posted to his project blog at the time and later maintained on his project page, describes using a Nintendo Wii remote's built-in infrared camera pointed at the viewer, paired with a head-mounted sensor bar of two infrared LEDs worn on a pair of glasses, to track head position in real time.

What creates the illusion, and its one real requirement

The mechanism is motion parallax, not stereoscopy: as the tracked head moves left, right, up, down, closer or further away, the software recalculates the 3D scene from that exact viewpoint and redraws it on the flat screen, the same geometric correction a real window performs automatically as a person walks past it. Lee's own description states the software 'renders view dependent images on the screen' so the display 'properly reacts to head and body movement as if it were a real window creating a realistic illusion of depth and space'. This requires only one tracked viewer, low-latency head position data, and a rendering engine that can update the projection matrix every frame; no glasses, no lenticular sheet, and no second image are involved at all.

What the demonstration does not solve

Because both eyes still see the identical flat image at any instant, the illusion is distinct from true stereoscopic or autostereoscopic depth, and Lee's blog post is explicit about the gap: without also adding stereoscopic glasses, 'conflicting stereo depth cues' remain, since motion parallax alone convinces the brain of depth from head movement while binocular disparity, the separate cue from having two eyes, still reports a flat surface. Adding real stereo would mean shutter or polarised glasses synchronised to the display's refresh rate, described in the post as requiring 'quite a bit more hardware' than the head-tracking demonstration alone used. This single-viewer, motion-parallax-only technique is also precisely the principle later commercial single-viewer spatial displays built on, tracking one viewer's eyes to drive a re-rendered, glasses-free image.

Questions to bring to a demo

• Is the depth effect coming from head-tracked motion parallax alone, from true stereo, or from both combined?
• How many viewers can the tracking system serve, and what happens to everyone else in the room?
• What is the tracking latency, and how does the illusion degrade at its edges or during fast head movement?

A 2007 hobbyist demonstration, built from a games console remote, established the working principle behind every glasses-free single-viewer spatial display that has followed: track the eyes, redraw the scene, and let parallax do the rest.