Kinect put a depth camera in millions of living rooms

↗

A living-room launch, dated

Kinect for Xbox 360 went on sale in North America on 4 November 2010, according to Microsoft's launch-day release, which describes a sensor that "sees you move, hears your voice and learns how you play" and lets a player control a console without a handheld controller. The release is a marketing document and, true to that genre, describes what the product lets a user do rather than how the sensor measures anything; it names no depth technology and gives no range or resolution figures. What it does establish, reliably, is the date and the premise: a consumer device that reads a room and a body well enough to replace a controller, sold at more than 30,000 retailers on day one.

What the technical documentation actually specifies

The engineering detail arrives later, in Microsoft's own developer documentation for the sensor family Kinect became. The archived Kinect for Windows SDK 2.0 API overview, covering the second-generation sensor introduced alongside Xbox One, specifies a depth frame as a grid of 16-bit values giving the distance to the nearest surface at each pixel, in millimetres, out to a stated maximum of eight metres with reliability degrading from roughly 4.5 metres onward. That depth frame is pixel-aligned with an infrared image from the same sensor, and the SDK layers a body-tracking frame on top, capable of following skeletal joints for up to six people at once. The successor product line, Azure Kinect DK, is described in Microsoft's documentation as pairing a depth sensor with a spatial microphone array and an orientation sensor in one device, the same combination of colour, depth and audio that made the original Kinect usable for full-body interaction without worn hardware.

From a game peripheral to a research instrument

A depth image with millimetre-scale distance values, delivered live at video frame rates, is precisely the raw material volumetric-capture and telepresence research needed and had not had cheaply before. Microsoft's own Holoportation project page states that its current system uses "Azure Kinects for depth + RGB capture," describing them as providing "some of the highest quality depth data of any RGB+D sensor on the market," and that the technology "allows high-quality 3D models of people to be reconstructed, compressed and transmitted anywhere in the world in real time." That is a documented line from a 2010 games accessory to a real-time 3D telepresence pipeline a decade later, though it is a lineage of sensor technology and price point, not evidence the original console hardware could do what Azure Kinect and modern reconstruction software do.

Questions to bring to a demo

• Which generation of Kinect-derived sensor supplied the depth data, and what is its documented range and resolution?
• Does the capture rely on one sensor or several, and how does that affect coverage of a moving subject?
• Is the reconstruction shown live, or was the depth data processed and cleaned up afterwards?

Kinect's lasting contribution to display-technology production is the depth stream it made affordable, not the console feature it originally shipped for; the documentation for the sensors that followed it is where that becomes measurable.