World's First 3-D Image Sensor Technology Made Available to Developers

Research and development professionals in robotics, materials handling, biometrics, vehicle safety, security, virtual control systems, and a wide variety of other potential or existing applications for computer or "machine" vision now have access to low-cost, real-time 3-D imaging technology based upon electronic perception technology, Canesta announced today. The announcement accompanied the unveiling of a complete development platform for the company's new Equinox perception chip - a low-cost, real-time 3-D sensor that makes it possible for ordinary devices and machines to "see" and interact with the world around them.

The development platform - which includes a single-chip 3-D "camera", a USB interface to personal computers, and a Windows-based software development environment - will permit researchers to easily envision and rapidly prototype a wide range of novel and potentially important machine vision applications, particularly those not practical before the existence of the tiny Equinox chip.

"True 3-D image sensing is extremely useful for applications that need to look at a scene, and decompose it into objects," said Canesta vice president Jim Spare. "If you can not only resolve objects from the background, but also can acquire information about their size, shape, and distance from the sensor, then you open up an entire field of applications that are otherwise difficult or impossible to do. And, if you can accomplish this with a low-cost chip, then the opportunities are nearly unlimited."

According to Spare, Canesta's Equinox chip resolves a scene into "pixels", as does an ordinary camera chip, but instead of simply providing the brightness of each pixel, Equinox additionally provides the distance from each picture element to the sensor chip. In effect, this renders the scene into three dimensional objects that are easily processed by even the tiny computers in ordinary devices such as cell phones or PDAs. "We call this 'primary 3-D data'," said Spare, "and it eliminates the need for massive calculations of one or more 2-D images to accomplish the same tasks."

First Single-chip 3-D Camera
The development kit bundles a small 3-D camera - built using the Equinox chip - with a standard USB interface and cable, an application programming interface (API), and documentation. Developers will be able to prototype their application and code on a Windows PC, and then at future date, embed the same application - along with an Equinox chip - in the end-user device, similar to the way single chip CMOS cameras are integrated into cell phones today.

Numerous Applications
"The significance of this announcement," said Dr. Carlo Tomasi, professor of computer science in artificial intelligence at Duke University and former software architect at Canesta, "is that now, thousands of creative individuals in companies, research labs, universities, or even in garage shops, will have the ability to explore electronic perception technology on their own, and to develop interesting and important new devices and applications that can actually 'see'." Tomasi said that in addition to the creativity that get encouraged, the availability of a development kit will give its users a clear path to widespread, volume commercial deployment of their applications. "This is a significant step forward for computer and machine vision," he said, "not unlike the first development kits for microprocessors, or embedded processors, which greatly spurred their growth."

The general class of applications possible using this version of Canesta's electronic perception technology include size and depth detection, image segmentation, object classification, object tracking and location analysis, and human interaction. Spare explained each of these in further detail:

"Size and depth detection is used to determine the size of an object, and/or its distance from the sensor," he said. "This would enable, for example, a factory robot to accurately pick up or avoid a package, or a 'smart truck' to evaluate and monitor its cargo load.

"Image segmentation, by comparison, refers to the process of separating an object out from the background in a scene." A conventional camera setup would have difficulty, Spare said, in recognizing a human form in the passenger seat of a car, particularly if that person's clothing and the car's upholstery had no discernable texture. "But with our technology, the location, size and shape of the person would be instantly determined," he said. This would be invaluable, he stated, for making an airbag that could choose to deploy or not depending on whether the occupant was an adult or child.

Additionally, with what is, in effect a "wire frame" view of the scene, you could actually classify objects such as "child" or "grocery bag." "This is object classification," said Spare.

Object tracking and location analysis is made possible by the sensor's ability to render a 3-D scene at 30 full frames per second. Said Spare: "Object tracking enables you to create a warehouse or home security application that could actually determine if a moving object is a human - worthy of a security alert - or simply a small animal."

Finally, he said, a wide range of machine-to-human interfaces make up the human interaction category. "With the technology, human gestures or actions can control numerous types of systems from a distance," said Spare.

The Electronic Perception Development Kit
Canesta's Electronic Perception Development Kit, or "EP DevKit" provides all of the tools necessary to create fully functional electronic perception applications using the provided real-time 3-D camera, and a personal computer. It is an "out of the box" solution, with no additional hardware development required, that provides access to raw frames of 3-D information via a standard, USB interface. This enables the rapid development of applications that depend upon 3-D information.

The EP DevKit includes a light source and matching optics which may be ordered in three versions, differing by field of view. EP DevKit Model # DP208 provides an 80-degree field of view; the DP205 gives 55 degrees; and the DP203 offers 30 degrees. Canesta will entertain developing other versions.

The included software development kit (SDK) provides an application programming interface (API) for writing and debugging C or C++ applications for the DP200 using Microsoft Visual Studio 6.0 on a Windows 2000 or Windows XP PC. Through the API, developers can access raw brightness and depth image frames, as well as set sensor parameters such as the frame rate, shutter speed, operation mode, window size, or the region of interest.

The software development kit comes with documentation and sample code to illustrate the operation of the kit. It also includes a demonstration program that shows a color-coded 3D view of any object that is placed in the imaging area of the sensor.

The API is common across Windows and embedded platform implementations. Therefore, application software can be ported from the PC to a future embedded environment with a minimal amount of changes. The SDK provides programming guidelines for writing efficient embeddable applications.

Source: Canesta