3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology

Since we are living in a three-dimensional world, an adequate description of our environment for many applications includes the relative position and motion of the different objects in a scene. Nature has satisfied this need for spatial perception by providing most animals with at least two eyes. This stereo vision ability is the basis that allows the brain to calculate qualitative depth information of the observed scene. Another important parameter in the complex human depth perception is our experience and memory. Although it is far more difficult, a human being is even able to recognize depth information without stereo vision. For example, we can qualitatively deduce the 3D scene from most photos, assuming that the photos contain known objects [COR]. The acquisition, storage, processing and comparison of such a huge amount of information requires enormous computational power - with which nature fortunately provides us. Therefore, for a technical implementation, one should resort to other simpler measurement principles. Additionally, the qualitative distance estimates of such knowledge-based passive vision systems can be replaced by accurate range measurements. Imaging 3D measurement with useful distance resolution has mainly been realized so far with triangulation systems, either passive triangulation (stereo vision) or active triangulation (e.g. projected fringe methods). These triangulation systems have to deal with shadowing effects and ambiguity problems (projected fringe), which often restrict the range of application areas. Moreover, stereo vision cannot be used to measure a contrastless scene. This is because the basic principle of stereo vision is the extraction of characteristic contrast-related features within the observed scene and the comparison of their position within the two images. Also, extracting the 3D information from the measured data requires an enormous time-consuming computational effort. High resolution can only be achieved with a relatively large triangulation base and hence large camera systems. A smarter range measurement method is the TOF ( T ime- O f- F light) principle, an optical analogy to a bat´s ultrasonic system rather than human´s stereo vision. So far TOF systems are only available as 1D systems (point measurement), requiring laser scanners to acquire 3D images. Such TOF scanners are expensive, bulky, slow, vibration sensitive and therefore only suited for restricted application fields. In this dissertation an imaging, i.e. non-scanning TOF-camera is introduced, based on an array of demodulation pixels, where each pixel can measure both the background intensity and the individual arrival time of an RF-modulated (20 MHz) scene illumination with an accuracy of a few hundreds of picoseconds (300⋅10 -12 s). The pixel´s working concept is based on the CCD principle ( C harge C oupled D evice), allowing the transportation, storage and accumulation of optically generated charge carriers to defined local sites within the imaging device. This process is extremely fast and essentially loss-free. We call our new, powerful high-functionality pixels demodulation pixels because they extract the target´s distance and reflectivity from the received optical signal. This extracted information is modulated into the active optical signal during the time of propagation of the light (or time of flight) through the observed scene. Each pixel works like an individual high-precision stopwatch, and since its realization is mainly based on CMOS technology this new technique will benefit from the ongoing technology developments in terms of improved time- and hence distance resolution. Thanks to the use of CMOS, all commonly known CMOS APS ( A ctive P ixel S ensor) features ( R egions O f I nterest addressing: ROI, AD conversion, etc.) can be implemented monolithically in the future. The imaging devices have been fabricated in a 2 µm CMOS/CCD process, a slightly modified CMOS process which is available as an inexpensive prototyping service ( M ulti P roject W afer: MPW). This process offers the freedom to implement CCDs with sufficiently good performance for our application, although the performance is inferior to dedicated CCD technologies. We have realized and characterized several different pixel structures and will present these results here. The demodulation pixel with the best fill-factor and demodulation performance has been implemented (1) as a line sensor with 108 pixels and (2) as an image sensor with 64 x 25 pixels. Both devices have been integrated in separate range cameras working with modulated LED illumination and covering a distance range of 7.5 up to 15 meters. For non-cooperative diffusely reflecting targets these cameras achieve centimeter accuracy. With the single exception of the demodulation pixel array itself, only standard electronic and optical components have been used in these range cameras. For a resolution of 5 centimeters, an optical power of 600 fW per pixel is sufficient, assuming an integration time of 50 ms (20 Hz frame rate of 3D images). This low optical power implies that only 0.06 electrons are generated per modulation period (T mod =50 ns at 20 MHz modulation frequency). Furthermore, we present an in-depth analysis of the influences of non-linearities in the electronics, aliasing effects, integration time and modulation functions. Also, an optical power budget and a prediction for the range accuracy is derived as a function of the ratio of active illumination to background illumination. The validity of this equation is confirmed by both computer simulations and experimental measurements with real devices. Thus, we are able to predict the range accuracy for given integration time, optical power, target distance and reflectance. With this work we demonstrate the first successful realization of an all-solid-state 3D TOF range-camera without moving parts that is based on a dedicated customized PhotoASIC. The measured performance is very close to the theoretical limits. We clearly demonstrate that optical 3D-TOF is an excellent, cost-effective tool for all modern vision problems, where the relative position or motion of objects need to be monitored.