Lokalisation und Verfolgung von Personen in Echtzeit unter Verwendung kooperierender 2D/3D-Kameras

The fundamental objective of intelligent systems targeted towards surveillance of indoor areas lies
in the automated interpretation of actions within the monitored space. One premise therefore is the
ability to detect, localize, track and classify objects within the surveyed area
In terms of system design, it’s also crucial to incorporate the targeted application area. For the case
of surveillance of just relatively small volumes and with the additional assumption that only very
few objects are simultaneously present within the detection area, data from a single sensor might
be sufficient in order to infer adequate information. Increasing the area to be monitored and also
allowing multiple objects to dynamically interact in the space to be surveyed, usually leads to the
necessity to apply multiple distributed sensor systems which, in order to generate useful
information, have to work together in a cooperative manner.
One in recent times increasingly adopted approach in order to increase the performance of
detection, localization and classification is to utilize a combination of devices delivering 2D- and
3D-data. Depending on the requirement, fusion of the information is then done on pixel-, feature or decision-level. The combination of different systems in a 2D/3D-measurement device is not
trivial, though. In case of taking moving objects into account, there’s a necessity to provide an
adequate temporal synchronisation mechanism between the participating devices. With regard to
an effective fusion process, the process of registration of the acquired data needs also to be
applied. Furthermore, by combining multiple of such 2D/3D-devices into one all encompassing
system, the additional need for a coordinated management emerges. Having to cope with all of the
aforementioned points might well lead to a system complexity which is intricate to handle and
which in turn might result in an unacceptable overall system performance.
The at the Center for Sensorsystems (ZESS) developed MultiCam, an at the moment unique sensor
system usable to acquire a combination of CMOS-based intensity (2D) and Time-of-Flight (ToF)
based distance information (3D) of a complete scene. Due to the monocular set-up, a drastic
simplification of the process of image registration of the 2D- and 3D-images is attained.
Furthermore, the internal logic, situated in an embedded FPGA, permits an accurate temporal
synchronisation of the different information streams. Due to the reduction of the overall
complexity of the processes of registration and temporal synchronisation of the 2D- and 3D-data, it
is feasible to combine a multitude of simultaneously operating MultiCams to a surveillance system
with, in comparison with conventional 2D/3D-systems, a drastic simplification in terms of
handling.
The additional application of the also at the ZESS developed dedicated MultiCam illumination
units enables to supervise an area up to a distance of maximal 9mat a camera frame rate of 20fps.
With similar ToF devices, the same distance range can only be achieved by an increase of the
sensor integration times and which in turn directly leads to a reduced frame rate. Furthermore, as a
result of the utilization of a lens with a focal length of 6mm, it is possible to monitor volumes in
which multiple persons can simultaneously be present and also interact with each other. Due to the
available temporal resolution, one can account for the dynamic behaviour of the objects
effectively. As a results of all of the aforementioned properties, the MultiCam is ideally suited for
the detection, localization, tracking and classification of objects in indoor areas.
Through the combination of multiple simultaneously operating MultiCams with different
orientations and partially overlapping field of views, it is possible to directly extend the surveyed
area and also reduce the amount of mutual object occlusion. This directly leads to the generation of
object trajectories with an increased information content.
In order to respond to the amount of data generated by the simultaneous operation of multiple
devices, an agent based system is utilized. Information delivering agents, composed of one
MultiCam with dedicated processing unit, are responsible for the direct processing of the acquired
raw sensor data. Based on their view onto the scene, they generate information in form of a local
status vector which they send to a dedicated fusion agent. This fusion agent is in turn responsible
for the combination of all of the local information received into one global status vector. Due to the mutual independence of the participating agents, parallel processing
of the acquired data is inherently possible which in turn results in the possibility to be able to scale
the system in an easy manner.