MUNICH, Germany — The development of CMOS image sensors and related algorithms and DSP chips, now so common in mobile phones, is about to have a dramatic effect on the automobile. With technologies honed in the mobile sector, camera-based driver assistance systems are set to become a hub for operational video, radar, and lidar data streams – effectively forming the brains of future cars.

Cameras are already being deployed in ever larger numbers. They are used for road sign recognition, they monitor the driver’s face for indications of fatigue, and they help drivers to back up and park. However, the combination of multiple linked CMOS image sensors and other sensors along with computing also enables new applications.

In order to understand the different types of camera-based assistance systems, it is important to discriminate their basic approach, said Thomas Kropf, vice president of engineering for Robert Bosch GmbH (Stuttgart, Germany).

Imaging systems work with either infrared or visible light cameras and provide their output signals to a dashboard screen. Supporting the driver in recognizing obstacles at night or during poor visibility conditions, they perform similarly to video or security cameras.

Nevertheless, their signals require some processing such as contrast improvement or noise reduction. This, in turn, requires dedicated processing. "Basically, this is what photographers do when they use their PC software, but car-based imaging systems have to do it 25 times per second," said Kropf. Non-imaging systems also process data provided by cameras, but they do not display them. Instead, they combine the input of up to six cameras to generate a full virtual image of the car and its situation, including blind spots.

Non-imaging systems include driver assistance systems such as lane departure warning or traffic sign recognition. Their output can be an acoustical signal or an optical warning light on the dashboard. "While, of course, the OEMs specify which type of signal to use, we tend to think that haptic [tactile] signals are preferable since they achieve the highest degree of driver alertness and the most intuitive form of a warning," said Kropf.

While getting such systems to work on a test track is relatively easy, engineering them for reliability under all conditions is tough. "For instance, a lane departure warning system has to be able to identify road works or turn-off lanes. In particular, during night, this is a non-trivial challenge to algorithms and hardware," said Kropf.