Birds Eye View on NVIDIA Jetson Boards

A Bird's Eye View (BEV) is an elevated view of an object from above, with a perspective as though the observer were a bird. It is often used in the making of blueprints, floor plans, maps and car parking systems. The result is obtained with a technique that generates an aerial view of a scene based on various lateral perspectives. Several cameras are used to obtain those lateral views, usually at least 4, to get a 360 view around an object. 

The project experiments with the notion of Bird’s Eye in Desktop PC and NVIDIA Jetson platforms.

To get a good field of view with the minimum amount of cameras, for example 180 degrees per camera, the image will be generated with a fisheye perspective. The RidgeRun’s BEV engine is able to remove the input distortion and using a preconfigured settings file generate an output image. This may be appreciated on Figure 1.

Figure 1. Example of a BEV image generated from 4 lateral fisheye cameras.

The process is composed of 6 smaller steps. First, the buffers are obtained from the camera. Next the fisheye distortion is removed. The third and fourth steps generate the elevated perspective with the technique known as IPM (Inverse Perspective Mapping). Fifth and sixth steps adjust the image to obtain the desired output section for each camera by applying an enlargement to match the perspective with the other cameras and by cropping the relevant section of the image. A summary of the process is depicted in Figure 2.

Figure 2. Simplified process to generate the BEV

The C++ library was developed in a modular way, open to include new image processing frameworks, but was initially tested with the help of OpenCV.

Version 0.2.0 includes:

• Fisheye and rectilinear test data sets (synthetic and real)
• Jupyter notebooks with detailed prototypes
• Fisheye correction and BEV computation
• C++ library with OpenCV based implementation
• CUDA acceleration
• Extensive set of examples

Some areas under development:

• Image Stitching integration
• GStreamer support

Learn more in our developers wiki or check out the API reference.

Contact us to be part of this research!