NVIDIA Jetson TX2-Camera image capture latency measurement techniques.

For better understanding of this latest blog from RidgeRun, please read the RidgeRun blog on Diving deep into NVIDIA Jetson TX2 - Video Input system and camera image capture latency.

The techniques used to measure the Jetson TX2 image capture latency are outlined in this blog.

Two timestamps are used to measure the latency :

SOF : The first timestamp is taken when the pixels of a frame start to arrive at the VI (t0), this is the start-of-frame (SOF).
The second timestamp is usually obtained at the time when the frame is available on a userspace application (t1).

Techniques can be divided into two main categories, techniques that use the CHANSEL_PXL_SOF (referred to as SOF) and the techniques that do not use the SOF .

Outline of both the methods:

Techniques that use the SOF timestamp

CHANSEL_PXL_SOF can be used as t0 and then, t1 can be the timestamp obtained at the instant when the frame arrives to userspace using ideally the same clock used to set the CHANSEL_PXL_SOF, then the latency can be computed as follows:
Latency = t1 - t0

Techniques that do not use the SOF timestamp

LED test
This test involves the use of a led connected to the TX2 GPIO.
The camera must start capturing when the led is still off, and then a kernel module must turn the led on and then record the current CLOCK_MONOTONIC timestamp to be used as t0. Each image that arrives to userspace must be timestamped also with the CLOCK_MONOTONIC at the instant when it's available, this will be t1.

More technical details about the techniques, working with TSC (Time Stamping System Clock) for the latency measurements and rtcpu, v4l2 tracing details are explained in the RidgeRun developer wiki page : NVIDIA Jetson TX2 - VI Latency Measurement Techniques

Contact Us

Please visit our Main Website for the RidgeRun online store or Contact Us for pricing information of the engineering support, product and Professional Services.
Please email to support@ridgerun.com for technical questions and for an evaluation version (if available).
Contact details for sponsoring the RidgeRun GStreamer projects are available at Sponsor Projects page.

NVIDIA Jetson Xavier Multi-Camera Artificial Intelligence Demo from RidgeRun

Jetson Xavier Multi-Camera Artificial Intelligence Demo from RidgeRun

This demo from RidgeRun shows the capabilities of the Jetson Xavier by performing :

• Multi-camera capture through FPD-LINK III with Virtual Channels support,
• Display of each individual camera stream on a grid, 
• Application of CUDA video processing filters, classification and detection inference, 
• Video stabilization processing and video streaming through the network.

Please watch Jetson Xavier Multicamera + AI + Video Stabilization + CUDA Video Processing FIlters demo from RidgeRun :

Demo components:

D3 Engineering-Nvidia-Xavier FPD-Link III interface card

                                   

D3 Engineering-D3RCM-OV10640-953 Rugged Camera Module

The 8 camera streams are downscaled to 480x480 resolution and displayed on a grid. Following are the extra processing is applied to different camera streams:

Camera_1: No extra processing, just normal camera stream. Intended to be used as a point of comparison against the streams with CUDA video processing filters.

Camera_2: Sobel in X-axis CUDA video filter applied with GstCUDA plugin.

Camera_3: Border Enhancement CUDA video filter applied with GstCUDA plugin.

Camera_4: Grayscale CUDA video filter applied with GstCUDA plugin.

Camera_5: No extra processing, just normal camera stream. Intended to be used as a point of comparison against the stream with video stabilization processing.

Camera_6: Video stabilization processing applied with GstNvStabilize plugin. 

Camera_7: InceptionV1 Classification Inference applied with GstInference plugin using GPU accelerated TensorFlow.

Camera_8: TinyYoloV2 Detection Inference applied with GstInference plugin using GPU accelerated TensorFlow.

One individual camera stream selected by the user from the demo menu is streamed to the network using the GstWebRTC plugin and an OpenWebRTC application.

Demo setup, demo features in detail, demo code and performance profiling information are explained in this RidgeRun & D3 Engineering - Nvidia Partner Showcase : Jetson Xavier Multi-Camera AI Demo RidgeRun Developer Wiki.

Contact Us

Please visit our Main Website for the RidgeRun online store or Contact Us for pricing information of the engineering support, product and services.                                                                              

You can also send an email to support@ridgerun.com for a technical support, more information about the features, evaluation version (if available) or for a details about how to sponsor a new feature.

Nvidia Jetson Xavier multi camera Artificial Intelligence demo showcase by RidgeRun

This demo from RidgeRun shows the capabilities of the Jetson Xavier by performing :

• Multi-camera capture through FPD-LINK III with Virtual Channels support, 
• Display of each individual camera stream on a grid, 
• Application of CUDA video processing filters, classification and detection inference, 
• Video stabilization processing and video streaming through the network.

RidgeRun demo screen:

RidgeRun & D3 Engineering Nvidia Partner Showcase Jetson Xavier Multi-Camera AI Demo.

Demo components:

D3 Engineering-Nvidia-Xavier FPD-Link III interface card

                                   

D3 Engineering-D3RCM-OV10640-953 Rugged Camera Module

The 8 camera streams are downscaled to 480x480 resolution and displayed on a grid. Following are the extra processing is applied to different camera streams:

Camera_1: No extra processing, just normal camera stream. Intended to be used as a point of comparison against the streams with CUDA video processing filters.

Camera_2: Sobel in X-axis CUDA video filter applied with GstCUDA plugin.

Camera_3: Border Enhancement CUDA video filter applied with GstCUDA plugin.

Camera_4: Grayscale CUDA video filter applied with GstCUDA plugin.

Camera_5: No extra processing, just normal camera stream. Intended to be used as a point of comparison against the stream with video stabilization processing.

Camera_6: Video stabilization processing applied with GstNvStabilize plugin. 

Camera_7: InceptionV1 Classification Inference applied with GstInference plugin using GPU accelerated TensorFlow.

Camera_8: TinyYoloV2 Detection Inference applied with GstInference plugin using GPU accelerated TensorFlow.

One individual camera stream selected by the user from the demo menu is streamed to the network using the GstWebRTC plugin and an OpenWebRTC application.

Demo setup, demo features in detail, demo code and performance profiling information are explained in this RidgeRun & D3 Engineering - Nvidia Partner Showcase : Jetson Xavier Multi-Camera AI Demo RidgeRun Developer Wiki.

Contact Us

Please visit our Main Website for the RidgeRun online store or Contact Us for pricing information of the engineering support, product and services.                                                                              

You can also send an email to support@ridgerun.com for a technical support, more information about the features, evaluation version (if available) or for a details about how to sponsor a new feature.