NVIDIA Jetson TX2-Camera image capture latency measurement techniques.

For better understanding of this latest blog from RidgeRun, please read the RidgeRun blog on Diving deep into NVIDIA Jetson TX2 - Video Input system and camera image capture latency.

The techniques used to measure the Jetson TX2 image capture latency are outlined in this blog.

Two timestamps are used to measure the latency :

SOF : The first timestamp is taken when the pixels of a frame start to arrive at the VI (t0), this is the start-of-frame (SOF).
The second timestamp is usually obtained at the time when the frame is available on a userspace application (t1).

Techniques can be divided into two main categories, techniques that use the CHANSEL_PXL_SOF (referred to as SOF) and the techniques that do not use the SOF .

Outline of both the methods:

Techniques that use the SOF timestamp

CHANSEL_PXL_SOF can be used as t0 and then, t1 can be the timestamp obtained at the instant when the frame arrives to userspace using ideally the same clock used to set the CHANSEL_PXL_SOF, then the latency can be computed as follows:
Latency = t1 - t0

Techniques that do not use the SOF timestamp

LED test
This test involves the use of a led connected to the TX2 GPIO.
The camera must start capturing when the led is still off, and then a kernel module must turn the led on and then record the current CLOCK_MONOTONIC timestamp to be used as t0. Each image that arrives to userspace must be timestamped also with the CLOCK_MONOTONIC at the instant when it's available, this will be t1.

More technical details about the techniques, working with TSC (Time Stamping System Clock) for the latency measurements and rtcpu, v4l2 tracing details are explained in the RidgeRun developer wiki page : NVIDIA Jetson TX2 - VI Latency Measurement Techniques

Contact Us

Please visit our Main Website for the RidgeRun online store or Contact Us for pricing information of the engineering support, product and Professional Services.
Please email to support@ridgerun.com for technical questions and for an evaluation version (if available).
Contact details for sponsoring the RidgeRun GStreamer projects are available at Sponsor Projects page.

GStreamer Video Stabilizer for NVIDIA Jetson Boards

Many applications require the removal of undesired camera movement. Professional video stitching, medical imaging such as colonoscopy or endoscopy and localization of unmanned vehicles are a few examples of use cases that benefit from video stabilization. Unfortunately, this is a very resource consuming technique that may be unfeasible for real time operations on resource constrained systems such as embedded systems.

The following video provides a hands-on overview of GstNvStabilize on the works!

GstNvStabilize is GStreamer based video stabilizer for NVIDIA Jetson boards. It's based on VisionWorks and OpenVX hardware processing units to accelerate the stabilization for real time applications.

Latest v0.4.0 release include:

- Region-of-interest configuration via GStreamer caps - Smoothing level configuration via GStreamer property - Smart compensation limit to avoid black borders - GPU acceleration - Supported platforms: - NVIDIA Jetson Xavier - NVIDIA Jetson TX1/TX2 - NVIDIA Jetson Nano

Learn more in our developer's wiki:
https://developer.ridgerun.com/wiki/index.php?title=GStreamer_Video_Stabilizer_for_NVIDIA_Jetson_
Boards

Purchase directly from our website:
https://shop.ridgerun.com/products/gstnvstabilize?_pos=1&_sid=0951b9cf7&_ss=r 

Contact Us

Please visit our Main Website for the RidgeRun online store or Contact Us for pricing information of 
the engineering support, product and services. 
You can also send an email to support@ridgerun.com for a technical support, more information about 
the features, evaluation version (if available) or for a details about how to sponsor a new feature.

GstCUDA: RidgeRun presentation at NVIDIA GTC 2019 on GStreamer and CUDA integration.

RidgeRun engineers presented a GstCUDA, a framework developed by RidgeRun that provides an easy, flexible and powerful integration between GStreamer audio/video streaming infrastructure and CUDA hardware-accelerated video processing at NVIDIA GTC 2019.

GstCUDA: Easy GStreamer and CUDA Integration

Please Watch the Video

For more information please contact us at support@ridgerun.com or for purchase related questions post your inquiry at our Contact Us page.

RidgeRun supporting The 2019 FIRST® Robotics by providing NVIDIA Jetson support

RidgeRun love helping academic projects! RidgeRun is proud to help the teams on FIRST Robotics Competition [1] with software for embedded systems to improve the acquisition, processing and analysis of Audio and Video signals!

[1] https://www.firstinspires.org/robotics/frc

Michael, Amanda, and Emily (in the pic below) of FIRST Robotics Competition Team 102, The Gearheads, from Somerville High School in NJ, investigate the NVIDIA Jetson technology from RidgeRun.  They hope that the combination of Nvidia TX1, Auvidea J90LC, and drivers from RidgeRun will provide a cost effective, state of the art Computer Vision solution for the 2019 season. Go Gearheads!’

RidgeRun help getting the kernel built with the IMX219 V4L2 driver for TX1 on the Auvidea J90-LC for state of the art Computer Vision solution for The 2019 FIRST® 2019 season.

Watch out! and we will keep updating as the Competition progress.

For more information please contact us at support@ridgerun.com or for purchase related questions post your inquiry at our Contact Us page.

Testing Deep Reinforcement Learning on the Jetson Xavier with PyTorch

Jetson-reinforcement is a training guide, provided by NVIDIA, for deep reinforcement learning on the TX1 and TX2 using PyTorch. The tutorial is not currently officially supported on the Jetson Xavier. We provide instructions to get the Deep Q Learning 'cartpole' demo running on the Xavier.

The objective of this example is to balance a pole that is attached by an un-actuated joint to a cart, which moves along a friction-less track. Deep Q Learning solves the problem by generating actions based just on pictures of the environment and the received reward.

If you want to test this demo on your Xavier please visit our  jetson-reinforcement wiki page.

If you are new to the Xavier or are planning on getting one please visit our Jetson Xavier wiki page.

Working with CUDA on the Jetson Xavier

A lot of CUDA samples are included , one of these samples is imageDenoising. This sample demonstrates two adaptive image denoising techniques: KNN and NLM, based on computation of both geometric and color distance between texel

Check out the samples included with CUDA and what they do in CUDA Samples.

If you are new to the Xavier or are planning on getting one please visit our Jetson Xavier wiki page.

Tuning Jetson Xavier's Performance

The JetPack provides the tegrastats utility program which reports memory, processor and gpu usage, power consumption and temperature for Tegra-based devices.




The JetPack also provides with a command line tool called nvpmodel which can modify the performance for a given power budget. It provides power budgets for 10W, 15W, 30W and a no-budget mode for max performance. This will modify number of CPUs online, maximum frequency for CPU, GPU, DLA, PVA and number of online PVA cores. Values set by nvpmodel will persist across power cycles.


Finally the jetson_clocks.sh script provides the best performance for the current nvpmodel by setting the clock frequencies to the max frequency and disabling dynamic frequency scaling.


For examples on how to use this utilities please visit performance tuning wiki page.

If you are new to the Xavier or are planning on getting one please visit  Jetson Xavier wiki page.

Deploying Deep Learning on the Jetson Xavier using the Deep Learning Accelerator

jetson-inference is a training guide for inference and deep learning on Jetson platforms. It uses NVIDIA TensorRT for efficiently deploying neural networks.The "dev" branch on the repository is specifically oriented for Jetson Xavier since it uses the Deep Learning Accelerator (DLA) integration with TensorRT 5.

With jetson-inference you can deploy deep learning examples on the Xavier in a matter of minutes. Some of the example applications are showed below.

ImageNet is a classification network trained with a database of 1000 objects. The input is an image and it outputs the most likely class and the probability that the image belongs to that class.

Image recognition networks output a class probabilities corresponding to the entire input image. Detection networks, on the other hand, find where in the image those objects are located. DetectNet accepts an input image, and outputs the class and coordinates of the detected bounding boxes.

For more examples and a tutorial on how to get jetson-inference running in your Xavier please visit our jetson-inference wiki page.

If you are new to the Xavier or are planning on getting one please visit our Jetson Xavier wiki page.

Object tracking in Jetson TX1/TX2 using GStPTZR

RidgeRun's new GStPTZR element allows to crop, zoom and rotate a video stream, simulating the behavior of a pan/tilt/zoom/rotate PTZR video camera.

These features, paired with information obtained from a jetson-inference pre-built model, can be used to provide a video stream focused on the detected object.

Captured video (left) is provided to a jetson-inferencce model. The model detects a person and provides the location. GstPTZR is used to crop the area of interest as a separate stream.

Using the GstPTZR element in an already-existing GStreamer pipeline is easy, and can provide a simple way to focus on the important parts of the video stream. 

Captured video (left) and the cropped version obtained with GStPTZR (right) that allow for detection of an object on a specific area of the video.

RidgeRun's GStPTZR is highly customizable and can be used for a wide variety of applications, both paired with detection models and other specific use cases.

For more information, visit www.ridgerun.com/gstptzr and contact us at support@ridgerun.com to request an evaluation version for your application.

The examples in the pictures above were created using models from the Jetson Inference guide.