Nvidia Jetson Xavier multi camera Artificial Intelligence demo showcase by RidgeRun

This demo from RidgeRun shows the capabilities of the Jetson Xavier by performing :

• Multi-camera capture through FPD-LINK III with Virtual Channels support, 
• Display of each individual camera stream on a grid, 
• Application of CUDA video processing filters, classification and detection inference, 
• Video stabilization processing and video streaming through the network.

RidgeRun demo screen:

RidgeRun & D3 Engineering Nvidia Partner Showcase Jetson Xavier Multi-Camera AI Demo.

Demo components:

D3 Engineering-Nvidia-Xavier FPD-Link III interface card

                                   

D3 Engineering-D3RCM-OV10640-953 Rugged Camera Module

The 8 camera streams are downscaled to 480x480 resolution and displayed on a grid. Following are the extra processing is applied to different camera streams:

Camera_1: No extra processing, just normal camera stream. Intended to be used as a point of comparison against the streams with CUDA video processing filters.

Camera_2: Sobel in X-axis CUDA video filter applied with GstCUDA plugin.

Camera_3: Border Enhancement CUDA video filter applied with GstCUDA plugin.

Camera_4: Grayscale CUDA video filter applied with GstCUDA plugin.

Camera_5: No extra processing, just normal camera stream. Intended to be used as a point of comparison against the stream with video stabilization processing.

Camera_6: Video stabilization processing applied with GstNvStabilize plugin. 

Camera_7: InceptionV1 Classification Inference applied with GstInference plugin using GPU accelerated TensorFlow.

Camera_8: TinyYoloV2 Detection Inference applied with GstInference plugin using GPU accelerated TensorFlow.

One individual camera stream selected by the user from the demo menu is streamed to the network using the GstWebRTC plugin and an OpenWebRTC application.

Demo setup, demo features in detail, demo code and performance profiling information are explained in this RidgeRun & D3 Engineering - Nvidia Partner Showcase : Jetson Xavier Multi-Camera AI Demo RidgeRun Developer Wiki.

Contact Us

Please visit our Main Website for the RidgeRun online store or Contact Us for pricing information of the engineering support, product and services.                                                                              

You can also send an email to support@ridgerun.com for a technical support, more information about the features, evaluation version (if available) or for a details about how to sponsor a new feature.

GstCUDA: RidgeRun presentation at NVIDIA GTC 2019 on GStreamer and CUDA integration.

RidgeRun engineers presented a GstCUDA, a framework developed by RidgeRun that provides an easy, flexible and powerful integration between GStreamer audio/video streaming infrastructure and CUDA hardware-accelerated video processing at NVIDIA GTC 2019.

GstCUDA: Easy GStreamer and CUDA Integration

Please Watch the Video

For more information please contact us at support@ridgerun.com or for purchase related questions post your inquiry at our Contact Us page.

RidgeRun at GTC 2019 as a NVIDIA Jetson partner

RidgeRun is excited about the new things coming for the NVIDIA Jetson partners and our team will be at  GTC 2019!!

RidgeRun Engineering Manager with Jensen Huang CEO NVIDIA

For more information please contact us at support@ridgerun.com or for purchase related questions post your inquiry at our Contact Us page.

RidgeRun support "Armstrong" Robot at The 2019 FIRST® Robotics competition by providing NVIDIA Jetson support

FIRST Robotics Competition [1] Team 102, The Gearheads, from Somerville High School in NJ, demonstrating the "Armstrong" - The robot for the 2019 season. They hope that the combination of Nvidia TX1, Auvidea J90LC, and drivers from RidgeRun will provide a cost effective, state of the art Computer Vision solution.  Go Gearheads!’

[1] https://www.firstinspires.org/robotics/frc

RidgeRun help getting the kernel built with the IMX219 V4L2 driver for TX1 on the Auvidea J90-LC for state of the art Computer Vision solution for The 2019 FIRST® 2019 season.

Pictures from the FIRST Robotics Competition at Bridgewater, NJ on March 16, 2019.

"Armstrong" - The robot.

Please note the Ridgerun graphic on the "Armstrong" - The robot.

"Armstrong" - The robot in action.

More close-up Ridgerun graphic on the "Armstrong" - The robot

Team 102, The Gearheads, qualified for the finals out of 35, went on to tie in the semi finals and ultimately finished very favorably.

Next competition is the Greater Pittsburgh Regional in PA on Mar 20 - 23.

http://pittsburghfirst.org/2019-greater-pittsburgh-regional/

Watch out! and we will keep updating as the competition progress.

For more information please contact us at support@ridgerun.com or for purchase related questions post your inquiry at our Contact Us page. 

GstInference - Bringing AI to GStreamer

AI is everywhere. Well, maybe this is not entirely true yet, but will be in a few years from now. I heard a prediction that 1,000,000,000 video cameras will be installed by 2020. At RidgeRun we've been working hard to join this revolution, and we want to share some of the technology we're developing with the community.

The truth is that the initial face-off with AI is never easy. A quick Google search reveals a handful of frameworks available to implement a neural network. For the sake of this example suppose you choose TensorFlow. You grasp the whole graph oriented processing paradigm, and finally understand and successfully run the image classification example you are trying out. Cool! This is exactly what your product needs to do. Almost.

Now to fit all the pieces together. Here's a small list of the most immediate tasks for your project:

• Modify the example to receive images from the camera, rather than PNG images.
• Scale down the 1920x1080 camera images to 224x224 needed by the neural network.
• Render a label in the image according to the network prediction.
• Save resulting images in a MP4 file.

If you are familiar with GStreamer, you immediately envisioned a nice, simple pipeline. Something like the following:

Putting it this way doesn't feel that overwhelming after all.

That's exactly where GstInference fits in this whole scenario. It's a set of GStreamer elements that execute AI models on a video stream. We've taken care of all the complexities and boilerplate code so that all the framework details are hidden, and you just link in the inference element as any other. And not only TensorFlow, but Caffe, NCSDK, TensorRT, and others as well (yet to come).

Here's how the whole pipeline looks altogether:

Note how the modular architecture of GStreamer allows you to make the most of the available hardware accelerators in your platform. The scaling could be done using the Image Signal Processor, the encoding using the Video Processing Unit and the inference using the GPU. Let's take it one step further: using the exact same pipeline you can migrate the inference processing from the GPU to the Tensor Processing Unit by simply changing the backend property in GstInference from TensorFlow to TensorRT, for example.

RidgeRun's main design goal is simplicity for the user.

We have made an early release available. Feel free to give it a try and give us your feedback. We'd love to hear from you!

https://developer.ridgerun.com/wiki/index.php?title=GstInference
https://github.com/RidgeRun/gst-inference/

RidgeRun's Sony IMX219 CMOS Image Sensor Linux Driver for NVidia Jetson Xavier and Jetson TX1/TX2

This blog highlights the RidgeRun support for Jetson Xavier and Jetson Tegra platform on developing a CMOS Image Sensor Linux Driver for Sony IMX219.

Driver Features:

• L4T 31.1 and Jetpack 4.1
• V4l2 Media controller driver
• One camera capturing (TODO: to expand to 6 cameras)
• Tested resolution 3280 x 2464 @ 15 fps
• Tested resolution 720p @ 78 fps
• Tested resolution 1640x1232 @ 30 fps
• Tested resolution 820x616 @ 30 fps
• Tested with J20 Auvidea board.
• Capture with v4l2src and also with nvarguscamerasrc using the ISP.

Images attached here are taken during the developing driver for Sony IMX219 image sensor, testing various image capture and display options, performance and latency measurement in our R&D lab located in CostaRica. Various tests are carried out using GStreamer pipelines. 

Enabling and building the driver with Auvidea J20 Expansion board, Example GStreamer pipelines, Performance, Latency measurement details are shared in the RidgeRun developer wiki's mentioned at the end of this blog.

RidgeRun's Sony IMX219 Linux driver for Jetson Xavier

Output image of the IMX219 camera sensor image capture @1640x1232 resolution with nvcamerasrc GStreamer element for the Jetson Xavier platform:

RidgeRun's Sony IMX219 Linux driver latency measurements on Jetson Xavier

Frames showing how the glass to glass latency measurement method is setup by our team while testing for image capture using IMX219 camera mode at 3280x2464@16fps resolution for Xavier platform.

 

Glass to glass latency measured is 215 ms (07:772 minus 07:557).Time readings can be seen in the displays.

RidgeRun's Sony IMX219 Linux driver for Jetson TX1

Image below is showing the IMX219 capture @1640x1232 resolution with nvcamerasrc on the Jetson TX1 platform. Camera aimed at computer monitor on the left which is reflecting the wall and ceiling shown on the right:

RidgeRun's Sony IMX219 Linux driver latency measurements on Jetson TX1

Image below is captured while measuring the Jetson TX1 glass to glass latency for 1080p 30fps IMX219 camera mode:

 

Glass to glass latency measured is 130 ms ((13.586 minus 13.456).Time readings can be seen in the displays.

You can find more information about the driver in these developer wiki's from RidgeRun : 

Sony IMX219 Linux Driver for Jetson Xavier

Sony IMX219 Linux driver for Jetson TX1

For technical information, please email to us at support@ridgerun.com or for purchase related questions post your inquiry at our Contact Us page.

RidgeRun's USB Video Class Gadget Library - LibGUVC v1.4.0 - NVidia Xavier and NXP iMX Support

The UVC Video Class Gadget Library or libguvc for short is a platform agnostic library that simplifies the development of UVC based gadget devices by encapsulating the most of the UVC communication leaving just the basic setup to the user. The USB video class gadget runs on top of the UVC function driver in the user space and takes care of the communication between the user application and the linux driver stack.

Release of libGUVC v1.4.0, is now supporting NVIDIA Jetson platform along with previously supported NXP-iMX6 family of processors, thanks to the new bulk transfer support.

It has never been easier to implement a UVC application on your hardware, libGUVC make it easy to interact with the UVC driver and expose a variety of useful features such as:

•     USB 2.0 and USB 3.0 support
•     Isochronous and bulk endpoint support
•     YUV2, MJPEG and H264 video streaming support
•     Extension Unit support
•     MMAP and UserPtr support.

Since the driver is agnostic to the platform, you can run it on almost any platform with quality USB and UVC drivers, making it really simple to convert it into a UVC capable device.

 libGuvc in action - Running libGuvc on NVidia Xavier

 

 libGuvc in action - Running libGuvc on NXP-iMX6

 

You can find more information about the library in this developer wiki from RidgeRun Engineering : USB Video Class Gadget Library - libguvc

For more technical information please contact us at support@ridgerun.com. Please post your inquiry at our Contact Us page for purchase related questions and also since the libguvc is platform agnostic, you can request for a custom demo image for any other platform using this link.