NVIDIA Jetson is now indispensable for applications that require video processing. NVIDIA has a wide range of software such as DeepStream SDK and NVIDIA Isaac SDK, as well as peripheral tools optimized for NVIDIA GPUs. We recommend that you However, depending on the application to be developed, there may be cases where a suitable SDK cannot be found or these SDKs are not suitable due to special requirements. In that case, it is necessary to develop an application that directly calls the API of the library included in JetPack.
In this series of articles, we assume that the system will process the image information captured from the camera with the GPU in the Jetson module and display the results. I will explain. In Part 1, let's consider the software configuration of the system built on Jetson.
[Jetson video processing programming]
Episode 1 What you can do with JetPack and SDK provided by NVIDIA
Episode 2 Video input (CSI-connected Libargus-compliant camera)
Episode 3 Video input (USB-connected V4L2-compliant camera)
Episode 4 Resize and format conversion
Episode 5 Image display
Episode 6 Video encoding
Episode 7 Video decoding
Episode 8 Image Processing
Episode 9 Deep Learning Inference
Episode 10 Maximum Use of Computing Resources
Jetson Software Development Kit
NVIDIA provides JetPack as a software development kit (SDK) for Jetson. By introducing JetPack into the Jetson developer kit, you can immediately start developing application software for Jetson. JetPack includes the following software: These software are general-purpose ones that do not specifically select the field of application.
|
Operating system (L4T)
|
|
|
|
|
|
|
|
|
|
|
computer vision
|
|
|
development tools
(installed on host PC) |
|
Use of application domain-specific SDKs
More application-specific SDKs can be deployed on top of JetPack. Typical examples are as follows. These SDKs are sometimes called "frameworks," "toolkits," "platforms," etc. If you can use an SDK that is specific to your application domain, you may be able to build your application with less programming effort. On the other hand, fine control may be difficult. It is a trade-off between taking programming man-hours and taking the ease of creating application-specific parts. Also, if the SDK is difficult to use, it may cost a lot to learn, so be careful.
|
SDKs |
Usage |
Features |
|
Intelligent video analytics
|
Provided as part of NVIDIA Metropolis and can be used in combination with pre-trained models and Transfer Learning Toolkit, a tool for transfer learning
|
|
|
AI-powered robot
|
A simulator is also provided.
|
|
|
Open SDK
ROS etc. |
Different for each SDK
|
Enhancing the ecosystem
|
|
Jetson-compatible SDKs sold by third parties
|
Different for each SDK
|
Each third party has its own characteristics
|
If you can't find an application-specific SDK or it isn't right for you
As mentioned at the beginning, basically we recommend using NVIDIA DeepStream SDK or NVIDIA Isaac SDK, but if you know how to directly call the API of the library included in JetPack, future application development It is possible to help
The assumed video processing steps and the libraries/APIs to be used are as follows.
|
process |
Libraries/APIs used |
|
|
|
|
|
|
|
|
|
video encoding
|
|
|
video decoding
|
|
|
Image processing
|
|
|
deep learning inference
|
|
Next time, I will explain the "video input" method that should be considered first in a video processing system!
How was the first episode of the series "Jetson Video Processing Programming", introducing what you can do with JetPack and SDK provided by NVIDIA?
Next time, I will introduce how to input video.
If you have any questions, please feel free to contact us.
We offer selection and support for hardware NVIDIA GPU cards and GPU workstations, as well as facial recognition, route analysis, skeleton detection algorithms, and learning environment construction services. If you have any problems, please feel free to contact us.
Latest Information
Case Study
Blog
Document List