Jetson Video Processing Programming Episode 8 Image Processing

Narrow down by specifying conditions

現在1888件がヒットしています。check

DesignAI /Artificial Intelligence SoftwareMicrocomputer /Processor/ DSP ModuleNVIDIA

The term “image processing” covers a very wide range. "Image processing" here refers to processing that is widely used for images, not limited to the AI field, such as filtering, feature extraction, and geometric transformation.

[Jetson video processing programming]

Episode 1 What you can do with JetPack and SDK provided by NVIDIA

Episode 2 Video input (CSI-connected Libargus-compliant camera)

Episode 3 Video input (USB-connected V4L2-compliant camera)

Episode 4 Resize and format conversion

Episode 5 Image display

Episode 6 Video encoding

Episode 7 Video decoding

Episode 8 Image Processing

Episode 9 Deep Learning Inference

Episode 10 Maximum Use of Computing Resources

Implementation method

Even for image processing on Jetson, there are multiple choices for the APIs used to implement it. Take a look at the features of each in the table below.

APIs	programming language	comment
NVIDIA Vision Programming Interface (VPI)	C++ Python (Developer Preview at the time of writing this article)	Processing can be distributed to multiple hardware processing units of Jetson (CPU, GPU, PVA, VIC) and optimized for each processing unit
CUDA	C Python is also possible by using open source	Maximum developer optimization and granular control
OpenCV	C Python	multi-platform open

Use of VPI

NVIDIAVision Programming Interface (VPI) is an image processing library that runs on x86 platforms with Jetson and NVIDIA GPU cards. VPI seamlessly accesses multiple compute resources and provides a computer vision and image processing API without sacrificing their processing power. For Jetson, the compute resources VPI can access are CPU, GPU (CUDA), PVA and VIC. The following table shows which computational resources each algorithm provided by VPI corresponds to.

algorithm	CPU	CUDA	PVA ¹	VIC ²	NVENC ²
Box Filter	yes	yes	yes	no	no
Bilateral Filter	yes	yes	no	no	no
Gaussian Filter	yes	yes	yes	no	no
Gaussian Pyramid Generator	yes	yes	yes	no	no
Laplacian Pyramid Generator	yes	yes	no	no	no
Convolution	yes	yes	yes	no	no
Separable Convolution	yes	yes	yes	no	no
Convert Image Format	yes	yes	no	yes	no
Rescale	yes	yes	no	yes	no
Remap	yes	yes	no	yes ¹	no
Perspective Warp	yes	yes	no	yes ¹	no
FFT	yes	yes	no	no	no
Inverse FFT	yes	yes	no	no	no
Lens Distortion Correction	yes	yes	no	yes ¹	no
Stereo Disparity Estimator	yes	yes	yes	no	no
KLT Feature Tracker	yes	yes	yes	no	no
Harris Corner Detector	yes	yes	yes	no	no
Temporal noise reduction	no	yes	no	yes	no
Pyramidal LK Optical Flow	yes	yes	no	no	no
Dense Optical Flow	no	no	no	no	yes ¹
Image Histogram	yes	yes	no	no	no
Equalize Histogram	yes	yes	no	no	no
Background Subtractor	yes	yes	no	no	no

¹ Only available on Jetson Xavier NX and Jetson AGX Xavier series
² Only available on Jetson platform (not supported on GPU card products)

Please refer to the following article for how to use VPI.

"Challenging Geometric Correction of Images with OpenCV ArUco Markers ~ NVIDIA VPI Techniques: Perspective Warp Edition ~"

VPI provides interoperability with OpenCV, as this article demonstrates.

VPI sample program

VPI comes with a sample program. The vpi1_install_samples.sh script prepares a set of sample programs in a user-specified directory. After that, you can create a Makefile with cmake and build the example program with make in the example program's directory.

The following example prepares a set of sample programs in the home directory, builds and executes the Temporal Noise Reduction sample program. The result after noise removal is output as a video file named denoised_cuda.mp4.

Please see the following web page for details.

VPI - Vision Programming Interface Documentation : Sample Applications

jetson@jetson-desktop:~$ vpi1_install_samples.sh $HOME Copying samples to /home/jetson/NVIDIA_VPI-1.1-samples now... Finished copying samples. jetson@jetson-desktop:~$ cd $HOME/NVIDIA_VPI-1.1-samples jetson@jetson-desktop:~/NVIDIA_VPI-1.1-samples$ ls 01-convolve_2d 05-benchmark 09-tnr 13-optflow_dense 02-stereo_disparity 06-klt_tracker 10-perspwarp 14-background_subtractor 03-harris_corners 07-fft 11-fisheye assets 04-rescale 08-cross_aarch64_l4t 12-optflow_lk tutorial_blur jetson@jetson-desktop:~/NVIDIA_VPI-1.1-samples$ cd 09-tnr/ jetson@jetson-desktop:~/NVIDIA_VPI-1.1-samples/09-tnr$ cmake . -- The C compiler identification is GNU 7.5.0 -- The CXX compiler identification is GNU 7.5.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found OpenCV: /usr (found version "4.1.1") -- Configuring done -- Generating done -- Build files have been written to: /home/jetson/NVIDIA_VPI-1.1-samples/09-tnr jetson@jetson-desktop:~/NVIDIA_VPI-1.1-samples/09-tnr$ make Scanning dependencies of target vpi_sample_09_tnr [ 50%] Building CXX object CMakeFiles/vpi_sample_09_tnr.dir/main.cpp.o [100%] Linking CXX executable vpi_sample_09_tnr [100%] Built target vpi_sample_09_tnr jetson@jetson-desktop:~/NVIDIA_VPI-1.1-samples/09-tnr$ ./vpi_sample_09_tnr cuda ../assets/noisy.mp4 NVMEDIA_ARRAY: 53, Version 2.1 NVMEDIA_VPI : 172, Version 2.4 Frame: 1 Frame: 2 Frame: 3 Frame: 4 Frame: 5 ... Frame: 146 Frame: 147 Frame: 148 Frame: 149 Frame: 150 jetson@jetson-desktop:~/NVIDIA_VPI-1.1-samples/09-tnr$ ls CMakeCache.txt cmake_install.cmake denoised_cuda.mp4 main.py vpi_sample_09_tnr CMakeFiles CMakeLists.txt main.cpp Makefile

Using CUDA

User-specific special processing can be realized by user programs using the CUDA Toolkit. For general processing, you can use NVIDIA PERFORMANCE PRIMITIVES (NPP), a CUDA library developed by NVIDIA.

CUDA sample program

Please refer to the following web page for using the CUDA sample program.

CUDA Toolkit Documentation: 2. Getting Started

The image processing examples are in the 3_Imaging directory and the NPP examples are in the 7_CUDALibraries directory.

Using OpenCV

As mentioned repeatedly in this series, the OpenCV included in JetPack from the beginning was built with CUDA disabled. So all OpenCV processing is done on the CPU. If you rebuild OpenCV from the source code with CUDA enabled, you can expect high-speed processing with the CUDA core. (Limited to functions in the cv::cuda namespace.) However, compared to VPI and NPP introduced earlier, OpenCV is not a programming model suitable for CUDA, so if performance is a priority, VPI and NPP are recommended. You can see the performance comparison between OpenCV and VPI with CUDA disabled on the following web page.

VPI - Vision Programming Interface Documentation: Performance Comparison

Next time, I will explain deep learning inference!

How did you like the eighth episode of the series "Jetson Video Processing Programming", which introduced image processing?

Next time, I will introduce deep learning inference.

Click here for Episode 9 Deep Learning Inference

If you have any questions, please feel free to contact us.

We offer selection and support for hardware NVIDIA GPU cards and GPU workstations, as well as facial recognition, route analysis, skeleton detection algorithms, and learning environment construction services. If you have any problems, please feel free to contact us.