Jetson Video Processing Programming Episode 4 Resizing and Format Conversion

Narrow down by specifying conditions

現在1888件がヒットしています。check

Design AI/Artificial IntelligenceNVIDIA

As preprocessing for image processing, resizing (enlarging/reducing) and format conversion are common processes. Therefore, if these processes are implemented inefficiently, they will consume computational resources unexpectedly and throughput will not increase. For the fourth article in the series, let's consider how to implement resizing and format conversion with Jetson.

[Jetson video processing programming]

Episode 1 What you can do with JetPack and SDK provided by NVIDIA

Episode 2 Video input (CSI-connected Libargus-compliant camera)

Episode 3 Video input (USB-connected V4L2-compliant camera)

Episode 4 Resize and format conversion

Episode 5 Image display

Episode 6 Video encoding

Episode 7 Video decoding

Episode 8 Image Processing

Episode 9 Deep Learning Inference

Episode 10 Maximum Use of Computing Resources

Comparison of implementation methods

Both resizing and format conversion can be implemented in multiple ways. Below is a comparison of them.

APIs	programming language	Strong Points	Cons/Challenges
GStreamer	C Python	clear interface Efficient due to pipelining multi-platform	Difficult to use for resizing and format conversion only (all image processing should be done with GStreamer) GStreamer-specific concepts take some getting used to
OpenCV	C Python	clear interface Abundant reference information for mastering multi-platform	The ones included in JetPack are built with CUDA (GPU) disabled Even if you build with CUDA enabled, using CUDA is not efficient compared to other methods.
VPI	C++ Python will also be supported	Can use multiple backends (CPU/CUDA/PVA/VIC) High performance with Jetson optimized implementation It is also considered to be used with OpenCV Balanced usability and performance	Lack of reference information
Multimedia API	C++	High performance due to low-level API close to hardware Granular control	Lack of reference information a bit esoteric
NVIDIA Performance Primitives (NPPs)	C	Lowest level API, high performance Granular control	CUDA programming itself can be used without need, but it is necessary to build applications with CUDA core (GPU) in mind

When using GStreamer

It can be implemented with the nvvidconv plugin. An example is shown below. This example can be run on the command line, but should be programmed using the GStreamer API (C or Python) when embedded in an application.

Note: The output format (YUYV) of the webcam used for testing (Logitech C270n HD webcam) does not support the input (sink) format to the nvvidconv plugin, so the videoconvert plugin (In this example, it is converted to NV12).

$ gst-launch-1.0 -e \ v4l2src device="/dev/video0" ! \ "video/x-raw, width=(int)640, height=(int)480" ! \ videoconvert ! \ "video/x-raw, format=(string)NV12" ! \ nvvidconv interpolation-method=Bilinear flip-method=horizontal-flip ! \ "video/x-raw(memory:NVMM), format=(string)RGBA, width=(int)1280, height=(int)960" ! \ nv3dsink

When using OpenCV

The OpenCV included in JetPack is built with CUDA disabled. If you want to use the CUDA version of the functions, you need to build and install with CUDA enabled.

Reference: JEP/script/install_opencv4.5.0_Jetson.sh

When using VPI

By using the Vision Programming Interface (VPI), processing with different backends (CPU / CUDA / PVA / VIC) can be described with the same API.

More information on resizing and format conversion can be found at Convert Image Format.

You can understand how to use the Image Resampling sample program. The method of executing this sample program is introduced below.

$ cp -r /opt/nvidia/vpi1/samples/04-rescale/ ./
$ cd 04-rescale/
$ mkdir build
$ cd build
$ cmake ..
$ ./vpi_sample_04_rescale cuda /opt/nvidia/vpi1/samples/assets/kodim08.png

When using Multimedia API

You can see how to use it in the 07_video_convert sample code. The sample program exists in the following path.

/usr/src/jetson_multimedia_api/samples/07_video_convert

When using NPP

NVIDIA Performance Primitives (NPP) enable very high throughput resizing and format conversion.

Next time, I will explain how to display images!

How was the fourth episode of the series "Jetson Video Processing Programming", introducing resizing and format conversion?

In the next episode 5, I will introduce how to display images.

Click here for Episode 5 Image Display

If you have any questions, please feel free to contact us.

We offer selection and support for hardware NVIDIA GPU cards and GPU workstations, as well as facial recognition, route analysis, skeleton detection algorithms, and learning environment construction services. If you have any problems, please feel free to contact us.