Jetson Video Processing Programming Episode 3 Video Input (USB-connected V4L2-compliant camera)

Narrow down by specifying conditions

現在1887件がヒットしています。check

Design AI/Artificial IntelligenceNVIDIA

Continuing from the previous CSI camera, Part 3 of this series will focus on V4L2 compliant cameras such as USB cameras, and will introduce considerations when connecting V4L2 compliant cameras to Jetson.

[Jetson video processing programming]

Episode 1 What you can do with JetPack and SDK provided by NVIDIA

Episode 2 Video input (CSI-connected Libargus-compliant camera)

Episode 3 Video input (USB-connected V4L2-compliant camera)

Episode 4 Resize and format conversion

Episode 5 Image display

Episode 6 Video encoding

Episode 7 Video decoding

Episode 8 Image Processing

Episode 9 Deep Learning Inference

Episode 10 Maximum Use of Computing Resources

V4L2 API

As you know, devices handled by Linux can be accessed through special files called device files (device nodes). The video capture device will be a file named /dev/videoX (where X is a number assigned by your OS). Device files can be controlled from user space using system calls such as open / close / ioctl / mmap. Based on these system calls, the Video for Linux API version 2 defines interfaces such as device open/close, video format selection, brightness setting, image input/output method selection, and actual data input/output loops. (V4L2 API).

Video capture flow with V4L2 API (example)

(1) Opening a device file (open)

(2) Device initialization

Cropping settings (ioctl VIDIOC_S_CROP)

Set capture image size and image format (ioctl VIDIOC_S_FMT)

Other settings

(3) Initialize image buffer (for V4L2_MEMORY_MMAP method)

Request a buffer (ioctl VIDIOC_REQBUFS)

Buffer memory allocation

Mapping a buffer into buffer memory (mmap)

(4) Start streaming (ioctl VIDIOC_STREAMON)

(5) Capture loop processing

Get buffer (ioctl VIDIOC_DQBUF)

image processing

Return a buffer (ioctl VIDIOC_QBUF)

Repeat until a termination signal is received

(6) Stop streaming (ioctl VIDIOC_STREAMOFF)

(7) Device termination processing

Finish mapping buffer to buffer memory (munmap)

(8) Device close (close)

Video capture sample code for Jetson

The sample code is included in JetPack and can be found in the path below.

/usr/src/jetson_multimedia_api/samples/12_camera_v4l2_cuda

Consuming V4L2 APIs from other APIs

Use from GStreamer

$ gst-launch-1.0 v4l2src ! xvimagesink

For web cameras that output in Motion JPEG

$ gst-launch-1.0 v4l2src ! jpegdec ! xvimagesink

Use from OpenCV

By utilizing the OpenCV VideoCapture V4L2 backend, you can acquire images from V4L2 cameras with OpenCV's VideoCapture.

reference:

MACNICA-CLAVIS-NV/camcapture

Get camera information

You can get information about V4L2 cameras with the v4l-utils tool. First, install v4l-utils.

$ sudo apt install v4l-utils

camera list

/dev/video0: Raspberry Pi camera module V2

/dev/video1: Logitech C270n HD webcam

$ v4l2-ctl --list-devices vi-output, imx219 6-0010 (platform:54080000.vi:0): /dev/video0 UVC Camera (046d:0825) (usb-70090000.xusb-2.2): /dev/video1

Get supported image formats

$ v4l2-ctl --device=/dev/video1 --list-formats-ext ioctl: VIDIOC_ENUM_FMT Index : 0 Type : Video Capture Pixel Format: 'YUYV' Name : YUYV 4:2:2 Size: Discrete 640x480 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 160x120 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 176x144 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 320x176 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 320x240 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 352x288 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 432x240 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 544x288 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 640x360 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 752x416 Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 800x448 Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 800x600 Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 864x480 Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 960x544 Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 960x720 Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 1024x576 Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 1184x656 Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 1280x720 Interval: Discrete 0.133s (7.500 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 1280x960 Interval: Discrete 0.133s (7.500 fps) Interval: Discrete 0.200s (5.000 fps) Index : 1 Type : Video Capture Pixel Format: 'MJPG' (compressed) Name : Motion-JPEG Size: Discrete 640x480 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 160x120 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 176x144 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 320x176 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 320x240 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 352x288 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 432x240 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 544x288 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 640x360 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 752x416 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 800x448 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 800x600 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 864x480 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 960x544 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 960x720 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 1024x576 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 1184x656 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 1280x720 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 1280x960 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.040s (25.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps)

Get control parameters

$ v4l2-ctl --device=/dev/video1 --list-ctrls brightness 0x00980900 (int) : min=0 max=255 step=1 default=128 value=128 contrast 0x00980901 (int) : min=0 max=255 step=1 default=32 value=32 saturation 0x00980902 (int) : min=0 max=255 step=1 default=32 value=32 white_balance_temperature_auto 0x0098090c (bool) : default=1 value=1 gain 0x00980913 (int) : min=0 max=255 step=1 default=64 value=192 power_line_frequency 0x00980918 (menu) : min=0 max=2 default=2 value=2 white_balance_temperature 0x0098091a (int) : min=0 max=10000 step=10 default=4000 value=4150 flags=inactive sharpness 0x0098091b (int) : min=0 max=255 step=1 default=24 value=24 backlight_compensation 0x0098091c (int) : min=0 max=1 step=1 default=0 value=0 exposure_auto 0x009a0901 (menu) : min=0 max=3 default=3 value=3 exposure_absolute 0x009a0902 (int) : min=1 max=10000 step=1 default=166 value=87 flags=inactive exposure_auto_priority 0x009a0903 (bool) : default=0 value=1

Access CSI cameras with V4L2 API

The Raspberry Pi camera module V2, which is a CSI camera, can be accessed with the V4L2 API (instead of the Libargus Camera API), but the only supported image format is Raw.

$ v4l2-ctl --device=/dev/video0 --list-formats-ext ioctl: VIDIOC_ENUM_FMT Index : 0 Type : Video Capture Pixel Format: 'RG10' Name : 10-bit Bayer RGRG/GBGB Size: Discrete 3264x2464 Interval: Discrete 0.048s (21.000 fps) Size: Discrete 3264x1848 Interval: Discrete 0.036s (28.000 fps) Size: Discrete 1920x1080 Interval: Discrete 0.033s (30.000 fps) Size: Discrete 1640x1232 Interval: Discrete 0.033s (30.000 fps) Size: Discrete 1280x720 Interval: Discrete 0.017s (60.000 fps)

Next time, I will explain how to resize and format conversion!

How was the video input method for V4L2-compliant cameras in Part 3 of the series "Jetson Video Processing Programming"?

Next time, I will introduce resizing and format conversion.

Click here for Chapter 4 Resize and format conversion

If you have any questions, please feel free to contact us.

We offer selection and support for hardware NVIDIA GPU cards and GPU workstations, as well as facial recognition, route analysis, skeleton detection algorithms, and learning environment construction services. If you have any problems, please feel free to contact us.