Prototyping the GStreamer pipeline

As with encoding, you can prototype the decoding process with the gst-launch-1.0 command.

For example, the pipeline that decodes a file called test.mp4 encoded with the H.264 codec and stored in an MP4 container and displays it on the monitor is as follows.

gst-launch-1.0 -e \ filesrc location=test.mp4 ! \ qtdemux name=demux demux.video_0 ! \ queue ! \ h264parse ! \ nvv4l2decoder ! \ nvegltransform ! \ nveglglessink

GStreamer API

As with encoding, after prototyping on the command line, the GStreamer API integrates into your application.

C API

Python API

Using GStreamer from OpenCV

The default OpenCV installed by JetPack does not have a Jetson-specific hardware accelerator enabled, but by using GStreamer as the backend of OpenCV VideoCapture, high-speed video decoding by NVDEC can be used from OpenCV.

You can pass the decoded stream data to OpenCV with the appsink plugin.

import cv2 import sys FILE = 'test.mp4' GST_COMMAND = 'filesrc location={} ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw,format=(string)I420 ! videoconvert ! appsink'.format(FILE) cap = cv2.VideoCapture(GST_COMMAND) if not cap.isOpened(): print("Cannot open stream") sys.exit() while True: ret, frame = cap.read() if ret == False: break cv2.imshow("Test", frame) key = cv2.waitKey(1) if key == 27: # ESC break cv2.destroyAllWindows() cap.release() sys.exit()