Object detection: Running SSD MobileNet with Snapdragon

Narrow down by specifying conditions

現在1890件がヒットしています。check

Design AI/Artificial Intelligence Software Microcomputer/Processor/DSP Qualcomm

Object detection with edge AI terminal equipped with Qualcomm Snapdragon

We will implement various AI models and test inference processing on an edge AIcomputing terminal (EB2)equipped withQualcomm's embeddedSoC. This time, with the theme of object detection, I would like to incorporate the SSD MobileNet v1 model using the TensorFlow Liteframework.

Click here for EB2 specification information

AIthe model hasTensorFlow.orgButTensorFlow LiteTrained model published in the tutorial(COCO SSD MobileNet v1)Use the.

Store AI model in device

GStreamerimplemented inEB2 providesQualcomm's own plugin(qtimletflite)for executing AI image processing using models of theTensorFlow Liteframework. This plugin does everything from pre-processing to post-processing. Pre-processing refers to the phase of image/video downscaling, color conversion, padding, etc. Post-processing refers to the phase of classification, detection, segmentation, etc.

In order to use the qtimletflite plugin with GStreamer, it is necessary to load the AI model and label data, so the downloaded AI trained model (ssd_mobilenet_v1_1_metadata_1.tflite) and the COCO dataset label list text file (labelmap. txt) in any directory. This time it is"/data/misc/camera/".

* You can refer to the contents of labelmap.txt from the link below.

labelmap.txt

AI inference processing

GStreamer command example (Save AI processing result as mp4 file)

root@tc-eb2:/data/misc/camera# ls
labelmap.txt ssd_mobilenet_v1_1_metadata_1.tflite test.mp4
root@tc-eb2:/data/misc/camera# gst-launch-1.0 filesrc location=/data/misc/camera/test.mp4 ! qtdemux name=demux demux. ! h264parse ! queue ! qtivdec ! qtimletflite model=/data/misc/camera/ssd_mobilenet_v1_1_metadata_1.tflite labels=/data/misc/camera/labelmap.txt postprocessing=detection ! qtioverlay ! video/x-raw\(memory:GBM\) ! queue ! omxh264enc control-rate=max-bitrate target-bitrate=6000000 interval-intraframes=29 periodicity-idr=1 ! queue ! h264parse ! mp4mux ! queue ! filesink location="/data/misc/camera/test_bbox.mp4"
root@tc-eb2:/data/misc/camera# ls
labelmap.txt ssd_mobilenet_v1_1_metadata_1.tflite test.mp4 test_bbox.mp4

When the command is executed and the process is completedfilesinkoflocationA video file (test_bbox.mp4) was generated in the directory specified in . thismovieThe file draws labels and bounding Box for the detection results. There seems to be room for improvement in accuracy because it is a sample model for tutorials, but I was able to easily run the object detection algorithm with a single command.

This time, the input was a video file that was saved in advance on the terminal, and the output was also saved as a video file in the terminal.GStreamerhas various plug-ins, such as real-time video from the camera as the input source,AIThere is also a plug-in that outputs the image with the bounding Box drawn after processing to the display, so it can be flexibly combined according to the use case.

For example, if you want to output the labels and bounding Box to the HDMI display as they are after rendering, you can do it by changing the latter part of the GStreamer command slightly and writing it like this.

GStreamer command example (AI processing result output to HDMI display)

root@tc-eb2:/data/misc/camera# ls labelmap.txt ssd_mobilenet_v1_1_metadata_1.tflite test.mp4 root@tc-eb2:/data/misc/camera# gst-launch-1.0 filesrc location=/data/misc/camera/test.mp4 ! qtdemux name=demux demux. ! h264parse ! queue ! qtivdec ! qtimletflite model=/data/misc/camera/ssd_mobilenet_v1_1_metadata_1.tflite labels=/data/misc/camera/labelmap.txt postprocessing=detection ! qtioverlay ! video/x-raw\(memory:GBM\) ! waylandsink sync=false fullscreen=true

We will continue to introduce execution examples of variousAImodels/algorithms. We have various lineups other thanEB2as a platform that can implementAI, so if you are looking for detailed product information related toQualcomm Snapdragon, please contact us.