What is image segmentation? I tried using Qualcomm's AI software development kit

Narrow down by specifying conditions

現在1889件がヒットしています。check

Product PickupAI /Artificial Intelligence SoftwareQualcomm

What is segmentation?

Segmentation is a method of labeling each pixel that makes up an image. It divides an image into regions with specific shapes and boundaries to define potentially meaningful regions for further processing such as image classification and object detection.

For example, the left image below performs object detection and emphasizes only its position. The image on the right is a segmentation image, and even the outline of the object is displayed by classifying it by pixel.

Tried segmentation with Qualcomm's AI software development kit

Qualcomm has released a free software development kit called Snapdragon Neural Processing Engine SDK (SNPE) that supports AI development for Snapdragon. This time, we converted a trainedAImodel for segmentation using SNPE and performed inference processing. Also, I would like to add processing to convert the background of the image to grayscale based on the inference results.

AIthe model hasDeepLab-v3I used DeepLab-v3 is a CNN-based semantic segmentation model developed by Google's AI research team, and you can obtain an open trained model.

Reference: What is CNN (Convolutional Neural Network)?

testing environment

・Ubuntu 18.04

・Qualcomm Snapdragon Neural Processing Engine SDKv1.68

・Python3.6.9

・(venv)

1. Install Python related tools

$ sudo apt update && sudo apt upgrade $ sudo apt install python3-dev python3-pip python3-venv protobuf-compiler python-pil python-lxml python-tk

Python package version used in venv environment

・protobuf==3.6.0

・matplotlib==3.0.3

・sphinx==2.2.1

・scipy==1.3.1

・numpy==1.16.5

・scikit-image==0.15.0

・Pillow==7.2

・tensorflow==1.6

2. Setting up the SNPE operating environment

Place the SNPE SDKzipfilein any pathand set up the environment.

(venv) $ unzip snpe-1.68.0.zip (venv) $ export SNPE_ROOT=<snpe-1.68.0.zipを展開したディレクトリのパス> (venv) $ source $SNPE_ROOT/bin/dependencies.sh (venv) $ source $SNPE_ROOT/bin/check_python_depends.sh (venv) $ export TENSORFLOW_DIR=<TensorFlowをインストールしたパス> (venv) $ cd $SNPE_ROOT (venv) $ source bin/envsetup.sh -t $TENSORFLOW_DIR

3. Download DeepLab v3 trained model

Create a directory forDeepLab v3 models and download theTensorFlow models.

$ cd $SNPE_ROOT/models && mkdir deeplabv3 $ cd deeplabv3 $ wget http://download.tensorflow.org/models/deeplabv3_mnv2_pascal_train_aug_2018_01_29.tar.gz $ tar -xzvf deeplabv3_mnv2_pascal_train_aug_2018_01_29.tar.gz

4. Convert AI model to DLC format

Convertthe prepared TensorFlow model to DLC format using snpe-tensorflow-to-dlc. DLCis an AImodel formatthat can run onQualcomm Snapdragon.

Reference:https://developer.qualcomm.com/sites/default/files/docs/snpe/overview.html (Qualcomm's web page will open in a separate window)

$ snpe-tensorflow-to-dlc --input_network deeplabv3_mnv2_pascal_train_aug/frozen_inference_graph.pb --input_dim sub_7 1,513,513,3 --out_node ArgMax --output_path deeplabv3.dlc

deeplabv3.dlcwill be generatedin the current directory.

5. Preparing to run inference

Inference processing inSNPEdoes not directly input image formats such as JPEG orPNG, but converts the image to RAW format before inputting. The RAWconversion applicationis based on the following github repository.

Reference:https://github.com/globaledgesoft/deeplabv3-application-using-neural-processing-sdk.git(GitHub web page will open in a separate window)

$ git clone https://github.com/globaledgesoft/deeplabv3-application-using-neural-processing-sdk.git

Create directories to store the original image data, preprocessed data, and postprocessed data, and save the original image data in theimage_original directory.

$ mkdir image_original image_preprocessed image_postprocessed

This time, I used an image with a person andset the file name toimg_1.jpg.

Preprocess the image.Running thePython script will generateblob.rawin theimage_preprocessed directory.

$ cd image_preprocessed $ python ../deeplabv3-application-using-neural-processing-sdk/BoardApplication/src/deep_input_raw.py --img_path ../image_original/img_1.jpg

blob.rawCreate a text file that describes the path of (example)raw_list.txt

$ touch raw_list.txt $ echo $SNPE_ROOT/models/deeplabv3/image_preprocessed/blob.raw > raw_list.txt

Preprocessing is now complete.

6. Inference execution

SNPE SDKincludesnpe-net-runYou can perform inference processing with an application called prepared in the previous stepsDLCFile(step4)andraw_list.txt (step5)as an optional argument.

$ cd $SNPE_ROOT/models/deeplabv3 $ snpe-net-run --container deeplabv3.dlc --input_list image_preprocessed/raw_list.txt

A directory called output is automaticallycreated, andArgmax:0.rawis generated in the directory. Thisrawfile contains the data defining the regions of the person. Finally, combine it with the original image and grayscale the area other than the recognized object.

$ cd image_postprocessed $ python ../deeplabv3-application-using-neural-processing-sdk/BoardApplication/src/post_process_deeplab_output.py --output_raw ../output/Result_0/ArgMax:0.raw --img_file_path ../image_original/img_1.jpg

image_with_background_changed.jpgwill be output to the image_postprocesseddirectory. When I checked the image, I was able to grayscale the area other than the person.

Image with segmentation + grayscale processing other than the human area

In actual development, the AImodel will be developedaccording to the use case, so please treat the recognition accuracy of the model used this time as reference information.

We will continue to introduce execution examples of various AI models / algorithms. We have a variety of lineups as platforms that can implement AI, so if you are looking for detailed product information related to Qualcomm Snapdragon, please contact us.