What is transfer learning in the first place?

As a preface, as mentioned in the learning flow introduced at the beginning, specialized knowledge is required to realize a highly accurate network model. Various network models have been developed in the field of deep learning, and many have been published. optimization is important.

This is done in the learning process, and network model learning is often compared to human learning.

In order for humans to learn what they see, we need a lot of experience and a good teacher. be able to guess. Even in neural networks, in order to tune the parameters from the initial state to the parameters that can demonstrate performance, it is necessary to perform a huge amount of experience (learning) with a lot of data and correct teachers (correct labels), and an environment that realizes this learning preparation and execution is a challenge.

Transfer learning is a method to quickly and efficiently generate a model that demonstrates performance in a target use case by utilizing a trained model that has already been generated by performing a large amount of training, rather than learning from a network model in the initial state. It's a good way to do it.

Specifically, when implementing a model that identifies the type of dog, we can consider using a trained model that identifies the type of cat. It is possible to apply the learning results of common characteristics of animals such as the shape of the face and the shape of four legs, and to re-learn the unique differences necessary to distinguish between dog breeds, so that the model can be realized efficiently. There is a nature.

AUGMENTATION (data augmentation)

In deep learning, learning is performed by comparing the output result obtained for the input data with the correct label. It can be expected that correct inference results can be obtained from data that is similar to the data used during training, but a phenomenon occurs in which correct inference results cannot be obtained for inputs with large differences.

For example, the same image data may be inferred as a different input image just by rotating it or changing the color tone. Humans naturally see and learn the same thing under various environmental conditions (angles, colors, etc.), but computers need to prepare input data with such diverse variations during learning.

By using the AUGMENTATION function of NVIDIA TAO TOOLKIT, it is possible to automatically generate data for various environmental conditions such as rotation and color shades from images in existing datasets, and perform learning under various environmental conditions. becomes.

Examples of AUGMENTATION

PRETRAINED MODEL

In deep learning, the network model is the part that corresponds to the human brain and is extremely important. As explained in transfer learning, not only the model structure but also the parameter adjustment of the model nodes greatly affects the inference performance. At NVIDIA, not only general models but also models that can be used with NVIDIA TAO TOOLKIT, various network models including parameters learned and optimized with huge data sets to support various use cases as shown below. are preparing.

For example, models that are good at detecting people include PeopleNet, which is trained using a large amount of human data.

Recommended hardware for NVIDIA TAO TOOLKIT

In NVIDIA TAO TOOLKIT training, it is recommended to prepare a hardware environment equipped with a large amount of memory and a high-performance GPU as shown below in order to perform network training of a huge number of nodes. Please see this link for details.

This is an implementation example of an application that incorporates the following two-stage inference processing into a video stream captured by a camera connected to Jetson.

・Inference processing 1: Hands are detected (Images of the hands are cut out in later processing.)

・Inference processing 2: Identify the shape of the extracted hand and judge the following gestures.

(Thumbs Up, Fist, Stop, Ok, Two, Random)

Source: NVIDIA

In this execution result, we can confirm that both hands are detected by the green bounding Box in inference process 1, and that the right hand is identified as OK and the left hand as Stop in inference process 2. In Part 1, we introduced an execution example of a person detection application, but this time, after recognizing hands, more advanced inference processing such as determining the meaning of the gesture is possible. NVIDIA TAO TOOLKIT makes it easy to configure inference using multiple models, including network models.

In inference process 1, the network model was generated by performing transfer learning with NVIDIA TAO TOOLKIT, and in inference process 2, the existing model file (GestureNet) was used as is. In the next article, I will focus on inference processing 1, and break it down into the dataset preparation, learning, and inference phases, and introduce the flow of development until inference processing 1 is executed.