AI開発環境の作り方　初心者向け - 半導体事業

Narrow down by specifying conditions

現在1888件がヒットしています。check

Design microcomputer, processor, DSP NVIDIA

AI・ディープラーニングが普及し、AIが様々な分野で活躍する時代になってきました。例えば、医療分野においては医療画像判断、お客様からの問い合わせや誘導にはチャットボットでAIが返答することが可能になりました。

本記事では、「AI開発環境の作り方　初心者向け」と題して、まだAI開発のご経験がなくこれから開発に取り組まれるエンジニアの方に向けて、まずは必要最低限のAI開発環境をご紹介し、将来的に考慮していただきたいポイントまで解説していきます。

AI development flow

AI開発を始めるにあたって、まずAIを使うことでどのようなことを実現できるのかをイメージすることが重要です。そのゴールが不明確であった場合、思った効果が出ずに中途半端なもので終わってしまう可能性があります。まずは、目的を明確に設定しましょう。

その上で、企画をまとめ、AIの学習に必要なデータを集め、学習モデルを作り、AIでさせたいことを実現するアプリケーション開発を経て、運用に移行していきます。ただし、運用後もより良いモデルの精度を求め、モデルの再度学習させたり機能拡張のためにAIモデルを追加作成するなどして、AIを活用した本当のシステム運用が始まります。

In order to utilize AI in this way, it is necessary to prepare the infrastructure environment for AI development at the appropriate timing of the following flow.

Environment required for AI development

To develop an AI model, you need an "environment" for it. In this chapter, I will talk about the environment that should be prepared first from the viewpoint of hardware and software.

hardware

Since a lot of computational processing is performed in AI model learning, development efficiency depends on whether computation time can be reduced. Therefore, GPUs (Graphic Processor Units) and processors capable of high-speed arithmetic processing are essential. There are two main ways to use the GPU.

・クラウドにあるGPUインスタンス

・GPUカードが挿さったワークステーションやノートパソコンなど

However, if you suddenly choose the cloud, you may be worried that the trial and error of deep learning will take time and your bill will increase, so we recommend that you purchase a workstation or laptop first.

Also, when purchasing, it is necessary to select the OS accordingly.

OS	参考となる WEBの情報量	GPU support	Conclusion
Linux (Ubuntu)	◎ abundant	〇 Yes	the best option. Especially for those who want to learn and practice AI in earnest with WS with GPU.
Windows	〇 Moderate	〇 Yes	It is ideal for those who want to use a PC for daily use and a familiar environment, but anxiety remains when encountering an error.
macOS	△ Not much	× None	It cannot be selected when using the GPU. If it is a CPU version, it is OK.

現時点で、GPUを使ったAI開発をおこなうにはLinuxを選択するのが最良の選択といえます。

購入や借りただけでは基本的にAI開発に必要なOS以外のソフトウェアは搭載されておらず、開発者自身で構築、整備を行っていく必要があります。下図にはAI開発に最低限必要なハードウェア、ソフトウェアの環境を図示しました。この要素について紐解いていきます。

software

There are many development methods, and I think that there are many people who do not know what kind of programming language and what kind of environment should be prepared. In AI development, the stage of building the environment is difficult, and the current situation is that it is difficult even for beginners to try. First of all, if you choose it, we will carefully select and introduce you to the one that you can't go wrong.

What is a language for AI development

When it comes to programming languages, most people think of C language. The C language is a relatively old language, and is not often used in AI development because of the large amount of source code and the time and effort required to compile programs.

In AI development, a language called Python is often used. The reason is that it is a lightweight programming language and requires less source code than C language, so it is easy to learn the language. Furthermore, many of the core frameworks for AI development (details will be explained later) are written in the Python language. Therefore, I think that Python is a good choice for AI development.

What is a framework library?

In the world, convenient tools for developing AI are provided as OSS (open source software). Among them, let's understand what frameworks and libraries are.

A library is a collection of code that provides a certain function, and is used to easily write Python code in the development of machine learning and deep learning. A collection of multiple libraries with various functions is called a framework. The terms library and framework are often used interchangeably.

Here are three frameworks that are easy to use for machine learning and deep learning development.

・　TensorFlow

Googleが開発している機械学習、ディープラーニングのフレームワークでTensor（多次元配列）の操作に特化しており、「脳を模した機械学習モデル」と呼ばれるニューラルネットを効率的に実装できます。

・　PyTorch

Facebookが開発を主導し、その書きやすさと使いやすさから人気があり、非常に柔軟なニューラルネットワークの記述ができ、今主流であるディープラーニングフレームワークの中でもかなりの人気を誇ります。

・　scikit-learn

Pythonの代表的な機械学習のフレームワークで様々な機械学習のアルゴリズムがクラスとして用意されていて、簡単に機械学習を試すことができます。

As long as you have the GPU + OS (Linux) + framework (library) that has been explained so far, you can start AI development. If you want to try to operate it, please refer to the information so far.

ただし、フレームワーク・ライブラリーとその上で開発するアプリケーションは非常に依存関係が強いという特徴があり、開発中にあるライブラリーを最新のものに更新すると、そのライブラリーが含まれるフレームワーク全体が動かない、過去に開発したものが使えないという事態が発生します。

その問題を予め回避する方法として、AI開発環境の仮想技術を活用することをおすすめします。ただの仮想技術というのではなくAI開発環境の構築を飛躍的に簡単にすることもできるので、なおさら知っておいてほしい技術です。

What is virtual technology Docker

When developing AI, it has become standard to use “Docker”, one of the virtual technologies that can receive various benefits. I will also explain what Docker is and the benefits of using it.

Docker is a tool for building containers, a virtualization technology that allows you to group applications, frameworks and libraries into independent environments. Updates to some libraries in one container do not affect environments in other containers. Also, in general, to implement a virtual machine, a hypervisor is installed on the host machine, a guest OS is run, middleware is run on top of that, and so on. Docker can realize those functions in one, so it is possible to start, stop, and destroy each container at light weight and high speed.

Here are three advantages of using Docker.

(1) Easy updating of complex frameworks

Frameworks used in AI development such as Tensorflow and PyTorch are evolving day by day, and we want to update to the latest environment. However, it is not easy to update these as it was said that there is a strong dependency. By building Docker in the environment, the latest framework is distributed in the form of a “Docker image” and can be easily updated. Anyone can use it easily, and NVIDIA also provides GPU-optimized Docker images on NGC (nvidia GPU cloud), so there is no reason not to use it.

②多数の人と、同じ環境で動作させて、動作の再現性を確認できる

By passing the created Docker image to other people and using it, you can prevent environment version discrepancies with other people and shorten the development environment preparation time.

③小さい環境を、簡単に構築・破棄ができる

It is suitable for trial-and-error AI development because a disposable test environment can be created and destroyed instantly without destroying the host environment.

ここまで構築ができるとAI開発の初期においては理想的な環境となります。最近ではDockerを含めた環境をインストールした状態で販売しているワークステーションやインスタンスもあるようですが、一般的にはここまでをご自身で構築されています。

Development environment to consider in the future

At first, projects that were developed by individuals will grow to the scale of team development as the examination progresses, and more results will be required. As the number of teams increases, many developer members will worry about the environment for development. I often hear stories like this.

・チームで効率的に計算リソース（GPU）を使いたい

・開発した資産を共有したい

・計算リソースが足りなくなった時に簡単にリソースを増やしたい

Using Docker will increase portability to some extent, but you will still need to think about how to use resources well. In that case, the secret is to consult an engineer with specialized knowledge.

Wouldn't you like to know before you "fail" in building the environment?

Now that the introduction of AI is progressing, do you know the reality that "preparation of the environment" before AI learns delays development? In this 10-minute video, we will introduce you to the reality of the site and the solution to it.

This is a must-see, especially if you are a person in charge of development and want to prevent delays in advance.