This article is recommended for those who
- I want to get an overview of the latest AI "GTP-3"
- I want to know the latest trends in language generation models
Time needed to finish reading this article
10 minutes
Introduction
hello!
This article is about the latest GPT-3.
GPT-3 is the latest version of the language model, but its popularity is increasing all over the world.
This time, we will deliver an overview of this latest model "GPT-3" and future prospects!
What is GPT-3 (Generative Pre-Training 3)?
GPT-3 is a “sentence generation language model” created by OpenAI. (OpenAI is a non-profit organization researching artificial intelligence.)
GPT-3 is the latest version of the sentence generation language model created by the organization.
The question is, “What can a sentence generation language model do?”
In a nutshell, it is "a product that automatically generates natural sentences and program code that look as if they were written by a human, from a simple directive."
Let's take a closer look at how revolutionary this is and what it can do in practice.
Reaction from various fields
Since the release of this latest model, what kind of reaction has been obtained in various fields?
Let's look at an example.
<About the collaboration between Microsoft and OpenAI>
On September 22, 2020, Microsoft announced on its official blog that it had acquired an exclusive license for GPT-3.
Microsoft EVP Kevin Scott wrote in his official blog:
"The range of commercial and creative possibilities unlocked by the GPT-3 model is enormous and has truly novel capabilities."
"The possibilities in areas like text generation are only limited by the ideas and scenarios we bring to the table."
and future expectations.
Feature ①
From here, let's take a look at the features of what can be done specifically with this GPT-3.
・You can create "natural" sentences from a few sentences!
For example, let's say you enter a simple three-line synopsis.
Then, it fleshes out the text and generates a “natural” text that looks like it was created by a human being.
A famous example is a blog post written by Manuel Araoz, founder of OpenZeppelin, which builds developer tools for distributed systems.
At the end of the article, the shocking fact is announced that "this article was created by GPT-3".
Feature ②
・Web design can be handled freely!
You can freely generate the source and code design in the same way as feature ①.
For example, if you just type "google logo search Box two light gray buttons with a gap between them and say 'google search' and 'I'm Feeling Lucky' the search Box is wide" and it almost looks like google. We can create the same design.
・You can also program!
You all know that by now. You can also program.
For example, if you enter the directive "500 yen income, 300 yen expenditure, display the balance", you can create a program that can realize it.
In this part, I wrote "web design 'also'" and 'programming 'also'', but there is a high possibility that programming generation will be put into practical use earlier than language generation. It is said that.
It is often pointed out that AI, which learns from data on the Internet, cannot produce well-balanced answers due to bias in the learning data.
Therefore, knowledge that cannot be found on the Web cannot be reflected.
If it is a programming language, a large number of source codes are open to the public on the Internet.
Therefore, it is said that there is a high possibility that it will be put into practical use earlier than natural language processing.
Feature ③
In addition to the features mentioned above, there are many things that can be done with GPT-3.
Let's introduce some of them.
<Automatic generation of design>
Significant breakthroughs will also be possible in automatic design generation.
For example, when designers create things, it is possible to automatically create content that people are interested in and automate AB testing, so it is possible to significantly reduce the man-hours required to optimize the product. .
In addition, these developments may lead to a future where AI creates art such as painting.
<Automatic generation of sentences>
In the automatic generation of sentences, many people will be able to realize things that they think would be convenient.
For example, it is possible to automatically generate manuals based on chat exchanges, or to automatically generate structural sentences simply by inputting voice.
It will also be possible to automatically generate a summary of the content from the video.
In this way, it will be possible to automatically generate things that had to be done manually until now.
Limit ①
However, there are limits to this kind of cunning.
First of all, it is "physically weak".
For example, in response to the question ``Will cheese melt if put in the refrigerator?'', humans would naturally answer ``it doesn't melt,'' but GPT-3 may answer ``melt.''
Sometimes we misinterpret the "natural providence" that humans physically understand.
Limit ②
The second point is that "they don't notice contradictions in the generated sentences and designs/they can't tell whether the values are unacceptable in this day and age."
Even if what is stated at the beginning of the generated text is completely different from what is stated at the end of the summary, it cannot be noticed.
Because they don't understand the meaning of the word itself.
Also, among natural language processing, the language generation model called GPT-3, which has developed as a method that applies deep learning, was created by learning a huge amount of data.
Therefore, we are learning data such as past literature.
The fact that it refers to past data means that it includes values that are not accepted in the modern world.
Therefore, the problem is that it is not possible to tell whether it is taboo in today's world.
Limit ③
The third limitation point is that it takes a huge amount of money and time.
Since GPT-3 boasts a huge number of parameters, it takes time and money to train a large model.
Future prospects
Startups and technology companies are actively trying to develop applications using GPT-3.
For example:
・Effective sentiment analysis in online shopping
・Automatic website generation for small businesses
・Next-generation real-time automatic simultaneous translation
・Distinguish harmful content on SNS by analyzing user sentiment
In Japan, on November 25, 2020, LINE announced that it had begun developing a ``super-giant language model'' specialized for Japanese using GPT-3.
Summary
How was that.
This time, I tried to summarize the outline and function explanation about GPT-3.
GPT-3 is the latest version of the text generation language model that has a huge number of parameters, and is expected to have a great impact on the future world.
It will have a great impact on the future promotion of DX by being able to generate sentences, programming, and web design that have been done manually so far.
However, there are still some shortcomings.
The lack of a physical perspective, the inability to distinguish social common sense, and the huge amount of time and money it takes to make it learn are all things that I am looking forward to seeing.
I would like to continue to pay attention to GPT-3, which is attracting a lot of attention in various fields.
Related article
* Tech Blog AI Women's Club *
JSAI2020 Pick up from about 900 papers! -Explanability and systemization of AI-
* Tech Blog AI Women's Club *
Machine Learning Event Update: NeurIPS 2020
* Tech Blog AI Women's Club *
Decoding the Human Brain: Neuroscience in NeurIPS 2020
Latest Information
Case Study
Blog
Document List