GPT-3: focus on the new Open AI’s technology

Obviously, the third generalist natural language model released by Open AI is breaking all performance records. GPT-3 has such potential that Microsoft has licensed its exclusive use for its products and services. For a few additional details about GPT-3, David Oldham (CEO Octopeek Inc. – USA) asked Ouassim AIT ELHARA, innovation and R&D manager at Octopeek.

What exactly is GPT-3?

Let me just start by saying that GPT-3 stands for “Generative Pre-trained Transformer 3”. It’s the latest edition of the Natural Language Processing (NLP) model developed by Open AI. It is also the most effective linguistic model ever released to date. What I mean is that it doesn’t just provide a new answer to natural language issues: GPT-3 can work on a much larger scale. The AI ​​is trained with 175 billion parameters, a hundred times more than the previous version released in 2019.

The quality of the results can also be explained by the fact that GPT-3 is what we call a feature learner. It is “pre-trained” on a version of a textual corpus extracted from a large proportion of the web. As its training is global, GPT-3 can be used as-is on a very large number of distinct tasks without additional training. For example, when directly asked “What is your favorite animal?”, GPT-3 replied “my favorite animal is a dog.”

So now we know AI’s favorite animal is a dog, which is a very important milestone. What kind of technological advances comes with GPT-3?

GPT-3 is a new step in our ability to model human language. Once again, the model relies on such a large amount of data that it provides a much more precise, much more complete representation of the knowledge available today on the Internet. By now, it’s too early to grasp all the changes this will bring to us, but its performance speaks for itself. This is a great engineering achievement so far, and a milestone in generic natural language processing.

How will companies use such technology? 

We can expect two types of use. First of all, I think that AI technicians, engineers, and data scientists will be able to use GPT-3 as a building block for much larger models, just like any other algorithm or model language. So far, GPT-3 is good at solving generic tasks, but it is not as accurate as a super-trained model on hyper-specific ones. 

Besides, it will probably be used as a super personal assistant. I mean “assistant” because GPT-3 cannot actually reason. It is rather like a huge memory which can find and report information in real-time. It seems almost magical. But unlike humans, who can produce new ideas, AI builds its answers from a statistical vision of the recurring proximity of words to one another.