Researchers at the University of California, Los Angeles have conducted a study to test the cognitive abilities of deep learning models. Using the GPT-3 large language model, they found that it performed at or above human capabilities for resolving complex reasoning problems. Specifically, the researchers tested the model on analogical tasks, such as the Ravens Progressive Matrices, which require test takers to identify patterns.

The results showed that the AI performed at the higher end of humans scores and made a few of the same mistakes. The researchers also asked the AI to solve a set of SAT analogy questions involving word pairs, in which it performed slightly above the average human level. However, the AI struggled with analogy problems based on short stories.

The study suggested that the AI could be employing a mapping process similar to how humans approach these types of problems. The researchers speculated that the AI might have developed some alternate form of machine intelligence.

It is important to note that the AIs performance was based on its training data, which has not been publicly disclosed by OpenAI, the creator of GPT-3. Therefore, it is unclear whether the AI is genuinely reasoning or if it is simply relying on its training data to generate answers.

Overall, this study adds to the ongoing discussion about the cognitive abilities of AI systems. While the AI showed promise in certain areas, there are still limitations and questions about its true intelligence. Further research is needed to understand the capabilities and limitations of deep learning models.

Read more:

The Cognitive Abilities of Deep Learning Models - Fagen wasanni