A Chinese lab has unveiled what appears to be one of the first ‘reasoning’ AI models to rival OpenAI’s o1. On Wednesday, DeepSeek, an AI research company funded by quantitative traders, released a preview of DeepSeek-R1, which the firm claims is a reasoning model competitive with o1.
What is Reasoning in AI?
Unlike most models, reasoning models effectively fact-check themselves by spending more time considering a question or query. This helps them avoid some of the pitfalls that normally trip up models. Similar to o1, DeepSeek-R1 reasons through tasks, planning ahead, and performing a series of actions that help the model arrive at an answer.
DeepSeek-R1: A Reasoning Model on Par with o1
This can take a while. Like o1, depending on the complexity of the question, DeepSeek-R1 might ‘think’ for tens of seconds before answering. DeepSeek claims that DeepSeek-R1 (or DeepSeek-R1-Lite-Preview, to be precise) performs on par with OpenAI’s o1-preview model on two popular AI benchmarks, AIME and MATH.
AIME and MATH: Popular AI Benchmarks
AIME uses other AI models to evaluate a model’s performance, while MATH is a collection of word problems. But the model isn’t perfect. Some commentators on X noted that DeepSeek-R1 struggles with tic-tac-toe and other logic problems (as does o1).
The Limitations of Reasoning Models
DeepSeek can also be easily jailbroken — that is, prompted in such a way that it ignores safeguards. One X user got the model to give a detailed meth recipe. And DeepSeek-R1 appears to block queries deemed too politically sensitive.
The Pressure from the Chinese Government
In our testing, the model refused to answer questions about Chinese leader Xi Jinping, Tiananmen Square, and the geopolitical implications of China invading Taiwan. The behavior is likely the result of pressure from the Chinese government on AI projects in the region.
Models in China must undergo benchmarking by China’s internet regulator to ensure their responses ’embody core socialist values.’ Reportedly, the government has gone so far as to propose a blacklist of sources that can’t be used to train models — the result being that many Chinese AI systems decline to respond to topics that might raise the ire of regulators.
The Emergence of New AI Approaches
The increased attention on reasoning models comes as the viability of ‘scaling laws,’ long-held theories that throwing more data and computing power at a model would continuously increase its capabilities, are coming under scrutiny. A flurry of press reports suggest that models from major AI labs including OpenAI, Google, and Anthropic aren’t improving as dramatically as they once did.
That’s led to a scramble for new AI approaches, architectures, and techniques. Reasoning models like DeepSeek-R1 are seen as a key development in this area.
DeepSeek: The Company Behind the Model
DeepSeek is an AI research company funded by quantitative traders. The company has been working on various AI projects, including natural language processing and computer vision.
Conclusion
The unveiling of DeepSeek-R1 marks an important milestone in the development of reasoning models. While there are limitations to the model, it shows promise as a rival to OpenAI’s o1. As the AI landscape continues to evolve, we can expect to see more innovative approaches like this one emerge.
Related Articles
- Nvidia’s Project Digits is a ‘personal AI supercomputer’
- Kyle Wiggers
- 5 hours ago
- Google is forming a new team to build AI that can simulate the physical world
- Kyle Wiggers
- 14 hours ago
- What are AI ‘world models,’ and why do they matter?
- Kyle Wiggers
- Dec 14, 2024
Categories
- AI (28)
- Startups (22)
- TechCrunch Daily News (20)
