The Large Language Models (LLMs) war is not going to end anytime soon, and it all happened because of OpenAI. And since then, there is going back. The world is never going to be the way it was back then.
When the company unleashed ChatGPT, its CEO Sam Altman has described the moment as a defining moment for the company. Altman called ChatGPT's rise the early discovery of scaling laws in LLMs, further saying that it was a "scientific wonder" that shaped not only the industry, but also the complete strategy for OpenAI.
Speaking at a podcast hosted by venture capital firm Andreessen Horowitz (a16z), Altman delved into the idea that the breakthrough of ChatGPT was a fluke of luck and a mix of variables that happened to be perfected.
"When we got the reasoning model breakthrough, I also thought that we were never going to get another one like that. We'll never get that lucky again."
Achieving something like creating a resoning model is indeed a milestone for a good reason.
This is because AI systems that can think through problems, generalize, and draw conclusions like humans, is already difficult. And achieving a proper, and strong reasoning model is even more difficult because most modern AI systems are fundamentally based on pattern recognition rather than true logical thinking.
LLMs learn by predicting the next word based on vast amounts of text, which allows them to imitate reasoning but not to reliably perform it.
Real reasoning, however, requires the ability to build and manipulate abstract concepts, maintain coherent logic across many steps, and verify conclusions. These all go beyond simple pattern matching.
Another challenge is that reasoning depends on structured, systematic processes similar to how humans use working memory, planning, and self-correction.
These cognitive tools do not exist natively in today’s models. Instead, models approximate them through correlations in training data, which makes their reasoning brittle. A model may solve a problem correctly once but fail unpredictably when the problem is rephrased, because it does not truly understand the underlying logic.
While scaling laws have shown that increasing model size improves performance, including some reasoning-like abilities, scaling alone cannot fully solve reasoning. Many reasoning tasks require algorithmic or symbolic steps, but language models rely on statistical prediction. As a result, they often generate logically inconsistent answers or hallucinations when confronted with unfamiliar or complex problems.
Reasoning is also difficult for AI because it requires maintaining long-term coherence. Complex reasoning tasks involve tracking many details, referring back to earlier steps, and ensuring that the final conclusion aligns with the initial premises. Even with large context windows, language models struggle to maintain perfectly consistent chains of thought across extended reasoning processes.
Finally, true reasoning demands reliable intermediate steps.
Humans demonstrate their reasoning by showing each step of their logic, allowing verification and error-checking. AI models, however, can generate convincing explanations that are not genuinely connected to how the model produced its answer. This makes reasoning appear correct on the surface while hiding underlying errors, which is one of the biggest challenges for building trustworthy reasoning systems.
Despite this, Altman noted that the technology has continued to outperform expectations, surprising even its creators.
Altman emphasized that moving ahead, OpenAI is working on a major expansion of its AI infrastructure footprint. The company said it is attempting to partner with industry heavyweights including Foxconn, AMD, Nvidia and Oracle to build more massive computing capacity.
"I have never been more confident in the research roadmap [...] and the economic value that will come from using those models. But to make the bet at this scale, we need the major chunk of the industry to support it."
At this time around, the combined value of OpenAI’s plans stands at $1 trillion, which is about a double of Nvidia's valuation of $500 billion.
Altman also warned that the race toward AGI is built on a "vertical stack of things" where research leads to great leaps, but future innovation is further research. The company’s long-term goal, he said, is to break through limitations of future models.
He also signalled that the limits of large language models are still far away.
"If LLM-based stuff can get far enough, then it can do better research than all of OpenAI put together."