A newly published Apple Machine Learning Research study has challenged the prevailing idea that large-language models (LLMs) like OpenAI’s o1 and Claude’s thinking variants truly possess “reasoning” capabilities. These custom environments allowed for a precise analysis of both the final answers produced by the LLMs and their internal reasoning traces across different complexity levels, according to the researchers. The study indicates fundamental limitations in these AI systems. For this study, Apple researchers designed controllable puzzle environments, such as the Tower of Hanoi and the River Crossing. This approach avoided standard math benchmarks, which are susceptible to data contamination.
The reasoning models tested by Apple’s Research team, including o3-mini, DeepSeek-R1, and Claude 3.7 Sonnet, saw their accuracy collapse entirely once problem complexity crossed certain thresholds, according to a report by MacRumors.
It is worth noting that this research surfaced just days before WWDC 2025.
Apple is expected to focus on new software designs rather than headline-grabbing AI features at this year’s event, according to Bloomberg.