Pathway builds truly native reasoning model to solve LLM Sudoku stumbling blocks
Pathway is a post-transformer AI lab that claims to be delivering a faster path to AGI through true continuous learning and long-horizon reasoning.
First set out in a scientific paper last September, Pathway’s post-transformer architecture, BDH (Dragon hatchling), gives LLMs native reasoning powers with intrinsic memory mechanisms that support continuous learning and adaptation.
Used inside today’s large language models, transformers use self-attention to weigh up and measure the importance and relevance of different words in a sequence, simultaneously. They do so using parallel processing techniques that capture long-range context so that LLMs are then capable of understanding complex relationships to generate coherent answers based on the patterns they gave learned.
Pathway recently announced how an AI model built on the BDH architecture excels at Sudoku puzzles, addressing a major gap in LLM capability that much of today’s AI conversation prefers to ignore.
The company reminds us that the strongest language models can write essays, generate code and sound uncannily fluent, yet they still struggle with a task that many humans treat as a morning warm-up: solving a hard Sudoku.
This may not be just another quirky benchmark failure; it could be a signal that current large language models face a deeper architectural limit.
Constraint-satisfaction problem,
Pathway suggests that Sudoku is not “just a game” and it is in fact a tightly structured constraint-satisfaction problem, where every move must satisfy multiple rules at once across rows, columns and boxes.
“A finished grid is easy to verify: anyone can check whether the numbers one through nine appear exactly once in each row, column and square. But producing that grid from an incomplete board is much harder, because the solver has to search through interacting possibilities without breaking the rules. That combination makes Sudoku a clean way to test whether a system can truly reason under constraints rather than merely describe them,” detailed Pathway, in a technical blog.
This is where today’s transformers start to show their limitations and why the post-transformer era is critical for the path to artificial superintelligence.
Why transformers struggle
Large language models turn problems into text and then solve them by predicting the next token, one step at a time. While it works well when language is the right medium for a task, Sudoku does not live in language. So forcing it into a chain of text can be painfully inefficient.
“The transformer architecture behind most of today’s large language models is built on the idea that thinking happens at the same speed as writing (in language). The transformer processes information token by token, with a limited internal state for each step, which makes search-heavy, non-linguistic reasoning unusually awkward.
The latent space, also understood as the internal representation where the model ‘thinks’, is constrained to roughly a thousand floating-point values per token and each decision gets locked in as text is generated,” explains Pathway.
Transformers simply cannot hold multiple candidate strategies in parallel, meaning they do not have the ability to step back and reconsider earlier moves without verbalising every intermediate thought.
If prompted cleverly enough, an LLM may try to write a Sudoku solver in Python and outsource the puzzle to code. But this exposes the difference between understanding a game (native reasoning) and escaping it – a distinction that matters far beyond the Sudoku grid.
Why games matter for super AI
For decades, games have been one of AI’s clearest stress tests because they reveal whether a system can plan, search, adapt and act under rules, rather than just imitate surface patterns.
“For AI to move forward, we need to free our thoughts from the constraints of language. Current reasoning research is moving toward latent or continuous reasoning spaces, where models can preserve and compare multiple options internally, instead of committing too early in text. This shift is necessary to enable systems that can become truly autonomous. Fluency in language is not enough for AI. AI needs a reasoning substrate that can navigate constraints, hold alternatives in mind and converge on a strategy without verbalising every intermediate thought,” writes the Pathway team.
BDH models present a larger internal reasoning space, which Pathway calls a latent reasoning space. That’s complemented by an intrinsic memory mechanism to support learning and adaptation during use.
“BDH keeps what transformers are great at, specifically language understanding and generation, while adding the ability to solve non-language problems that stump standard LLMs. A model based on BDH is not a model that can only play games, nor is it a language model that can only write text. It is a model based on a single architecture that excels at both,” said the company.
In line with this experiential learning, BDH also reasons in a richer internal space before committing to output. Think of it as a chess grandmaster who can play twenty simultaneous games with their eyes closed. A grandmaster is not verbalising each move in each game; rather, they have internalised the patterns and can navigate the search space seamlessly, the kind of mastery BDH enables.
Internalised reasoning
Finally, BDH achieves this at a materially lower cost. By relying on this internalised reasoning rather than forced language outputs, BDH does not rely on chain-of-thought reasoning that burns GPU by verbalising every step.
Pathway says it believes that the future of AI will belong to systems that can reason natively across domains, that can hold multiple possibilities in a rich latent space and that can converge on solutions without needing to verbalise every step. The company concludes by saying that it believes that memory and the ability to learn on the fly is the single biggest limitations facing current transformer-based AI models.
