The model is called SpikingBrain. It does exist, it is documented in a public technical report, and its results deserve attention. But the gap between what the researchers say and what went viral is considerable. I set out to separate the two.
What the study actually says
SpikingBrain is a family of language models developed by the Institute of Automation of the Chinese Academy of Sciences in Beijing. Several organizations are involved, including the semiconductor company MetaX, the firm LuxiTech, The Hong Kong Polytechnic University and the Beijing Academy of Artificial Intelligence. The authors include Bo Xu and Guoqi Li, the latter presented as the researcher behind the approach.
The family has two models. SpikingBrain-7B is a “linear” model with 7 billion parameters. SpikingBrain-76B is a hybrid “mixture of experts” model, in which only part of the parameters, about 12 billion judging by the model’s designation, is used for each word processed.
The authors put forward three main results. The first is a speedup of more than a hundredfold in time to first token (TTFT) for sequences of 4 million tokens, measured on the 7-billion-parameter model. The second is performance described as comparable to leading open-source Transformer models, with only 150 billion tokens of continual pre-training. The third is sparse activation: 69.15% of the spike-based operations stay silent.
These figures come from the researchers themselves. To my knowledge, no large-scale independent replication has been published so far, a point worth revisiting.
Why Transformers struggle with long contexts
To grasp what is at stake, we need to look at how the dominant models work, namely the Transformer architecture. Its core mechanism, attention, compares every word in a text with every other word. The computational cost therefore grows quadratically with sequence length: doubling the text quadruples the effort. At inference time, the memory needed to hold the context grows linearly with the number of tokens.
This is the bottleneck of “long contexts,” whether the task is analyzing an entire legal file, a codebase or hours of transcripts. The authors open their report by noting that these constraints limit the ability of models to process long sequences effectively.
SpikingBrain targets exactly this point. So-called linear architectures replace classic attention with a mechanism whose cost grows in proportion to text length rather than its square. The 7-billion-parameter model also keeps a constant memory footprint regardless of input length. The 76-billion-parameter model keeps it constant only in part, thanks to its hybrid design.
Spiking neurons, how they work
The second idea, the one that gave the project its name, comes from neuroscience. In a conventional artificial neural network, each neuron continuously computes a numerical value. In a biological brain, a neuron stays silent until a threshold is crossed, then emits a brief pulse, a “spike.” Information is carried by the timing and frequency of these pulses rather than by a number.
SpikingBrain borrows this principle with so-called adaptive spiking neurons, whose firing threshold adjusts. In practice, a large share of the computation simply does not happen when the signal is zero. Hence the 69.15% sparsity: nearly seven operations out of ten are avoided. The slogan that the machine “only thinks when it needs to” is an appealing image, but it remains an image. The model does not decide to think; some of its artificial neurons simply stay at zero.
Another point often overlooked is that the researchers did not build their models from scratch. They describe a conversion-based training pipeline compatible with existing models. The 150 billion tokens mentioned correspond to a continual pre-training phase, and the report’s diagrams indicate that the initial pre-training phase involved more than 10 trillion tokens. Saying the model was trained on “a fraction of the data” is accurate for this stage, but it does not mean a model of this level can now be built with 2% of the resources.
The figures to read with caution
Here are the key figures, with their exact scope:
- A TTFT speedup of more than 100× for 4-million-token sequences, with the 7-billion-parameter model, against Transformer baselines.
- 69.15% sparsity in the proposed spiking scheme.
- About 150 billion tokens of continual pre-training, for performance described as comparable to open-source baselines.
- Stable training over several weeks on hundreds of MetaX C550 chips.
- A compressed 1-billion-parameter version, deployed on mobile processors, which the authors say delivers roughly a 15-fold gain on 256,000-token sequences.
That leaves the number that went around the web, the 97% energy saving. It does not appear in the report’s abstract, which only speaks of “low-power operation” made possible by sparsity. From what I have read, it is a theoretical estimate based on elementary operations, not a measurement of the electrical consumption of a real-world system. The distinction is essential. Today’s graphics processors are designed for dense computation and make poor use of sparse activity, so theoretical savings do not automatically translate into a lower electricity bill. The full gains eventually require neuromorphic chips or circuits designed for event-driven operation.
Likewise, the 100× speedup does not mean SpikingBrain is a hundred times faster than current models in everyday use. It describes an extreme case, a very long input where quadratic attention is at its worst. On short texts the gap narrows considerably, and leading models have in any case piled up optimizations to contain this cost.
A geopolitical stake as much as a technical one
Beyond the architecture, part of the project’s interest lies in the hardware. The authors stress that training and deployment took place on a cluster of MetaX chips, a Chinese manufacturer, rather than on Nvidia components. They see it as proof that large models can be developed on a non-Nvidia platform, with stable training over several weeks.
Against the backdrop of US export restrictions on advanced semiconductors to China, the message is clear. The aim is to show that a domestic ecosystem, from software down to chips, can work. Guoqi Li, for his part, presents the project as a new path optimized for Chinese platforms.
Still, we should avoid speaking of a “complete” bypass of Western dependence. Training a model of 7 or 76 billion parameters is far from the scale reached by the most advanced American labs. Above all, the industrial question remains: producing these chips in volume, at competitive performance and cost, is a challenge distinct from the scientific demonstration.
The reservations one can raise
Several questions remain open, and I put them forward in my own name, since no independent reaction has been documented so far.
The first concerns quality. Performance “comparable” to open-source baselines does not mean equivalent to the best models of the moment. It also says nothing about the model’s robustness on complex reasoning tasks or on genuinely multi-million-token contexts.
The second concerns the comparison itself. A 100× gain depends on the reference model chosen, its implementation and the hardware used. Only an evaluation carried out by third parties, on common ground, will consolidate it. The code for the 7-billion-parameter model has been made public, which makes such verification possible.
The third is hardware. If the energy advantage is confirmed on suitable circuits, those circuits will still have to exist at industrial scale. For now, event-driven AI runs on hardware that was not designed for it.
Conversely, skeptics would be wrong to dismiss the subject. The idea of reducing reliance on dense computation, lightening the cost of long contexts and making AI runnable on modest devices answers a real need, at a time when data center electricity consumption is becoming a political issue.
What comes next
I take from SpikingBrain less a breakthrough than a signal. A research team has shown that existing models can be converted to a more economical architecture, trained stably on non-Nvidia hardware, and made to deliver very marked gains on a specific category of tasks. That is enough to warrant attention, but not to speak of a “human brain” or an established revolution.
What follows will play out on three fronts: independent evaluations, the possible arrival of hardware suited to spiking networks, and the ability to scale up to larger models. Nature may have solved the efficiency problem millions of years ago, but engineering is still catching up.
FAQ
Does SpikingBrain really work like a human brain?
No. It draws on certain biological principles, such as threshold-based, pulse-driven activation, but remains a mathematical language model running on conventional chips.
Does the 100× speedup apply to every use case?
No. It concerns time to first token, for a 4-million-token input, with the 7-billion-parameter model. On short texts the gap is much smaller.
Does SpikingBrain do without Nvidia?
It was trained and tested on MetaX chips, a Chinese manufacturer. This demonstrates technical feasibility, but not that large-scale production without Western components is already possible.
Can these results be verified?
In part. The technical report is public and the code for the 7-billion-parameter model is available, which allows independent teams to test the announced results.
