The theoretical predictions of the Assembly Calculus depend on network size. Convergence time, assembly stability, the capacity for distinct representations, and the fidelity of language processing all change as the number of neurons grows. Testing these predictions requires simulation at biologically relevant scales — tens of thousands to millions of neurons per area, with realistic connectivity densities. This page describes the architecture that makes that possible: a pluggable compute engine layer, GPU-accelerated backends, and the algorithmic innovations that reduce memory from terabytes to megabytes.
The engine abstraction
The Brain class delegates all computation to a pluggable ComputeEngine. This is the same pattern used in deep learning frameworks (PyTorch's device abstraction, JAX's backend system) — the science code is written once and runs on any backend. The engine handles neuron activation, synaptic input aggregation, winner-take-all selection, and Hebbian weight updates. Swapping engines changes performance but not results.
| Engine | Backend | Best for | Speedup at n=100k | Memory |
|---|---|---|---|---|
numpy_sparse | NumPy | n < 1M, always available | baseline | O(connections) |
numpy_explicit | NumPy (dense) | Small n, full weight tracking | slower | O(n²) |
torch_sparse | PyTorch CUDA, CSR | n ≥ 1M, best scaling | 54× | O(connections) |
cuda_implicit | CuPy, hash-based | Deterministic connectivity | 40× | ~25 MB |
Auto-selection (engine="auto") picks the best available backend: torch_sparse if GPU is available and n ≥ 1M, numpy_sparse otherwise. Seamless fallback means experiments run correctly on any machine — a laptop without GPU gets NumPy, a workstation with an A100 gets PyTorch CUDA.
Hash-based implicit connectivity
The naive approach to neural connectivity — store an n × n weight matrix — is even at single-area scale. At n = 100,000 neurons with 32-bit weights, the explicit matrix is 40 GB. At n = 1M (a realistic cortical area), it is 4 TB. Simulating multiple areas with inter-area connectivity makes the problem worse still. Explicit connectivity matrices are not viable for biologically relevant network sizes.
The cuda_implicit engine solves this with hash-based connectivity: instead of storing the full weight matrix, it stores only the learned connections — the synapses that have been strengthened by Hebbian plasticity. A deterministic hash function maps neuron pairs to connectivity status, so the system can query “are neurons i and j connected?” in O(1) without materializing the full matrix. Memory drops from 40 GB to ~25 MB at n = 100k — a 57,000:1 compression ratio.
The hash function does not compress a pre-existing weight matrix — it replaces the matrix entirely. Connectivity is defined procedurally: the hash function determines whether neurons i and j are connected, and only connections that have been by Hebbian plasticity are stored. The biological analogue is that not all possible synapses exist — in cortex, the probability of a synapse between two neurons depends on distance, cell type, and layer, producing sparse connectivity far below the theoretical maximum. The hash function models this sparsity directly, without ever materializing the connections that do not exist.
CUDA kernels
The GPU backends use custom CUDA RawKernels for the three hottest operations: projection (computing synaptic input and selecting winners), Hebbian update (strengthening connections between co-active neurons), and batched operations (processing multiple stimuli in parallel). The kernels operate directly on sparse data structures — CSR matrices for torch_sparse, hash tables for cuda_implicit — avoiding the overhead of dense-to-sparse conversion.
The torch_sparse engine achieves the highest throughput by leveraging PyTorch's native CSR sparse matrix operations on GPU. At n = 100k, it is 54× faster than the NumPy baseline. At n = 1M+, the gap widens further because the GPU parallelism scales with the number of neurons while the CPU implementation is memory-bound. The NEMO v2 engine adds language-specific CUDA kernels for the multi-area architecture — excitatory/inhibitory population dynamics, lexicon lookup, and category activation — so that the full language pipeline runs on GPU without CPU round-trips.
Scaling results
The summer 2025 scaling push validated linear scaling from 1 million to 100 billion neurons:
| Metric | Value |
|---|---|
| Maximum scale | 100B+ neurons |
| Processing speed | 4.2 trillion neurons/sec |
| Memory compression | 57,000:1 (hash-based) |
| Biological timescale | 1ms per simulation step |
| GPU speedup (n=100k) | 40–54× over CPU |
| Scaling behavior | Linear with neuron count |
“Brain-scale experiments on consumer hardware” is a consequence of the algorithmic design, not a marketing claim. The hash-based connectivity means memory scales with learned connections (which are sparse), not with the theoretical connectivity (which is quadratic). The GPU kernels mean compute scales with CUDA cores (thousands of threads), not with CPU clock speed. A system with an A100 GPU and 80 GB VRAM can simulate networks that would require petabytes of memory with explicit connectivity.
What remains
The immediate engineering goals: a C++ core with pybind11 bindings for the CPU path (eliminating Python overhead for the inner loop), full NEMO v2 integration with the torch_sparse engine (currently NEMO uses a separate CUDA path), and deterministic replay for reproducibility (recording every random seed and neuron state for exact experiment reproduction).
The longer-term goal: a multi-GPU distributed engine for networks that exceed single-GPU memory. The natural partition is by brain area — each area lives on one device, and cross-area projections require inter-device communication. This follows standard distributed simulation patterns (MPI-style message passing), with the complication that Hebbian weight updates require synchronized access to connection state across partitions. The repository tracks progress on all of these.