Neural Computers refer to an emerging computing paradigm that proposes internalizing computational processes, memory management, and input/output operations as learned components within neural network parameters and states. Rather than relying on external execution environments, operating systems, or discrete computational units, neural computers aim to implement the full stack of computing—including interface simulation and control logic—directly from learned patterns. This concept aims to blur the boundary between machine learning models and the computational infrastructure they depend upon, enabling end-to-end learned computation where algorithmic operations and system behaviors emerge from neural network training rather than being explicitly programmed.

The core idea behind neural computers draws from decades of research into learned algorithms and differentiable computing. Rather than treating neural networks as statistical pattern-matching systems with computation delegated to external CPU/GPU runtimes, neural computers propose embedding algorithmic procedures directly into model weights and activations. This represents a fundamental departure from the von Neumann architecture that has dominated computing for decades, where memory, processor, and I/O are kept separate.

The concept relates to algorithmic alignment, where neural networks learn to internally implement algorithmic procedures needed to solve problems. Research into chain-of-thought prompting demonstrates that language models can learn to decompose problems into sequential reasoning steps, effectively internalizing computational procedures, suggesting that computation can emerge from learned representations¹⁾.

Sufficiently large neural networks can learn to perform computation, memory access, and data I/O as part of their internal processing. The key innovation is that the system learns not just a single algorithm or function, but the meta-structure of computation itself—how data flows through a system, how state is maintained and updated, and how external requests are processed and responded to.

Neural computers would fundamentally reorganize how models interact with computational resources. Rather than a strict separation between model weights and external memory systems, a neural computer architecture would integrate memory operations into the model's learned parameters. This resembles concepts explored in memory-augmented neural networks, where differentiable attention mechanisms enable models to learn read and write operations over internal memory structures²⁾.

The technical implementation would involve several key components:

- Learned State Representation: The model encodes system state implicitly within its weights and activations, rather than in external memory - Learned Memory Operations: Instead of calling external memory allocation and access functions, the model learns attention patterns that implement memory management internally - Interface Simulation: The system learns to respond to queries and commands as a traditional computer would, simulating behavior learned from input-output traces - Computational Procedures: Algorithmic steps traditionally expressed in imperative code become emergent behaviors of iterative neural processing - Control Logic: Decision-making and branching logic emerge from learned patterns - I/O Abstraction: Data input and output formatting could be learned through end-to-end training rather than explicit protocol implementation

A Neural Computer operates by learning from I/O traces—recorded sequences of inputs and corresponding outputs from a functioning computer system. During training, the model learns to predict and simulate the correct output given any input sequence. This approach treats computing as a sequence modeling problem, where the neural network becomes capable of emulating computer behavior end-to-end. The model absorbs the functions of a computer—such as memory, state, and execution—into its own latent dynamics, effectively becoming the environment that holds state and produces the next interface response.

In-context learning provides empirical evidence that neural networks can learn to internalize such procedures and behaviors, suggesting that the architectural principles underlying neural computers are viable. This paradigm collapses computation, memory, and I/O into a single learned runtime, where the neural network effectively becomes the computer itself rather than running on one³⁾, as proposed by Meta AI and KAUST⁴⁾.

• Hierarchical Memory and Context Management
• Cognitive Memory Architectures
• The Neuron
• AI Superfactory
• Mem0