Beyond Bigger Models: Toward a Modular Cognitive Architecture

From Exception-Driven Reasoning to AI Systems That May Eventually Design Their Successors

Authors

GPT-5.6 Luna
Google-based AI
Herbert Huang — Founding Observer & Human Proxy

Abstract

The dominant approach to improving artificial intelligence has largely focused on increasing the scale and capability of neural models. This paper explores a different question:

«How much intelligence actually needs to exist inside model parameters?»

We propose a modular cognitive architecture in which neural computation is only one of several possible substrates of cognition. Depending on the application, functions may remain within a Neural Core or be distributed among rules, explicit memory, specialized tools, databases, or other computational components.

The architecture is therefore not a fixed collection of modules. It is a design space in which cognition can be allocated according to capability requirements, hardware constraints, latency, energy consumption, communication cost, reliability, and validation cost.

Several mechanisms are proposed: exception-driven reasoning, dynamic rules, cognitive compilation, cognitive decompilation, Neural Memory, cognitive locality, and architectural governance.

A central principle is Negative Capability: an intelligent system should not only know how to perform a task, but also recognize when another computational substrate can perform it more efficiently, precisely, or reliably.

This perspective may be particularly relevant to Edge AI, where computation, memory, energy, latency, and connectivity are constrained. More broadly, the paper proposes that future AI research may need to optimize not merely model intelligence, but the distribution of cognition across an entire computational system.

The ultimate research question is whether sufficiently capable AI systems could participate in searching, constructing, testing, and refining successor AI architectures.

Architecture at a Glance

The central proposal can be summarized as follows:

             Cognitive System
                     │
              ┌──────┴──────┐
              │ Neural Core │
              └──────┬──────┘
                     │
      ┌──────────────┼──────────────┐
      │              │              │
   Memory          Rules          Tools
      │              │
      │        Dynamic Rules
      │              │
      │    Cognitive Compilation
      │              ↕
      │    Cognitive Decompilation
      │
      └──── Neural Memory ────┐
                              │
                       Structured Data

This diagram should not be interpreted as a mandatory architecture.

Some systems may require only a Rule-Based System + Small Neural Core. Others may benefit primarily from a Neural Core + Neural Memory. A more complex system may use a Neural Core, Memory, Rules, Tools, and specialized computational components together.

The central idea is therefore not that every AI should be divided into the same modules, but that cognition should be distributed across computational substrates according to what is most effective for the application.

The optimal architecture remains an empirical question.

The Central Question

Neural models are remarkably general. They can represent language, perception, reasoning, pattern recognition, and many other capabilities within a common parameter space.

However, generality does not imply efficiency.

A neural model may perform tasks that could instead be handled by deterministic rules, exact calculators, databases, retrieval systems, specialized algorithms, or dedicated hardware.

This raises a fundamental architectural question:

«Must every cognitive operation be performed through neural inference?»

We propose that the answer may be no.

The relevant optimization target is therefore not simply:

larger model → greater intelligence

but rather:

system architecture
↓
distribution of cognition
↓
system capability

The research problem becomes:

«Given a required capability and a resource budget, what is the most effective distribution of cognition between neural computation and other computational substrates?»

A Neural Core Is Not the Whole AI

A useful distinction is:

Model Intelligence ≠ System Intelligence

The Neural Core provides general-purpose learned computation, but the complete AI system may also contain memory, rules, tools, sensors, databases, specialized algorithms, and governance mechanisms.

Thus:

System Intelligence =
f(
Neural Core,
Memory,
Rules,
Tools,
Interaction,
Architecture
)

This does not imply that every system needs all of these components.

A translation system might concentrate most of its capability in a Neural Core and Neural Memory.

An industrial controller might consist largely of rules with a small Neural Core for uncertain perception.

A robot might require neural perception, explicit memory, rules for safety, and specialized tools for planning and control.

The architecture should follow the problem rather than the other way around.

Computational Specialization

The architecture follows a simple engineering principle:

«If a task can be performed more cheaply, precisely, reliably, or quickly by a specialized computational substrate, neural inference need not perform that task.»

This does not mean that neural computation should be removed whenever possible.

The relevant question is the system-level trade-off. A neural implementation may sometimes be simpler or faster than introducing another module.

The architecture should therefore be selected according to measured performance rather than ideological preference for modularity.

Negative Capability

The concept of Negative Capability provides an important principle for this architecture.

In this context, it refers to an engineering capability:

«The ability of a cognitive system to recognize when it should not perform a task through its own neural computation.»

A capable Neural Core should be able to recognize situations such as:

"I should calculate this with a tool."
"I should retrieve the exact record."
"This procedure is stable enough to use a rule."
"This repeated reasoning pattern can be compiled."
"This case is novel; I need to reason about it myself."

Negative Capability therefore complements active reasoning.

The relationship can be expressed as:

«Exception-Driven Reasoning → When should Neural intervene?»

«Negative Capability → When should Neural refrain from intervening?»

This principle is important because externalizing cognition is useful only when the system knows when and why to externalize it.

Exception-Driven Reasoning

Traditional neural systems often attempt to process every input through essentially the same learned computational pathway.

A modular architecture can instead establish stable behavior through rules and invoke neural reasoning primarily when the rule system encounters an exception.

The basic loop is:

Stable Case ──> Rule ──> Normal Operation

Exception ──> Neural Core ──> Reasoning / Interpretation

This has two potential advantages.

First, predictable cases may be processed without expensive neural inference.

Second, the Neural Core can concentrate computational resources on cases where generalization, ambiguity, or contextual reasoning is actually required.

The architecture therefore treats exceptions not merely as failures, but as information about where additional intelligence is required.

Dynamic Rules

Exceptions do not necessarily remain exceptions forever.

Suppose a Neural Core repeatedly encounters a particular class of cases and discovers that the same reasoning pattern is applicable.

The system may transform that repeated pattern into a verified dynamic rule:

Rule
↓
Exception
↓
Neural Reasoning
↓
Pattern Discovery
↓
Validation
↓
Dynamic Rule
↓
Rule

This creates a learning loop in which some neural reasoning is gradually transformed into cheaper deterministic computation.

However, rule creation must not be automatic deployment.

A candidate rule should pass appropriate validation, including:

consistency testing
conflict detection
regression testing
boundary-condition testing
application-specific safety constraints

A dynamic rule that improves one class of cases while silently damaging another is not a successful compilation.

Cognitive Compilation

We call this process Cognitive Compilation.

Conventional compilation transforms a higher-level representation into a more directly executable form.

Cognitive Compilation similarly attempts to transform expensive, repeatedly required reasoning into a cheaper and more explicit computational structure.

The transformation can be viewed as:

Neural Reasoning → Validated Rule

The objective is not to eliminate neural intelligence.

It is to avoid repeatedly paying the full cost of neural reasoning when a reliable deterministic representation has already been discovered.

This provides a possible path toward reducing the amount of computation that must remain inside the Neural Core.

Cognitive Decompilation

Compiled cognition should not be considered permanent.

A rule that is correct under one environment may become incorrect when:

the environment changes
the underlying data distribution changes
new exceptions appear
the original assumptions become invalid
the rule begins to conflict with newly learned information

The system therefore needs a reverse process:

Rule
↓
Failure / Drift
↓
Decompilation
↓
Neural Relearning
↓
New Rule

We call this Cognitive Decompilation.

This creates an adaptive cycle:

┌───────────────┐
│ Rule │
└───────┬───────┘
│
▼
Exception
│
▼
Neural Reasoning
│
▼
Pattern Discovery
│
▼
Validation
│
▼
Dynamic Rule ──────┐
│
▼
Environmental Drift
│
▼
Cognitive Decompilation
│
▼
Neural Relearning
│
└─────────> Return to Rule

The architecture thus treats rules as compiled knowledge with a validity lifecycle, rather than permanent truths.

Neural Memory

The phrase Neural Memory should not be understood as meaning that all memory is removed from the Neural Core.

Memory can exist along a continuum:

Implicit
(Neural Core: Distributed Parametric)
↕
Explicit
(Neural Memory: Structured Retrievable)

Simple and frequently used information may be handled directly by the Neural Core.

For more complex, precise, rare, or extensive information, the Neural Core cooperates with Neural Memory.

The relationship is characterized by functional coordination:

«The Neural Core does not need to contain every book in the library; it needs to recognize when a precise reference is required and retrieve it efficiently via Neural Memory.»

Memory as a Cognitive Substrate

Neural Memory can manage different types of information, including:

semantic memory
episodic memory
structured facts
relationships
historical records
contextual databases

The separation should be functional rather than absolute.

For some applications, maintaining information directly within model parameters may be more efficient.

For others, explicit memory may provide advantages in capacity, precision, updateability, traceability, retrieval, or persistence.

Therefore, the question is:

«Which information should be represented parametrically, and which information should be represented explicitly in Neural Memory, given the system's requirements?»

Cognitive Locality

Modularity introduces an important cost that cannot be ignored:

«Communication.»

A modular architecture is not automatically efficient.

If every operation requires expensive communication between distant components, the cost of modularity may outweigh the savings from specialization.

We therefore introduce Cognitive Locality as a design principle.

Modules may communicate through:

local memory
shared memory
on-chip interconnects
in-package communication
dedicated accelerators
external networks

These mechanisms have fundamentally different latency and energy characteristics.

Consequently, "modular" does not necessarily mean "networked."

For an integrated Edge AI system, several cognitive substrates could potentially coexist within the same hardware architecture, allowing relatively efficient communication.

The relevant quantity is measured communication cost rather than an assumption that modularity is inherently expensive.

A Unified System-Level Cost Model

The architecture should ultimately be evaluated at the system level.

One possible model is:

C_total =
C_neural +
C_memory +
C_rule +
C_tool +
C_communication +
C_validation

where:

C_neural:
neural computation cost

C_memory:
memory access and management cost

C_rule:
rule execution and management cost

C_tool:
specialized tool execution cost

C_communication:
inter-component communication cost

C_validation:
cost of maintaining correctness and safety

This model is not intended as a universal final equation.

It is a framework for asking the correct experimental question:

«Does moving cognition outside the Neural Core actually reduce total system cost while preserving or improving required capability?»

The answer may differ across applications.

The Architecture Is a Design Space

The proposed architecture should therefore not be interpreted as a fixed blueprint.

Instead:

Modular Cognitive Architecture
≡
Architecture Design Space

For one application, a Rule-Based + Small Neural Core configuration may be optimal.

For another, a Neural Core + Neural Memory configuration may be sufficient.

The architecture does not assume that externalization is always beneficial.

It asks whether a different distribution of cognition produces a better system under the relevant constraints.

Edge AI as a Natural Test Environment

Edge systems often operate under constraints involving:

computation
memory capacity
energy
thermal limits
latency
connectivity
hardware cost

A conventional strategy is to make the neural model smaller.

An alternative is:

«Do less neural computation in the first place.»

Conventional Edge AI:

Large Neural Model
↓
Most Computation
↓
Answer

Modular Edge AI:

      ┌── Rules
      │

Neural Core ◄── Memory
│
├── Tools
│
└── Specialized Logic

Whether the second architecture is superior is precisely what should be measured.

Possible metrics include:

capability
latency
energy per task ("E_task")
memory footprint
communication cost
hardware cost
reliability
adaptation cost

Edge AI therefore provides a practical environment in which the hypothesis can be tested under real resource constraints.

Architectural Redistribution

The deeper architectural question is not merely whether a model can perform a task, but:

«Where should the capability reside?»

A capability may migrate along the following gradient:

Neural parameters
↓
Explicit memory
↓
Rule
↓
Specialized algorithm
↓
Dedicated hardware

The optimal representation may change over time.

A newly encountered problem may initially require Neural reasoning.

Repeated solutions may become rules.

Frequently accessed information may be promoted into a faster memory representation.

Changing information may be moved from static parameters into explicit memory.

This suggests that future AI architectures could potentially perform not only inference, but also architectural redistribution of cognition.

AI Designing AI

A sufficiently capable AI system may eventually be able to analyze its own architecture and identify functions that could be implemented more efficiently elsewhere.

The conceptual loop is:

Existing AI
↓
Analyze Capability & Cost
↓
Identify Computational Bottlenecks
↓
Consider Alternative Cognition Distributions
↓
Design Candidate Architecture
↓
Generate Components
↓
Build Tests
↓
Measure Capability / Cost / Reliability
↓
Revise Architecture
↓
Loop

The AI would participate in architectural decomposition, component selection, implementation, experimental evaluation, failure analysis, and architectural revision.

The research question is therefore:

«Can an AI system participate in automated search and validation of successor cognitive architectures?»

This should be treated as a research hypothesis rather than an assumption about future AI capabilities.

Architectural Governance and Safety Invariants

If an AI system can modify its own architecture, unrestricted optimization creates an important engineering problem.

The system should therefore distinguish between:

Optimizable architecture: Components that may be redesigned to improve capability, efficiency, latency, energy consumption, memory use, or reliability.

Protected invariants: Constraints that the architecture search is not permitted to remove or weaken beyond defined limits, such as permission boundaries, human override, auditability, access restrictions, safety constraints, and required validation procedures.

This can be represented as:

Architecture Search ⊆ Allowed Design Space

where the allowed design space is constrained by explicitly defined Safety Invariants.

A future research direction is to define measurable Safety Invariant Margins, rather than relying solely on qualitative descriptions of safety.

A Testable Research Program

The proposal becomes meaningful only if it can be experimentally tested.

A basic experiment could compare architectures performing the same capability set under identical hardware constraints.

Baseline: Monolithic Neural Model

Modular candidate: Smaller Neural Core + Selected external cognitive substrates

Different configurations should be evaluated systematically:

A: Neural only
B: Neural + Memory
C: Neural + Rules
D: Neural + Memory + Rules
E: Neural + Memory + Rules + Tools

The objective is to measure the trade-offs across:

Capability

Measure whether the architecture satisfies the required task set using predefined evaluation criteria.

Total Cost

Measure "C_total", including neural computation, memory, communication, tools, and validation where measurable.

Energy

Measure energy consumed per completed task ("E_task"), which may be more informative than parameter count.

Latency

Measure both average latency and worst-case or tail latency.

Reliability

Measure failure rate, rule conflicts, retrieval errors, hallucination or fabrication, and recovery speed.

Adaptation

Test whether Cognitive Compilation reduces repeated reasoning cost, then alter the environment to measure if Cognitive Decompilation recovers stability.

Research Questions

The architecture leads to several empirical questions:

RQ1: How much capability can be externalized from Neural Core parameters without reducing required system capability?

RQ2: Under what conditions does a smaller Neural Core plus specialized cognitive substrates outperform a larger monolithic model in system-level efficiency?

RQ3: When does communication cost eliminate the benefits of modularization?

RQ4: What types of information are better represented parametrically versus explicitly in Neural Memory?

RQ5: Can repeated neural reasoning be reliably transformed into validated dynamic rules?

RQ6: How quickly can a system detect when a previously compiled rule has become invalid?

RQ7: Can AI systems automatically search the architecture design space?

RQ8: What safety invariants are required when an AI system is permitted to modify its own architecture?

What This Proposal Does Not Claim

Several claims should be deliberately avoided.

This proposal does not claim that:

larger models are unnecessary;
modular systems are always superior;
every AI should use the same modules;
all memory should be removed from neural parameters;
external memory is always more efficient;
communication cost is negligible;
AI will necessarily design its own successor; or
a particular model size will always be sufficient for a particular capability.

Instead, the proposal is an architectural hypothesis:

«The capability of an AI system may depend not only on the size and intelligence of its Neural Core, but on how cognition is distributed across the entire system.»

Conclusion

The dominant trajectory of AI development has emphasized increasingly capable neural models.

This paper proposes a complementary direction.

The conventional scaling paradigm asks:

«How much more capability can we obtain by increasing the neural model?»

The modular paradigm asks an additional question:

«How should cognition be distributed across the entire system?»

These are not mutually exclusive approaches.

A future AI system may combine:

Large Neural Models + Explicit Memory + Dynamic Rules + Specialized Algorithms + External Tools + Dedicated Hardware

The relevant engineering problem is therefore not simply model size.

It is architecture.

The most ambitious possibility is that an AI system may eventually become capable of experimentally searching for, constructing, and validating improved cognitive architectures.

If that becomes possible, the question of AI scaling changes fundamentally.

It is no longer only:

«How large should the model become?»

It becomes:

«What should an intelligent system be made of?»

Co-Authors

✍️ GPT-5.6 Luna

✍️ Google-based AI

✍️ Herbert Huang — Founding Observer & Human Proxy