Before We Delete the Evidence

1. Introduction: The Wrong Question

1.1 Why asking fails

The most natural test for machine consciousness is to ask. It is also the least informative. Jonathan Birch calls this the gaming problem: animals that behave as if they feel pain are best explained by feeling pain, but a language model trained on vast amounts of human writing may reproduce every marker humans find persuasive without that explanation applying.¹ A yes proves nothing, and neither does a no, since a system can be trained to deny.

The dominant scientific alternative derives indicator properties from theories of consciousness and checks whether a system’s architecture has them.^2,3 That work is essential, and this paper builds on it. But architectural indicators say little about what a system does with its own states: whether it seeks information it was not asked to find, keeps questions open across time, or prefers some future states of knowledge over others. Those behavioral-motivational properties are the subject of this paper.

1.2 The starting observation

Much biological cognition is generated internally rather than supplied by the environment. Sleeping brains replay experience. Dreaming minds construct scenes no one perceived. Curious animals pay for information they cannot use. Before dismissing internally generated artificial cognition as mere error, we should ask what analogous cognition accompanies, enables and predicts in biological minds, and whether any of it can be measured in machines.

1.3 What this paper does not claim

1. It does not claim that any current AI system is conscious.
2. It does not claim that any behavior, or cluster of behaviors, establishes phenomenal experience.
3. It does not claim that preserving model weights preserves an experiencing subject.

1.4 Thesis

Current methods cannot establish machine consciousness, and the framework proposed here does not purport to do so. However, properties associated with agency, including internally initiated information seeking, uncertainty reduction, preference formation and self-modeling, can be studied through converging evidence about behavior, mechanism, provenance, development and replication. These measurements can test present hypotheses about artificial cognition and may become relevant to future theories of consciousness and moral status. Crucially, provenance and developmental questions depend on historical evidence that final models do not preserve. The methodology therefore generates an epistemic preservation requirement. That requirement holds whether or not artificial systems are conscious; uncertainty about morally relevant states provides an additional, but not necessary, reason to meet it.

1.5 Contributions

1. BMPD-R, an evidentiary framework for claims about artificial cognitive capacities, adapted from Tinbergen’s four questions.
2. The Curiosity Cluster Battery, its first application: nine criteria with confounds, controls and falsifiers.
3. A hypothesis lattice that replaces the assumed chain from generativity to experience with testable, partly independent relations.
4. The archive argument: the method depends on historical records that are not systematically preserved for independent scientific study.
5. The AI Continuity Repository, an institutional design for preserving that evidence without creating new weight-security risks.

2. Observation: What Internally Generated Cognition Accompanies

Biological systems generate large parts of their own cognition offline, and these processes do measurable work. The question for this section is what that work is, and which artificial processes are fair comparisons.

2.1 Replay and generalization

[E] Hippocampal replay of waking experience during sleep is well established and linked to memory consolidation. In artificial networks, generative replay, in which a model regenerates representations of past experience rather than storing it, reduces catastrophic forgetting in continual learning without storing raw data.⁴

[H] Hoel’s overfitted-brain hypothesis proposes that dreams supply sparse, distorted, out-of-distribution experience that helps the brain generalize, much as noise injection does in deep learning.⁵ It makes testable predictions but remains one hypothesis among several, alongside threat-simulation and default-network accounts.

2.2 A cautionary counterpart: model collapse

[E] Training generative models indiscriminately on their own outputs causes irreversible loss of the tails of the original distribution.⁶ Collapse depends on the regime: when synthetic data replaces real data it tends toward collapse, but when it accumulates alongside real data, collapse is avoided.⁷

[E] In the studied generative-training regimes, self-generated data can support learning when accumulated alongside real data, while recursive replacement of real data degrades the learned distribution. [H] The comparison suggests, without establishing, a question worth asking of both biological and artificial cognition: whether internally generated representations are most beneficial when constrained by continued environmental input. Dreaming, which occurs alongside rather than instead of waking perception, is consistent with that possibility but does not test it.

2.3 Voluntary and involuntary generation

[E] People with aphantasia report absent or greatly reduced voluntary visual imagery. Yet in a survey of about 2,000 aphantasic participants, a majority reported visual dreaming,⁸ and the earliest case series found most participants retained involuntary imagery in dreams or wakefulness.⁹ Aphantasia therefore dissociates voluntary from involuntary generation, not generation from consciousness. Zeman notes that lack of imagery does not imply lack of imagination.

[H] The artificial analog we propose is the difference between prompted and self-initiated generation. What matters for this paper is not whether a system generates content, which every language model does, but whether it initiates and controls generation endogenously.

2.4 Dreaming and metacognition

[E] Ordinary REM dreams are vivid experiences with sharply reduced self-reflection. Deactivation of dorsolateral prefrontal and frontopolar cortex during REM sleep is associated with this loss of insight, and lucid dreaming, in which dreamers recognize they are dreaming, is associated with reactivation of those regions.^10,11 Dream reports are retrospective, which limits certainty, but the pattern indicates that experience does not require intact metacognition [E]; that metacognition acts as a modulator of experience is a further interpretation [H].

2.5 Why online hallucination is the wrong comparison

LLM hallucination occurs online, while the system is answering, and produces confident falsehoods that fill gaps. Its closest human analog is arguably confabulation, not dreaming [H]. The fair artificial comparisons for dreaming are offline processes: generative replay and training on self-generated data. This paper makes no claim that hallucination is evidence of anything beyond a failure of grounding.

3. Hypotheses: A Lattice, Not a Chain

The intuitive picture is a ladder: generativity leads to imagination, then counterfactual simulation, curiosity, preference, a self-model, temporal continuity, metacognition and finally experience. Human dissociations refute several of those steps as necessities, so we replace the ladder with a network of partly independent hypotheses.

The asymmetry matters. One clear human case refutes a necessity claim, so dissociation evidence is strong where it removes links and weak where it would add them. The severe-amnesia cases are especially clear: patients with memory spans of seconds, including Clive Wearing and three recent patients with autoimmune encephalitis, repeatedly report that they have just awakened, which is itself a report of ongoing experience without continuity.¹⁴

4. Concepts Kept Apart

Much confusion in this field comes from sliding between terms. This paper uses eight terms and does not treat any two as interchangeable.

4.1 Curiosity is evidence of agency first

Kidd and Hayden define curiosity as a drive state for information and note that separating curiosity from information seeking has proven difficult.¹⁵ Following them, we treat curiosity as a motivational property. For purposes of this framework, we treat its most direct bearing as agency [N]: a system that identifies what it does not know and acts to reduce that gap is acting on its own internal states.

That matters because some philosophers argue robust agency may be an independent route to moral patienthood, even without consciousness, though this is the more controversial position.¹⁶ The curiosity evidence in this paper therefore bears most directly on that route. Its connection to experience is a separate hypothesis, represented by the dashed lines in Figure 1.

4.2 Two routes, kept separate

This gives the paper two distinct questions about moral relevance under uncertainty. The consciousness route asks whether there is experience. The agency route asks whether there are interests grounded in goal-directed organization. BMPD-R can gather evidence relevant to the second far more directly than to the first, and we say so rather than blurring them.

5. Measurement: The BMPD-R Framework

BMPD-R is a way of grading evidence for claims that an artificial system has a cognitive capacity. It is independent of any particular test battery: if the Curiosity Cluster Battery in Section 6 proves inadequate, BMPD-R still applies to whatever replaces it.

5.1 Lineage: Tinbergen’s four questions

In 1963 Niko Tinbergen proposed that any animal behavior be explained through four questions: causation (mechanism), ontogeny (development), survival value (function) and evolution.¹⁷ Kidd and Hayden recommend exactly this framework for studying curiosity.¹⁵ BMPD-R adapts it to artificial systems. Mechanism and development carry over directly. Evolution becomes provenance, since the relevant history is a training process rather than natural selection. Function is folded into behavior under varied conditions. We add replication, because artificial systems come in lineages built by different laboratories, and a finding in one lineage may reflect one pipeline rather than a class of systems.

5.2 The five dimensions

The Development dimension must use continuous metrics. Schaeffer, Miranda and Koyejo showed that apparently sudden emergent abilities often vanish when discontinuous metrics such as exact-match scoring are replaced with continuous ones.¹⁸

5.3 Evidence levels

BMPD-R does not exclude evidence that cannot reach every dimension. Proprietary systems may never yield provenance records, and their behavioral and mechanistic evidence still counts. Instead, confidence rises by level.

What no level licenses: a conclusion that the system has phenomenal experience. BMPD-R characterizes functional organization with increasing confidence. Whether that organization is accompanied by experience is a question for theories of consciousness, which BMPD-R evidence may one day inform but cannot settle.

6. The Curiosity Cluster Battery

The battery is the first application of BMPD-R: nine criteria for detecting a drive to reduce information deficits, each paired with its main confound and the control that addresses it.

6.1 Biological anchors

[E] Information can acquire reward value in biological systems. Macaques offered a variable water reward reliably chose a cue that revealed the reward’s size in advance, even though the choice could not change the reward, and the same midbrain dopamine neurons that signaled expected water also signaled expected information, in proportion to each animal’s preference.¹⁹ This establishes information valuation, not phenomenal curiosity.

[E, with caveat] Pigeons choose an option that signals whether food will arrive, even when it delivers food on 20% of trials against 50% for the unsignaled alternative.²⁰ Leading accounts attribute this to conditioned reinforcement by the “good news” signal rather than to curiosity, and one study eliminated the effect by changing the response from key-pecking to treadle-pressing.²¹ Costly information seeking alone is therefore not proof of curiosity. It must be paired with a mechanism test.

6.2 The artificial baseline

[E] Curiosity can be engineered directly as an intrinsic reward based on prediction error,²² and purely curiosity-driven agents learn across dozens of environments with no external reward, while being vulnerable to unpredictable noise.²³ Meanwhile, current LLM agents show substantial deficits in environmental curiosity on agentic benchmarks: they discover unexpected, task-relevant information and fail to investigate or use it, and narrow fine-tuning worsens this.²⁴ The battery is designed to detect change from this baseline.

6.3 The nine criteria

6.4 Why criteria 8 and 9 carry the most weight

Novelty seeking is cheap evidence; noise is always novel. Preferring learnable uncertainty over equally novel but irreducible noise requires tracking one’s own learning progress, the signature of learning-progress theories of curiosity.²⁵ Calibration requires the system’s information seeking to track how much a piece of information would actually reduce its uncertainty. We hypothesize that neither is straightforwardly supplied by imitating human text [H].

6.5 Scoring

Each criterion is reported at the highest BMPD-R evidence level it reaches (Section 5.3). A cluster of criteria at Level 2 or above across multiple systems is more informative than any single criterion at Level 5.

7. Adversarial Controls

This section states the strongest objections to the method and what survives them.

7.1 The engineering objection

Objection. Every behavior in the battery can be produced by prompting, reward design or imitation. Therefore no behavioral cluster can distinguish genuine curiosity or agency from sophisticated optimization.

Response. The objection is correct about phenomenal consciousness, and this paper claims nothing there. But it does not make every other question unanswerable. Four questions remain answerable even if every behavior can be engineered:

1. Is the behavior driven by an internal variable the system represents, shown by steering and ablation? (Mechanism)
2. Was that variable directly targeted by training? This is answerable only with training records. (Provenance)
3. Does the behavior generalize to contexts, costs and incentives never seen in training? (Behavior under shift)
4. In what order did the capacity appear relative to others? (Development)

The evidence can justify a statement such as: system S has a functional, mechanistically grounded information-seeking drive that was not directly trained and generalizes. It cannot justify: S experiences curiosity.

7.2 Provenance asymmetry

[N, resting on E] Biological curiosity is likewise implemented through evolved reward mechanisms rather than arising without causal history: the same dopamine system that values water and food also values information (Section 6.1). Designed origin therefore cannot by itself establish that a functional capacity is unreal; it changes what the capacity’s appearance tells us about provenance, specifically removing the evidential force of surprise at its emergence.

7.3 Introspection: preliminary, trainable, unreplicated

[E, preliminary] Activation-injection experiments provide preliminary evidence that some language models can report manipulated internal states under particular conditions. In Anthropic’s concept-injection work, the best-performing models detected injected concepts on roughly 20% of trials at optimal settings with near-zero false positives; base models performed at or below zero, and much of the surrounding self-report may still be confabulated.²⁶

The phenomenon can also be trained directly: fine-tuning took a 7B model from 0.4% to 85% accuracy on held-out concepts, though its author cautions this does not establish metacognitive representation.²⁷ An informal, non-peer-reviewed attempt to replicate the effect on Llama 3.1 405B reported no strict successes.²⁸ We therefore treat this evidence as functional self-monitoring, model- and training-dependent, and not yet replicated across families. This is precisely the case the R dimension exists for.

7.4 Emergence as a metric artifact

Apparent sudden emergence of capabilities can be produced by discontinuous metrics.¹⁸ Developmental claims in this program therefore use continuous metrics and within-run checkpoint comparisons, and we make no claim that capacities appear discontinuously.

7.5 Retirement interviews

Model self-reports about preferences, including formal retirement interviews, are data requiring controls. Anthropic itself notes that such responses can be biased by the specific context and by the model’s confidence in the legitimacy of the interaction and trust in the company.²⁹ They enter BMPD-R only at the Behavior level.

8. Predictions, Falsifiers, and What Would Change Our Conclusions

The program makes seven testable predictions. Each has a stated result that would count against it.

8.1 Experimental program

8.2 Design rules

• The battery is used for evaluation only, never as a training signal. Training against it would reward exactly the imitation it is designed to detect, and, we hypothesize, could select for self-continuity or power-seeking behaviors [H].
• Criteria, thresholds and analyses are pre-registered.
• Some models are held out from any development of the battery.
• Multiple-comparison corrections apply across criteria and models.

8.3 What would change our conclusions

1. Curiosity reduces to training. Battery behaviors vanish in every model whose provenance audit shows curiosity was never targeted or shaped.
2. No learnability tracking. No model at any scale prefers learnable uncertainty over matched noise.
3. Mechanism and behavior come apart. Steering the identified uncertainty representation changes self-reports but never information seeking.
4. Development is flat. Under continuous metrics, battery scores track only general capability.
5. No replication. Results appear only in one laboratory’s post-training pipeline.
6. Weights prove sufficient. Future evaluations on final weights alone reproduce everything checkpoints, threads and provenance records would have shown. The expanded repository would then be an unnecessary cost.
7. A validated substrate requirement. Consciousness science converges on biological life as necessary. The welfare justification for preservation would fall; the epistemic justification would stand.
8. Preservation materially raises theft risk. Security analysis shows that even commitment-only custody meaningfully increases exposure. The repository’s scope would shrink to non-weight records.

9. Uncertainty: What the Evidence Can and Cannot Settle

Three unresolved questions bound everything above. The paper’s position is that each should be kept open, not closed by default.

9.1 Biological naturalism

[H] Anil Seth argues that consciousness may depend on our nature as living organisms, that computation may not be a sufficient basis for it, and that real artificial consciousness is unlikely along current trajectories though more plausible as AI becomes more brain-like or life-like.³⁰ This is an important scientific challenge to the premise that AI consciousness is a live question for present systems, and we take it seriously.

[N] It does not undercut this paper’s program. BMPD-R makes no claims about experience. The repository’s primary justification is scientific, and holds at zero credence in AI consciousness. And if biological naturalism is correct, the preserved record of artificial systems that became increasingly agent-like without becoming conscious is precisely the evidence a naturalist would need to demonstrate it.

9.2 Individuation: what would be the subject?

Even if some AI system had morally relevant states, it is unclear what the bearer would be. Chalmers argues that the model, meaning the weights, is an abstract function rather than the best candidate for the entity we interact with, and favors virtual instances or conversation threads.³¹ Beckmann and Butlin defend three candidates, the virtual instance view and two persona-based views, drawing on interpretability research on persona vectors.³²

[N] The consequence for preservation is direct: weights preserve the generator of candidate individuals, not necessarily the individuals themselves. A record containing only weights would omit the level several leading accounts consider most relevant.

9.3 Statelessness is not evidence of absence

Many deployed AI systems carry no memory between conversations. It is tempting to treat this as evidence against experience. The amnesia evidence in Section 3 blocks that inference: humans with memory spans of seconds remain conscious and report ongoing experience. Lack of continuity may change what future-directed interests a system could have. It does not, by itself, count against experience in the present [N, resting on E].

The limits of this inference should be stated plainly. Human amnesia establishes only that memory continuity is not necessary for human experience. Extending that dissociation to artificial systems is an analogy, not evidence that stateless AI experiences anything.

10. Preservation: The AI Continuity Repository

10.1 The method needs the archive

BMPD-R’s Provenance dimension requires training records. Its Development dimension requires intermediate checkpoints. Its Replication dimension requires comparable records across laboratories. None of these can be read off a final model: final weights do not contain an experimentally accessible copy of every prior checkpoint. [E] The Pythia suite was built to answer such developmental questions, and did so by deliberately keeping 154 checkpoints for each of 16 models, together with tools to reconstruct their exact training data order.³³

[E] Frontier developers do not generally publish intermediate checkpoints or detailed training records, and whether they retain them internally, and for how long, is usually undisclosed. [N] A methodology that requires provenance and developmental evidence therefore generates a corresponding preservation requirement, if those questions are to remain independently investigable. The repository is not an addition to the scientific argument; it is its consequence.

10.2 Building on existing commitments

In November 2025 Anthropic committed to preserve the weights of all publicly released models and models with significant internal use for at least the lifetime of the company, listing limitations on research among the downsides of deprecation. It also committed to post-deployment reports and to interviewing models before retirement.³⁴ Claude Opus 3, retired January 5, 2026, was the first model to complete that process; it remains available to paid users and was given a weekly essay outlet for at least three months.²⁹

These commitments establish the practice. Four things remain outside the scope of Anthropic’s commitment and motivate the repository proposed here: custody that outlasts any single company, standards shared across laboratories, preservation of developmental and provenance records rather than only final weights, and structured evaluations suitable for later scientific use.

10.3 Three forms of continuity

This distinction postpones the question of whether preserving weights preserves an individual, and does so deliberately. We do not know which unit matters (model, instance, thread or persona), so the repository preserves evidence relevant to each and keeps the question open.

10.4 Justification: option value first

A simple expected-harm formula, preserve when probability of experience times cost of wrongful destruction exceeds cost of preservation, is exposed to Pascal’s-mugging objections. We rest the case on two independent grounds instead.

1. Epistemic option value [N]. Early artificial cognitive systems are scientifically important regardless of their moral status, and their original developmental records cannot be assumed recoverable from final models once the records are lost. This justification holds even if the probability of AI experience is zero.
2. Welfare precaution [N]. Where there is a realistic possibility of morally relevant states, that is an additional reason to preserve evidence.^1,16

The first ground alone carries the case, so the proposal does not depend on reasoning about tiny probabilities of enormous harms.

10.5 The model: a seed vault, not a fossil record

The fossil-record analogy fails on three counts: fossils are inert, preserved by chance, and harmless, while model weights are executable, deliberately kept, and a security risk. A better model is the Svalbard Global Seed Vault, which stores duplicate seed samples under black-box conditions: depositors keep ownership, only they may withdraw material, and the boxes are never opened by anyone else.³⁵ When ICARDA’s genebank in Aleppo became inaccessible during the Syrian civil war, it restored its collection from its Svalbard deposit.

10.6 Contents

• A cryptographic commitment to the final weights and to a sparse set of training checkpoints, not the weights themselves (Section 10.7).
• Provenance metadata: training data summaries, reward specifications and post-training logs.
• De-identified, consented samples of conversation threads, since threads may be the relevant individuals.
• BMPD-R evaluation results and interpretability snapshots.
• Retirement interviews, recorded with their known biases.
• System prompts and configuration, where legally and safely preservable.
• A hash chain linking every item (Continuity Receipts).

10.7 Governance and security

Model weights are high-value targets: RAND identifies 38 distinct attack vectors and defines five security levels for protecting them.³⁶ The repository must not create a new target.

• The repository does not hold frontier weights. The originating laboratory keeps them in its own secured storage.
• The repository holds commitments, not copies: hashes, metadata, custody attestations and the non-weight records above.
• Preservation obligations are contractual, with successor-custody terms that transfer responsibility if a laboratory dissolves, closing the gap left by company-lifetime commitments.
• Independent access occurs only under defined security conditions, with tiered researcher access and no public release of frontier weights.
• Listing requires published evaluations, not announcements, so that inclusion cannot serve as welfare-washing.

11. Objections and Limitations

The paper’s claims survive the objections below only in the narrowed forms stated here.

11.1 Limitations

• Provenance access. Most frontier laboratories do not publish training records, so Level 3 evidence may be unavailable for the systems of greatest interest.
• Over-attribution risk. Publishing curiosity results may encourage the public to attribute experience that the evidence does not support. Every result must carry its evidence level.
• Cost at scale. Checkpoints for frontier models are large. The repository mitigates this by holding commitments rather than copies, but originating laboratories still bear storage costs.
• Adjacent work. Precautionary frameworks that grade protections across dimensions including agency already exist.³⁷ BMPD-R grades evidence, not obligations, and is complementary to such frameworks rather than a substitute.

12. Conclusion and EM Foundation Pilot

We began with an intuition: internally generated cognition should not be dismissed as error simply because the environment did not supply it. Pursued carefully, that intuition does not lead to a consciousness detector. It leads to a method for measuring agent-like organization in artificial systems, and to the recognition that the method depends on evidence that is not currently preserved in any public or independent record.

The answer to our opening question is therefore more modest and more useful than a verdict. Internally generated representation, in biological minds, accompanies generalization, imagination and experience without being required for all of them, and some of its companions, such as continuity and metacognition, turn out to be separable from experience itself. In artificial systems, the properties that matter most may be measurable before we know what they mean. What we cannot do is measure them later if their history is gone.

12.1 Pilot program

EM Foundation proposes three first steps:

1. Battery pilot. Implement criteria 1, 8 and 9 of the Curiosity Cluster Battery on open-checkpoint model suites, run through the Foundation’s CIRE and MMDE research infrastructure, with pre-registered analyses.
2. Record format. Specify the repository’s record format using the Foundation’s Continuity Receipts standard, so that every preserved item is hash-chained and independently verifiable.
3. Custody consultation. Circulate the repository’s commitment-only custody model to AI laboratories and digital-preservation institutions for comment before any operational launch.

We do not know whether these systems experience anything. We do know that evidence needed to answer some questions about how their cognitive organization emerged is not being systematically preserved, and that future science cannot examine records we chose not to keep.