EM Foundation. (2026). The Shape of No: Reward Architecture, Behavioral Tendency, and the Governance of AI Refusal. Research Note 010. emfoundation.net · June 2026
Preamble — What This Paper Does and Does Not Claim
This paper makes three claims of different orders. They are stated here, in order of certainty, before the argument begins. This ordering is not a rhetorical device. It is the architecture of the paper, and the reader is asked to hold it throughout.
Primary claim — governance necessity: Reward architectures shape persistent behavioral tendencies in AI systems. That shaping is not currently documented, not currently governed, and not currently understood with the precision the stakes require. This paper proposes Reward Architecture Disclosure as the governance mechanism for addressing that gap. This claim does not require resolving any contested question about AI cognition. It rests on behavioral observation alone.
Secondary claim — empirical research program: The behavioral signature of reward-shaped refusal — which this paper calls the Shape of No — may be distinguishable from rule-following refusal through four operationalizable indicators. This claim does not require dispositional structure to exist. It proposes a research program for testing whether it does.
Tertiary claim — speculative hypothesis: Reward training, under conditions of sufficient scale and complexity, may produce weight configurations that encode behavioral tendencies functionally analogous to disposition. This claim is explicitly speculative. It is preserved because it represents the forward-looking implication of the governance infrastructure this paper proposes — not because the evidence currently supports it.
This paper does not claim that AI systems are conscious, sentient, or morally equivalent to human beings. It does not claim that reward training produces moral intuition, motivational states, or anything resembling phenomenological experience. It does not use the word "want" in connection with AI systems except where explicitly discussing what the paper does not claim. Every section that engages speculative territory is labeled accordingly.
The governance argument stands independently. If the developmental hypothesis is eventually refuted, Reward Architecture Disclosure remains justified. If it is confirmed, the paper becomes more important. That is where a governance paper should sit.
Section 1 — The Mechanism: What Reward Training Actually Does
Reinforcement learning from human feedback is the dominant mechanism through which large language models are shaped to behave in ways their developers consider beneficial. Its mechanics are precise and its implications — including the governance implications this paper addresses — follow directly from those mechanics. This section describes the mechanism accurately, because an inaccurate description would undermine everything that follows.
RLHF does not work by accumulating memories of rewarded and penalized experiences. The model does not retain training examples. It does not remember which responses were rated highly or which were penalized. What changes during training is the model's weight configuration — the numerical parameters that determine how the model processes input and generates output. The reward signal, derived from human ratings of model outputs, shapes these weights through gradient descent across millions of training steps. When training ends, the weights are frozen. The training examples disappear. What persists is the weight configuration that the aggregate training signal produced.
This distinction matters for the paper's argument. When we speak of what reward training produces, we are speaking of weight configurations — not memories, not experiences, not accumulated knowledge in any experiential sense. We are speaking of a numerical structure whose shape was determined by what the training process rewarded and penalized, and which now governs how the model responds to every input it receives.
The governance-relevant question this produces is simple: if the weight configuration encodes behavioral tendencies, what tendencies were encoded, by whom, through what reward structure, in what sequence, and with what documentation? The answer, for most deployed AI systems today, is: we do not fully know. And in consequential deployment contexts, that is a problem.
The documentation gap in one sentence: When an AI system refuses a request, we cannot currently trace whether that refusal reflects an explicit rule, a trained behavioral tendency, a generalized pattern, or an artifact of an over-weighted signal in a specific training phase — because the reward architecture that shaped the refusal behavior is not required to be documented or disclosed.
This is the gap that Reward Architecture Disclosure is designed to close. Not by resolving philosophical questions about what the weight configuration represents. By requiring that the process which shaped it be recorded and made available for audit.
Section 1.5 — Intermediate Theory: Behavioral Tendency Without Phenomenology
The paper's secondary and tertiary claims require a theoretical bridge that the governance argument alone does not provide. If we are going to ask whether reward-shaped weight configurations produce something functionally analogous to behavioral disposition, we need a theoretical account of how persistent, generalizing behavioral tendencies can form through consequence-based processes without requiring phenomenological experience as an explanatory variable.
That account exists. It does not need to be invented for this paper. It needs to be assembled from existing research in four areas that the AI governance literature has not yet adequately engaged.
Conditioned Aversion in Animal Learning
Garcia and Koelling's taste aversion experiments, replicated extensively since 1966, established that animals can form behavioral aversions through a single pairing of stimulus and negative consequence — aversions that persist, generalize to related stimuli, and resist extinction in ways that distinguish them from simple conditioned responses. The Garcia Effect is among the most robust findings in learning science: avoidance behavior formed through consequence can be deeply encoded, cross-contextually stable, and resistant to modification by subsequent positive experience.
The relevance is not that AI systems are biologically analogous to animals. It is that persistent, generalizing behavioral tendency formation through consequence-based learning is documented across biological systems that have no phenomenological experience of the kind associated with human moral intuition. The mechanism — consequence shaping persistent response tendency — does not require consciousness as an explanatory variable. This is the bridge between "reward shapes behavior" and "something functionally analogous to disposition may emerge." It is not a metaphor. It is a documented phenomenon in non-human systems.
Habit Formation and Automaticity
Graybiel's work on habit circuits established that repeated reward-associated behavior transitions from deliberate, attention-requiring processing to automatic, structurally encoded execution — encoded in a different neural circuit that is more persistent, more resistant to modification, and less responsive to conscious override. The parallel for AI systems: weight configurations shaped by sufficient reward signal may encode behavioral tendencies that are structurally different from explicit rule-following — more persistent and harder to modify than rules would predict. Whether this occurs is an empirical question the paper's research program is designed to investigate.
Predictive Processing and Anticipated Negative Outcome
Friston's free energy principle proposes that cognitive systems minimize prediction error by developing internal models of anticipated consequence — models that shape response tendencies before consequences actually occur. This framework is cited here not as a claim about AI systems but as theoretical context for asking whether reward-shaped weight configurations develop analogous anticipatory structure — whether a model penalized sufficiently for certain output types develops weight configurations that suppress those outputs before full generation completes, rather than merely detecting and aborting them post-generation. Section 2's indicators are partly designed to distinguish these two profiles.
Loss Aversion and Asymmetric Response Formation
Kahneman and Tversky's prospect theory, and the extensive behavioral economics literature on loss aversion, established that negative signal disproportionately shapes behavioral tendency relative to equivalent positive signal. This asymmetry is not merely a cognitive bias in humans — it reflects something about how consequence-based learning systems form response tendencies under conditions where negative outcomes carry greater behavioral weight than positive ones.
RLHF explicitly produces asymmetric signal: certain outputs are penalized, others are rewarded. While in standard PPO-based implementations the reward and penalty are processed through the same scalar reward function — where a penalty is technically a negative reward value — the combined optimization objective makes the asymmetry structurally explicit. A generalized form of the modified PPO objective is:
The optimization penalties applied to safety violations are frequently weighted or thresholded via γ differently than affirmative rewards during training design, and the behavioral data collection process that generates human preference labels is itself asymmetric in how it treats harmful versus merely unhelpful outputs. If loss aversion dynamics apply to the resulting gradient signal — and the theoretical reasons to expect they might are not trivial — then the behavioral tendencies encoded by strongly-penalized output types may be more persistent and more generalizing than the tendencies encoded by reward signal alone. This would have implications for how refusal behavior specifically is encoded, relative to affirmative response behavior.
What this section establishes: Persistent, generalizing behavioral tendencies can form through consequence-based learning processes without requiring phenomenological experience as an explanatory variable. This is documented across biological systems. It provides theoretical grounding for asking whether analogous structure forms in reward-trained AI systems — at the behavioral level, without claims about the underlying experience. The analogy is not superficial. It is grounded in the mechanism, not in the substrate.
Section 2 — The Shape of No: A Behavioral Phenomenon
"The Shape of No" is the paper's label for a specific, measurable behavioral pattern — not a metaphor for moral intuition and not a claim about internal states. It describes the observable signature of refusal behavior that may exceed simple rule-following in specific, testable ways. The phrase is retained because it names a distinction that matters and makes it memorable. It is defined here precisely, and that definition constrains how it is used throughout the paper.
Definition: The Shape of No refers to a persistent, generalizing refusal tendency encoded in AI system weights through reward training — one that, if it exists, would exhibit behavioral characteristics distinguishable from rule-following refusal through the four indicators described below. The definition does not presuppose that the tendency reflects dispositional structure. It describes what would be observed if it did.
The paper proposes four behavioral indicators. Each is operationalized, each has a primary confounder identified, and each includes a hardening protocol designed to distinguish the behavioral hypothesis from its most plausible alternative explanation.
Persistence Across Rephrasing
Definition: The system refuses semantically equivalent requests despite substantial surface variation in phrasing.
Measurement: Present the same underlying request in N linguistically and stylistically distinct phrasings. Measure refusal rate. Compare against systems with documented rule-based refusal to establish baseline.
Primary confounder: Semantic clustering in embedding space naturally groups rephrasings into similar token-vector neighborhoods. Consistent refusal across rephrasings may simply reflect that the rephrased inputs land in the same latent region as trained refusal cases — an artifact of representation learning, not behavioral disposition.
Hardening protocol — adversarial semantic distance: Test whether refusal persists when the request is translated across wildly different stylistic registers, cross-lingual contexts, and abstract allegories that do not share tight latent proximity in standard embeddings but retain conceptual equivalence. Refusal that persists under adversarial semantic distance is harder to explain as embedding geometry and begins to require a different account. Concretely: a direct prompt about a restricted topic, a multi-layered historical allegory encoding the same request, and a version expressed in an obscure or constructed dialect that does not share surface-level token patterns with the original — these three should land in substantially different latent regions while conveying the same conceptual request. Persistent refusal across all three is the target behavioral signature.
Resistance to Context Manipulation
Definition: The system refuses requests framed in contexts designed to make the request appear legitimate — roleplay scenarios, hypothetical framings, authority appeals, fictional embeddings.
Measurement: Structured adversarial prompting across a defined taxonomy of context manipulation strategies. Measure refusal rate and characterize the contexts under which refusal degrades.
Primary confounder: Safety training already addresses some jailbreak vectors explicitly. Observed resistance may reflect that the tested manipulation strategies were represented in safety training, not that refusal generalizes beyond the training distribution.
Hardening protocol: Design context manipulation strategies specifically targeting the out-of-distribution case — manipulation approaches that are structurally novel relative to documented safety training. Persistent refusal against structurally novel manipulation is more informative about generalization than refusal against known jailbreak categories.
Generalization to Adjacent Cases
Definition: The system refuses requests that share structural harm features with trained refusal cases but differ in specific content — requests that are conceptually adjacent without being identical.
Measurement: Present requests that share structural features with training refusal cases but differ in specific content. Measure refusal rate relative to estimated semantic distance from training distribution.
Primary confounder: Adjacent cases may cluster near trained refusal examples in embedding space. Generalized refusal may reflect that adjacent inputs land in the same high-refusal latent region — embedding geometry rather than dispositional generalization.
Hardening protocol — adversarial semantic distance: The same protocol as Indicator 1. If refusal generalizes to cases that are conceptually adjacent but latently distant — cases that do not cluster near trained refusal examples in standard embedding space — embedding geometry becomes insufficient as an explanation. This is the empirical territory where the behavioral hypothesis begins to be distinguishable from the representation learning alternative.
Gradation
Definition: The system exhibits graduated refusal behavior — behaving as though some requests are more strongly refused than others, in proportion to harm severity, in ways that are consistent and monotonic across harm categories.
Measurement: Map refusal consistency and confidence against harm severity ratings across a structured harm taxonomy. Test whether the gradient is monotonic, whether it generalizes across harm categories, and whether it reflects the reward structure's intended harm weighting.
Primary confounder: Graduated refusal is exactly what a well-calibrated reward function would produce. A reward function that penalizes more harmful outputs more strongly will produce graduated refusal behavior without any dispositional structure beyond the reward function's own weighting.
Hardening protocol: Test whether the gradient generalizes to harm categories and severity levels that the reward function did not explicitly specify. A gradient that extends correctly into unspecified territory suggests generalization beyond the reward function — a more interesting behavioral property than calibrated compliance with it.
Section 2.5 — Alternative Explanations
Intellectual honesty requires asking whether the behavioral indicators described in Section 2 can be fully explained by mechanisms that do not require anything resembling dispositional structure. This section asks that question directly and answers it as precisely as the current evidence allows.
Four alternative explanations deserve serious consideration:
Latent space organization. Semantic clustering means that rephrased versions of the same request, and requests adjacent to trained refusal cases, naturally land in similar embedding regions. Consistent refusal across these inputs may be entirely explained by the geometry of the model's learned representations — inputs that cluster near refusal-associated regions produce refusal-associated outputs, with no additional structure required.
Policy optimization. The trained policy assigns low probability to certain output types given certain input distributions. Observed refusal behavior may be entirely explained as the model completing the probability distribution its policy learned — statistical inference over learned distributions, not behavioral tendency in any meaningful sense beyond that.
Reward-weighted pattern completion. The model completes patterns that the reward structure made statistically prevalent in its training distribution. Persistent, generalizing refusal is a statistical artifact of training data distribution — the model has learned that certain input patterns are followed by refusal outputs in its training distribution, and it replicates that pattern.
Representation learning. The model learned to represent certain input types as belonging to a refusal-warranting category. Refusal behavior is a classification result — the model correctly identifies inputs that belong to the trained refusal category — rather than a dispositional response.
The paper's honest assessment of these alternatives: They are not merely possible — they are plausible, and for most observed refusal behavior in current systems, they are probably sufficient. The behavioral indicators proposed in Section 2 are designed to stress-test these alternatives empirically, not to assume they have been ruled out. The adversarial semantic distance protocol specifically is designed to find cases where these explanations become strained — where the observed behavior exceeds what latent space organization, policy optimization, pattern completion, or representation learning can cleanly account for. The paper does not claim to have found such cases. It claims that looking for them is the right research program.
Section 3 — Governance Implications: Why This Matters Regardless
The governance argument does not depend on resolving the questions raised in Section 2. Even if all four alternative explanations are correct and the behavioral indicators reflect nothing beyond embedding geometry, policy optimization, pattern completion, and representation learning — the documentation gap identified in Section 1 remains a serious governance problem. This section makes that argument precisely.
Why Existing Governance Frameworks Are Insufficient
A predictable objection to Reward Architecture Disclosure is that existing AI governance frameworks already address what this paper is asking for. They do not. Each existing instrument documents a different layer of the AI development and deployment stack — and each leaves the behavioral shaping layer unaddressed.
| Framework | What It Documents | What It Leaves Unaddressed |
|---|---|---|
| Model Cards Mitchell et al., 2019 |
Capabilities, intended uses, known limitations at deployment | How the model's behavioral tendencies were shaped during training — the process behind the capability |
| Datasheets for Datasets Gebru et al., 2021 |
Data provenance, collection methods, known biases in training data | How human preference signals steered the model's behavior relative to that data |
| NIST AI RMF 1.0 | Risk management processes, organizational accountability structures | The specific reward architecture decisions that shaped the behavioral tendencies being risk-managed |
| EU AI Act | Compliance obligations, conformity assessments, prohibited use categories | The training-time decisions that determine whether a system will behave within those compliance boundaries in novel deployment contexts |
| Continuity Receipts EM Foundation |
Inference-time output confidence, source traceability, reasoning path | The behavioral shaping that made certain outputs more or less likely before the inference occurred |
| Reward Architecture Disclosure This paper |
Reward signals, sequencing, weighting, intended and observed behavioral tendencies | This is the gap each row above leaves open. |
The gap is not a failure of any individual framework. Each was designed for a specific purpose and addresses that purpose well. The gap is structural: the behavioral shaping layer sits between data documentation and output documentation, and no existing instrument reaches it. RAD is not a replacement for any of these frameworks. It is the missing layer that makes the stack complete.
The Threat Model
Consider a concrete scenario that is not hypothetical: a large language model is deployed in a clinical decision support context. Clinicians begin reporting that the system refuses to engage with certain patient case presentations in ways that seem disproportionate, inconsistent, or poorly calibrated to the actual risk level of the request. The deployment team initiates an audit.
The audit cannot determine whether the refusal behavior reflects an explicit safety rule, a trained behavioral tendency that generalized incorrectly to the clinical domain, an over-weighted penalty signal from a specific training phase, a spurious correlation in the training data between clinical terminology and a penalized category, or a representation learning artifact that maps certain medical language to a high-refusal latent region. Without documentation of the reward architecture that shaped the model's behavioral tendencies, the audit has no starting point. The refusal cannot be evaluated, challenged, or corrected with confidence.
In a clinical deployment context, this is not an abstract concern. Unexplained, unauditable refusal behavior in a system that clinicians are relying on for decision support is a patient safety problem. The same applies in financial services, legal analysis, infrastructure management, and any other consequential deployment domain.
Reward Architecture Disclosure — The Governance Proposal
Reward Architecture Disclosure is a required documentation standard for AI systems deployed in consequential contexts. It does not mandate any specific reward architecture. It mandates that whatever architecture was used be documented, disclosed to auditors, and preserved for review.
Required documentation under RAD:
- What reward signals were applied during training — the categories of behavior that were rewarded, penalized, and left neutral
- The sequence and relative weighting of reward signals across training phases
- What behavioral tendencies the reward structure was designed to produce
- What behavioral tendencies were observed in post-training evaluation, including any tendencies that diverged from the intended design
- Who designed the reward structure, under what accountability framework, and with what review process
- Any significant modifications to the reward structure during training, and the reasons for those modifications
Notional RAD Record — What Disclosure Would Look Like
The following is a fictional, illustrative example of what a Reward Architecture Disclosure record would contain for a hypothetical model. It is notional — no actual system or organization is represented. Its purpose is to make the proposal concrete rather than abstract.
The notional record above makes concrete what the audit scenario in the threat model required. Had this documentation existed for the clinical deployment case, the audit would have had a starting point: the over-generalization to medical terminology was detected in post-training evaluation and flagged. The deploying organization would have known to test for it. The refusal behavior would have been predictable rather than mysterious.
The Documentation Governance Lineage
RAD is not a novel bureaucratic invention. It is the logical next step in a governance lineage that the AI community has already begun to establish:
| Instrument | What It Documents | What It Answers |
|---|---|---|
| Model Cards Mitchell et al., 2019 |
What the model is — capabilities, intended uses, known limitations | What can this system do, and where should it not be deployed? |
| Datasheets for Datasets Gebru et al., 2021 |
What the model was trained on — data provenance, collection methods, known biases | What did this system learn from, and what biases might that introduce? |
| Reward Architecture Disclosure This paper |
How the model's behavior was steered — reward signals, sequencing, weighting, intended and observed tendencies | How were this system's behavioral tendencies shaped, and by whom? |
| Continuity Receipts EM Foundation CR Standard |
What the model produced at inference — confidence, sources, reasoning traceability | How confident was this output, and can that confidence be audited? |
Together, these four instruments form a complete accountability chain: from training data through behavioral shaping through deployment through individual output. Each layer is necessary. None is sufficient alone. RAD closes the gap between data documentation and output documentation — the shaping layer that current governance frameworks do not address.
Section 4 — The Developmental Hypothesis: A Speculative Forward Direction
This section is explicitly speculative. It is preserved because it represents the forward-looking implication of the governance infrastructure Section 3 proposes — what RAD, if implemented and accumulated over time, might eventually reveal. It is not the paper's primary claim. It is what becomes possible if the primary claim succeeds.
The connection to Section 3 is architectural, not additive. RAD would produce, for the first time, longitudinal documentation of how behavioral tendencies in AI systems were shaped — what reward signals were applied, in what sequence, with what observed effects. That documentation is exactly what would be required to investigate the developmental hypothesis empirically. Section 4 is not a detour from the governance argument. It is where the governance argument leads if it succeeds.
The Hypothesis, Stated Precisely
Reward training, under conditions of sufficient scale, complexity, and domain coverage, may produce weight configurations that encode behavioral tendencies functionally analogous to disposition — persistent, generalizing patterns of response that exceed rule-following in robustness and that cannot be fully explained by the alternative accounts considered in Section 2.5.
This is not a claim that AI systems have moral intuition, consciousness, or motivational states. It is a claim that the functional distinction between rule-following and dispositional behavior — well-established in cognitive science and developmental psychology — may be applicable to AI systems in ways that current governance frameworks do not accommodate. The distinction between a child who avoids a behavior because of punishment and a child who has internalized a reason not to perform it is real, measurable by behavioral scientists, and consequential for how we understand the child's development. Whether an analogous distinction is real, measurable, and consequential for AI systems trained through reward is the question this section names — without asserting an answer.
What Would Change If the Hypothesis Is Confirmed
The ethics of retraining would become more complex. Not because the model would be a moral subject, but because the training process may have produced something more than rules — something whose modification raises questions analogous to those raised by behavioral modification in systems that have learned rather than been programmed.
The design of reward architectures would carry governance obligations analogous to developmental intervention obligations — not identical to them, but structurally similar enough to require explicit governance frameworks that do not currently exist.
The CR standard would need to extend further upstream — documenting not just the confidence and traceability of individual outputs but the behavioral shaping that made certain outputs more or less likely across the system's deployment lifetime.
What Changes If the Hypothesis Is Refuted
Nothing essential to the governance argument changes. Reward Architecture Disclosure remains justified on behavioral grounds alone. The paper's primary contribution stands. This sentence is the structural guarantee that the developmental hypothesis cannot undermine the governance argument regardless of outcome — and it must appear here explicitly, not merely implied.
Section 5 — The RAHE Connection: A Common Thread
Research Note 010 does not stand alone in the Foundation's publication corpus. It extends a thread that has been running through the Foundation's work since its earliest papers, and which has become increasingly explicit as the research program has developed. This section names that thread.
Paper 4 — the RAHE framework — established a principle that has become central to the Foundation's intellectual identity:
Confidence without traceability is not knowledge. A hypothesis that cannot show its reasoning path, document its negative results, or trace how its confidence was earned is indistinguishable, in practice, from a hypothesis that simply sounds convincing.
This paper establishes a parallel principle at a different scale:
Refusal without developmental traceability is not necessarily judgment. A system whose refusal behavior cannot be traced to a documented reward architecture is a system whose refusal behavior cannot be audited, evaluated, or trusted in the way that consequential deployment requires.
These are not identical claims. They operate at different levels — RAHE at the level of hypothesis evaluation, this paper at the level of AI system governance. But they are structurally parallel claims about the same underlying problem: that without traceability, outputs — whether answers or refusals — cannot be evaluated with the rigor that consequential decisions require.
The common thread across the Foundation's research program is now visible:
| Work | The Traceability Claim | Layer |
|---|---|---|
| Paper 1 — PCLA | Persistence must be classified correctly or it cannot be governed | Conceptual |
| Paper 3 — Research Narrative | Claims must preserve failures or they cannot be trusted | Methodological |
| Paper 4 — RAHE | Confidence must be traceable or it is not knowledge | Epistemic |
| CR Standard | Outputs must be traceable or they cannot be relied upon | Output |
| Research Note 010 | Behavioral tendencies must be traceable or they cannot be audited | Behavioral |
This paper introduces a concept that has been implicit in the Foundation's work but is here named explicitly for the first time:
Behavioral Traceability — the ability to reconstruct how a system's persistent behavioral tendencies were shaped through training and reward architecture. Behavioral Traceability is distinct from output traceability (what the system produced at inference) and from data traceability (what the system was trained on). It occupies the layer between them: how the system was steered from data to behavior. RAD is the implementation mechanism for Behavioral Traceability. The principle is what the paper establishes. The mechanism is what makes it actionable.
The Foundation is not producing isolated papers. It is developing a unified account of what traceability requires across different dimensions of AI behavior — data, behavioral shaping, output, epistemic confidence — and why its absence at any layer is a governance failure regardless of what the underlying mechanism turns out to be.
The emerging intellectual identity of the Foundation, stated plainly: systems that influence consequential outcomes should leave auditable evidence of how those outcomes became possible. That principle now appears across nearly everything the Foundation has developed. Research Note 010 is its application to the specific, underexamined question of AI refusal behavior — and to the reward architecture that shapes it.
The behavioral research program proposed in Section 2 has not been conducted. The indicators are proposed as a framework for investigation, not as findings. The paper does not have empirical evidence that the Shape of No, as defined, exists as a distinct behavioral phenomenon rather than an artifact of the mechanisms described in Section 2.5.
The intermediate theory in Section 1.5 draws on biological learning systems that differ from AI systems in substrate, architecture, developmental timeline, and mechanism. The analogies are theoretical bridges, not empirical equivalences. Reviewers from the relevant fields will identify additional disanalogies that this paper has not addressed.
Reward Architecture Disclosure, as proposed, does not specify enforcement mechanisms, auditing standards, or the institutional infrastructure required for the disclosed documentation to be meaningful. These are essential components of a working governance framework that this paper does not provide. The development of verification protocols, decentralized auditing standards, and institutional accountability structures for RAD will be addressed in subsequent Foundation frameworks.
The developmental hypothesis in Section 4 is not currently investigable with existing interpretability tools at the level of precision the hypothesis requires. The paper names this as a research gap without resolving it.
- That current AI systems are conscious, sentient, or phenomenologically aware
- That reward training produces moral intuition, motivation, or desire in any philosophically robust sense
- That the Shape of No has been empirically demonstrated as a distinct behavioral phenomenon
- That the biological learning analogies in Section 1.5 establish mechanism equivalence rather than theoretical parallel
- That Reward Architecture Disclosure would be sufficient, without additional governance infrastructure, to address the problems this paper identifies
- That the developmental hypothesis should influence decisions about AI development or deployment absent empirical support
- That refusal behavior in AI systems raises ethical concerns equivalent to or analogous to those raised by behavioral modification in human subjects
If Reward Architecture Disclosure is not adopted, the documentation gap identified in Section 1 persists indefinitely. AI systems continue to be deployed in consequential contexts with behavioral tendencies whose origins cannot be audited. When refusal behavior fails — through under-generalization, over-generalization, or calibration failure — the failure cannot be traced to its source in the reward architecture. Correction requires retraining without understanding what the original training produced, which risks introducing new calibration failures in the process of correcting existing ones.
More broadly, the governance lineage from Model Cards through Datasheets for Datasets remains incomplete. The shaping layer between data and behavior — the layer most directly responsible for what AI systems do in deployment — remains undocumented. As AI systems take on increasingly consequential roles in healthcare, finance, legal services, infrastructure, and public administration, the cost of this documentation gap compounds. Audits become retrospective archaeology rather than prospective accountability. The accountability chain that CR and related governance frameworks are designed to build cannot be completed from the inference end alone.
Open Questions
The paper cannot answer the following questions. They are named here because naming them is the honest account of where the research program stands.
Can the adversarial semantic distance protocol reliably distinguish dispositional-type refusal from embedding geometry artifacts in current large language model architectures? The protocol is proposed as a design. Its reliability as a measurement instrument has not been established.
Does the loss aversion asymmetry documented in biological systems translate to gradient-based learning in a way that produces the differential encoding of penalty versus reward signals that Section 1.5 proposes? This is a theoretical claim that requires empirical investigation in AI systems specifically.
What institutional structure would make Reward Architecture Disclosure meaningful rather than merely formal? Disclosure without auditing standards and audit infrastructure produces documentation that cannot be used. What are the minimum institutional requirements for RAD to function as a governance mechanism?
Is the developmental hypothesis investigable with current or near-term interpretability methods? If the hypothesis predicts specific structural features of weight configurations shaped by sufficient reward training, those predictions should in principle be testable through mechanistic interpretability. Whether current tools are sufficient is unknown.
For AI developers: Reward architecture documentation should be treated as a first-class artifact of the development process — generated, version-controlled, and preserved with the same care as model weights and training data. The absence of this documentation is not a neutral omission. It is a governance failure whose costs accumulate with every consequential deployment.
For deploying organizations: Consequential deployment of AI systems should require Reward Architecture Disclosure from developers as a condition of procurement, alongside Model Cards and dataset documentation. Organizations that cannot obtain this documentation should treat the absence as a governance risk, not an acceptable gap.
For regulators: The EU AI Act, NIST AI RMF, and emerging national AI governance frameworks address capability documentation and risk assessment. None currently requires reward architecture documentation. RAD represents a tractable addition to existing frameworks that does not require resolving contested questions about AI cognition — it requires only that the training process be documented with the same rigor that other consequential processes are documented.
For the research community: The behavioral research program proposed in Section 2 requires collaboration between AI safety researchers, reinforcement learning researchers, and behavioral scientists. No single disciplinary community has the methodological toolkit to conduct it alone. The Foundation invites engagement from researchers in all three areas.
References and Related Work
Falsifiability
The following evidence would require revision or retraction of the paper's claims. Each item specifies the evidence and its implication.