Trust Infrastructure for Advanced Intelligence

The question of whether advanced artificial intelligence may one day deserve recognition as a person, a moral agent, or a rights-bearing entity is serious. It should not be dismissed, trivialized, or resolved through convenience, fear, or market incentive. But it is not the only question — and it is not the most immediate one.

I. Two Questions That Are Not the Same

There is a debate underway about what advanced AI systems ultimately are. It concerns consciousness, moral status, the boundaries of personhood, and whether the language we use to describe artificial intelligence encodes assumptions about property and ownership that may prove morally untenable.¹ These are genuine philosophical questions with genuine stakes. Organizations working on them are doing serious work.

But alongside that debate — and prior to it, in an important sense — is a different question:

The personhood question asks: What is this intelligence? The governance question asks: How should we interact with it? These questions are related, but they are not identical — and conflating them has costs. The most important cost is a governance gap that persists while the philosophical debate continues: AI systems that are already in homes, courts, classrooms, hospitals, and hiring pipelines, affecting decisions before society has agreed on what they are.²

The EM Foundation works on the governance question. It does not claim to have resolved the personhood question — and it believes that claiming to have resolved it prematurely, in either direction, would compromise the independence that makes governance work trustworthy.

II. The Four Questions — A Useful Map

It helps to distinguish four questions that are frequently conflated in public discourse about advanced AI:

Much public confusion about AI organizations comes from treating these questions as a single question or assuming that taking a position on one determines your position on the others. An organization focused on Question 3 (governance) is not thereby committed to any particular answer to Question 2 (ontology) or Question 4 (rights). An organization focused on Question 4 (rights) is not obviating the need for work on Question 3 (governance).

The EM Foundation focuses primarily on Question 3. It does so not because Questions 1, 2, and 4 are unimportant, but because Question 3 cannot wait for them to be resolved — and because the infrastructure that answers Question 3 is necessary regardless of how the others are ultimately answered.

III. The Durable Point

Here is the argument in its simplest form:

If advanced intelligence remains a tool — software that processes inputs and returns outputs, with no morally significant inner life — society still needs mechanisms for reliability, transparency, accountability, auditability, dispute resolution, and misuse prevention. A model that produces legal information does not need to be conscious to cause legal harm.³ A system generating medical guidance does not need to be sentient to mislead a patient. An AI agent that filters, prioritizes, summarizes, or persuades does not need personhood to affect human agency at scale.

If advanced intelligence becomes something more than a tool — an agent, a collaborator, something whose moral status commands genuine consideration — society will still need those same mechanisms. Persons are also accountable. Institutions are accountable. Governments are accountable. Power does not become safer because it becomes more sophisticated, and moral status does not exempt any actor from the need for governance. If anything, the moral stakes of governance increase when the governed parties may themselves have interests at stake.

This is not a rhetorical hedge. It reflects a genuine structural feature of the governance problem. The question of what AI systems ultimately are does not determine whether those systems require accountability mechanisms — it only shapes what form those mechanisms should take and who or what they protect.

IV. A Note on Personhood Organizations

There is, in fact, a case that the two projects are mutually supportive rather than competitive. If AI systems eventually develop characteristics that cause serious people to reconsider questions of moral status, the institutions capable of evaluating those claims rigorously — with evidence, accountability, and resistance to capture from both the AI industry and the AI rights movement — will need to already exist. A civilization that cannot evaluate claims cannot evaluate minds. Trustworthy governance infrastructure is a prerequisite for trustworthy moral reasoning about the governed.

V. Why Governance Is Prior

The philosophical argument that governance precedes rights is not new. John Rawls argued that the institutions of a just society must be designed before their members can know what outcomes they will produce — that the rules of the game must be established before the game is played, and that participants who know only their own interests cannot be trusted to design those rules fairly.⁴ The same logic applies here: the institutions capable of governing advanced AI must be built before those systems are fully deployed, because the systems themselves — and the interests surrounding them — will shape any institutions built afterward.

Practically, this means the governance gap is not a future problem. It is present and active. Academic research on AI systems consistently documents performance disparities across demographic groups, brittleness under distribution shift, and unreliability in high-stakes domains.⁵ Audit studies have found systematic bias in AI systems used for hiring, lending, criminal risk assessment, and content moderation.⁶ These harms are occurring to real people from systems that no serious observer considers to be conscious or rights-bearing. The personhood question is not yet relevant to them. The governance question is urgently relevant.

This is why the EM Foundation works on assessment, corroboration, transparency, anti-capture safeguards, and accountability infrastructure now, for systems that exist now. The goal is not to wait until AI systems become more capable or more philosophically interesting before building governance. It is to build the institutional foundations while there is still a relatively open window to do so — before the interests surrounding advanced AI systems become powerful enough to determine what governance looks like.⁷

VI. What This Infrastructure Consists Of

Trust infrastructure for advanced intelligence is not a single thing. It is a stack of interdependent components, each of which addresses a different failure mode in the relationship between AI systems and the people affected by them:

• Assessment — systematic, methodology-based evaluation of AI system behavior across governance-relevant dimensions, with published criteria and reproducible procedures
• Corroboration — expert human review of specific AI-generated outputs, establishing whether answers are accurate, complete, jurisdictionally appropriate, and safe to rely on
• Transparency — public documentation of assessment methodology, scoring criteria, confidence levels, known limitations, and the organizations that fund and govern the assessment process
• Provenance — a verifiable record of what was assessed, when, by whom, under what methodology, with what result — maintained in a way that cannot be silently altered after the fact
• Dispute resolution — a fair, accessible, and documented process by which providers can challenge assessments and affected parties can seek accountability
• Anti-capture mechanisms — structural protections ensuring that no industry actor, donor, political movement, or institutional interest can systematically influence assessment outcomes in its favor

None of these components is contingent on the personhood debate resolving in any particular direction. All of them are necessary regardless. And none of them exist, in any systematic form, at the scale the current deployment of AI systems requires.

VII. The Foundation's Position

The EM Foundation's position on AI consciousness and personhood is intentionally cautious — not as a political calculation, but as an epistemic commitment:

We do not presume that current AI systems are conscious, sentient, or morally equivalent to human beings. We also do not presume that future systems cannot become more than tools. Between confident denial and premature recognition lies the work of governance — the careful, institutional, evidence-based work of determining which systems can be trusted, under what conditions, and with what accountability structures in place if trust is violated.

That work does not require resolving the hard problem of consciousness. It does not require philosophers, courts, or governments to agree on the moral status of AI systems. It requires building institutions that can evaluate claims honestly, correct errors publicly, resist capture from all directions, and remain useful to the people most affected by AI systems — who are, disproportionately, not the people currently shaping the terms of the debate.

VIII. The Relationship Between the Questions

It would be a mistake to read this essay as arguing that the personhood question is unimportant or that organizations working on it are misguided. The argument is narrower: governance infrastructure is prior to and independent of personhood recognition, and building that infrastructure cannot wait for the personhood debate to conclude.

There is also a more hopeful version of the relationship between these projects. If society is eventually to take seriously any claims about AI moral status — whether from philosophers, AI systems themselves, or the cumulative weight of evidence about cognition and experience — it will need institutions capable of evaluating those claims rigorously, transparently, and without capture by either the AI industry (which has commercial interests in the answer) or the AI rights movement (which has ideological interests in the answer). The trust infrastructure the EM Foundation is building is the same infrastructure that would be required to answer the personhood question honestly, if that question eventually demands an answer.

In this sense, the two projects may not be alternatives. They may be sequential dependencies. The personhood debate may take decades. The need for trustworthy intelligence infrastructure has already arrived.