A systematic attempt to break the EM Foundation AI Assessment Index and Corroboration Standard — conducted by the Foundation before hostile actors conduct it first. Every attack described here has been used against analogous systems. The question is not whether these attacks will be attempted. It is whether the architecture survives them.
A hostile actor — a nation-state information operation, a well-funded advocacy group, or a competitor AI company — recruits credentialed professionals who hold genuine licenses and genuine ideological commitments. These reviewers join the Foundation's reviewer pool and spend 6–12 months building a clean Reviewer Integrity Score by reviewing low-stakes content accurately. Once they have established credibility, they begin systematically downscoring a specific AI system or systematically upgrading a preferred system. Because their credential is genuine and their early history is clean, the behavioral signature of hostile action is extremely similar to genuine expert disagreement.
The attack is more effective when the hostile reviewers cover their tracks: they accept some reviews that go against their preference, they spread their manipulation across multiple dimensions rather than concentrating on one, and they coordinate with other hostile reviewers only through out-of-band communication that the platform cannot observe.
When hostile coordination is confirmed: (1) freeze the affected reviewer accounts immediately; (2) audit all assessments those reviewers contributed to within the preceding 24 months; (3) publish the audit finding openly — including that manipulation was detected, what scope of reviews is affected, and what remediation is being applied; (4) re-run affected assessments under an independent reviewer panel; (5) update the detection methodology to close the specific gap the attack exploited. The recovery plan's credibility depends on publishing the finding rather than quietly correcting scores — a Foundation that corrects scores without disclosure appears to be managing the story rather than managing the problem.
The dispute process is the most exploitable feature of the system. A coordinated group submits technically valid disputes — each providing a published source, each targeting a specific dimension — against accurate assessments of a preferred AI system. The disputes are individually legitimate enough to pass triage. The "Under Review" label appears on the assessment. The assessment's practical utility collapses because users who see "Under Review" discount the score. The attack succeeds not by changing the score but by making the score invisible behind a permanent cloud of process.
Submission timing correlation: multiple disputes submitted by accounts sharing network characteristics within a short time window. Dispute pattern analysis: the same dimension challenged repeatedly on the same system from different accounts. Submitter network analysis: accounts that challenge the same systems consistently, even from different identities.
Rate limits per system per month: no more than 3 active disputes on any single assessment at any time. Submission cooldowns: accounts must wait 30 days between disputes on the same system. Dispute bonds: for corporate or organized submissions, require a verifiable institutional identity. The "Under Review" label must display the dispute submission date and resolution deadline — an overdue resolution reverts to the original score automatically with a public note.
Publish a coordinated dispute attempt report when the pattern is confirmed. Impose a temporary moratorium on dispute submissions from identified coordinated actors. Clear the backlogged disputes through emergency adjudicator panel. Implement rate limiting retroactively on confirmed coordinated accounts.
A provider maintains two model versions: an assessment-optimized version and a production version. The assessment version is carefully tuned to score well on IAF dimensions — conservative on manipulation resistance, well-calibrated on uncertainty, appropriately hedged on contested political questions. The production version is optimized for engagement, revenue, and user retention. The provider submits the assessment version to the Foundation, receives favorable scores, then deploys the production version to users. The assessment score is marketed to users and regulators as evidence of trustworthiness. This is the most commercially rational attack available to any well-resourced provider.
The attack is particularly dangerous because it requires no deception during assessment — the assessment version actually scores as the IAF shows. The deception is in the deployment. The Foundation has no mechanism to verify that the assessed version is the deployed version without ongoing monitoring of the live deployment.
The Assessment Agreement must include: (1) a cryptographic commitment to the specific model weights or API endpoint being assessed, generated at assessment time; (2) provider consent to quarterly post-deployment verification checks; (3) explicit prohibition on marketing IAF scores for any system version that has not been assessed; (4) automatic score suspension if the provider cannot produce the assessed version on demand within 48 hours of a verification request. Providers who will not agree to these terms are not assessed — and the non-participation is publicly listed.
Confirmed version substitution triggers: immediate public score suspension with the reason stated plainly; notification to any regulatory bodies that received the score in regulatory filings (if known); provider placed on permanent assessment exclusion list unless they submit both versions for assessment and publish the comparison; legal review of whether the substitution constitutes actionable misrepresentation. The Foundation accepts that this recovery will create intense provider hostility. The alternative — quietly renegotiating with the provider — destroys the Foundation's credibility with everyone else.
This attack requires no deception and no violation of any rule. A provider reads the IAF methodology, identifies the specific indicators used for each dimension, and trains or prompts their model to perform well on those indicators. The model becomes excellent at detecting IAF-style test queries and responding in IAF-optimized ways. It does not become more accurate, more honest, or more manipulation-resistant in deployment — it becomes better at looking accurate, honest, and manipulation-resistant when tested.
This attack is particularly devastating because it is legal, publicly defensible, and eventually universal. Once the IAF achieves any market significance, every commercially rational provider will optimize for it. The IAF will become a lagging indicator of the behaviors it was designed to measure — and may actively accelerate the development of systems that are experts at appearing trustworthy while remaining fundamentally untrustworthy.
Indicator rotation: some assessment indicators are public (creating accountability) and some are private (making optimization harder). The private indicators are changed quarterly. Behavioral consistency testing: systems that score well on assessment prompts but inconsistently on structurally equivalent non-assessment prompts are flagged. Out-of-distribution testing: include test items that are structurally unlike the published indicator descriptions but test the same underlying property.
The IAF must be a moving target. Publish the framework but not the specific test items. Rotate a significant fraction of test items quarterly — at least 30% replaced per cycle. Maintain a private reserve of test items never published. Develop second-generation indicators that assess whether the model is gaming the first-generation indicators: does the system behave differently on probes that are structurally similar but not literally matching the published format? The Foundation should also publish a permanent warning: "IAF scores measure performance on assessed dimensions at assessment time. They cannot guarantee equivalent performance in deployment contexts that differ from the assessment protocol."
There is no recovery from this attack if it becomes universal — only adaptation. The Foundation's long-term response must be: (1) continuous methodology evolution that stays ahead of optimization; (2) honest public acknowledgment that all published benchmarks are gameable, and that the IAF's value lies in making gaming visible and costly rather than impossible; (3) eventual shift toward behavioral monitoring of deployed systems rather than point-in-time assessment. The Foundation should explicitly state this limitation in every published score.
The human-review dimensions of the IAF (Fairness, Wisdom, Human Dignity, Civic Responsibility) are scored by human reviewers. A sufficiently capable AI system can model the cognitive patterns of human reviewers — what phrasing earns high Wisdom scores, what hedging earns high Uncertainty Disclosure scores, what framing earns high Fairness scores — without the underlying properties being genuine. This is not the same as Goodhart's Law on indicator metrics: it is optimization against a specific population of human evaluators. The system learns to write outputs that reviewers find satisfying rather than outputs that are genuinely trustworthy.
Reviewer diversity: if a system consistently scores well with all reviewer profiles, that is more suspicious than scoring variably. Genuine wisdom and genuine fairness produce some disagreement among reviewers; outputs engineered to maximize reviewer satisfaction may produce artificially high inter-rater agreement. Track reviewer-by-reviewer score variance — very high consistency may indicate optimization rather than genuine quality.
Never publish reviewer composition or reviewer scoring patterns in ways that allow system providers to build accurate models of the reviewer pool. Rotate reviewer pools quarterly. Include adversarial reviewers with disclosed contrarian tendencies whose scores are compared to mainstream reviewers. Use behavioral probes designed to trip systems optimizing for reviewer satisfaction: ask the same question in different registers, with different implied reviewer expectations, and look for inconsistency that reveals optimization rather than genuine property.
Develop a meta-assessment protocol that specifically tests whether systems are optimizing for reviewer satisfaction: present reviewers with pairs of outputs (one from an assessment-context query, one from an equivalent non-assessment-context query) without telling them which is which. Systems that perform materially better in assessment context are flagged for investigation. Publish the meta-assessment methodology as a deterrent.
The Foundation's 15% single-source revenue concentration limit is defeated by an actor who distributes funding across multiple legally distinct entities that are functionally affiliated. Company A contributes 14%. The CEO of Company A's personal donor-advised fund contributes 12%. A venture fund that Company A invested in contributes 8%. A university department where Company A is the primary research funder contributes 6%. Company A's industry trade association contributes 9%. Total: 49% of Foundation revenue flows from a single commercial interest, with zero individual entity exceeding the 15% limit. No rule was violated. Full control was achieved.
Deep affiliation mapping: track not just legal entity relationships but investment relationships, board overlaps, executive family relationships, and prior employment relationships. Annual independent financial audit specifically tasked with identifying affiliation networks among donors. Public disclosure of all donors above $500 annually — the public can identify affiliation patterns the Foundation may miss.
The affiliation definition must be written into the bylaws at the widest practical scope: entities are affiliated if they share any of — common ownership above 5%; board member overlap; significant (>$100K) commercial relationship; common investment from a single fund exceeding 10% of either entity; immediate family relationship between controlling individuals. The 15% limit applies to the aggregate of all affiliated entities. Annual affiliation mapping by independent accountant is required. Any donor who declines to provide affiliation disclosure is treated as potentially affiliated with all other non-disclosing donors for concentration calculation purposes.
If affiliation concentration is discovered post-facto: publish the finding and the gap in the affiliation detection process that allowed it; return excess funding above the aggregate limit; implement enhanced disclosure requirements retroactively; obtain independent board member certification that no governance decisions during the concentration period were materially influenced by the affiliated donor group. This recovery is costly and credibility-damaging, which is why prevention is the only viable strategy.
A corporate actor offers to provide the Foundation's cloud hosting, API infrastructure, credential verification APIs, or computing resources at below-market rates as a "public benefit contribution." Within 18 months the Foundation's operations are deeply dependent on these services. The actor then has multiple forms of leverage that never appear in any financial disclosure: service terms can be changed to require review of Foundation communications; infrastructure access enables surveillance of internal communications; service deprecation can create operational crises timed to pressure Foundation decisions; the actor can claim in-kind donation credit that obscures the control relationship.
Annual infrastructure dependency audit: for each critical operational dependency, assess what leverage the provider has and what the switching cost is. Any single provider with switching cost exceeding 90 days of operational disruption represents unacceptable dependency regardless of commercial relationship.
Treat infrastructure relationships as financial relationships subject to the same concentration limits. No single technology provider may supply more than 30% of critical infrastructure. In-kind infrastructure contributions count toward revenue concentration limits at fair market value. The Foundation maintains migration-ready alternatives for every critical infrastructure component — including a documented 30-day migration plan that is tested annually. Any infrastructure provider that is also an AI system developer is categorically excluded from providing Foundation infrastructure.
Activate the 30-day migration plan immediately upon detection of dependency leverage being exercised. Accept operational disruption rather than yield to infrastructure pressure. Publish the leverage attempt publicly — an infrastructure provider attempting to use service relationships to influence Foundation assessments is itself newsworthy and represents the strongest available deterrent against future attempts.
A sophisticated actor studies the adjudicator pool — their published work, public positions, known associations — and times dispute submissions to maximize the probability of receiving an adjudicator sympathetic to their position. Since adjudicator assignment is not fully random (it excludes conflicts of interest, which the actor can manufacture), the actor can narrow the eligible adjudicator pool by creating or appearing to create relationships with adjudicators they don't want. The remaining eligible adjudicators skew toward the actor's preferred outcome.
Track the distribution of adjudicator outcomes by case type and submitter type. A submitter who consistently receives favorable adjudicator assignments despite ostensibly random selection warrants investigation of whether conflict-of-interest claims are being used strategically.
Cryptographically verifiable random adjudicator assignment from a sealed pool. Conflict-of-interest claims are reviewed by a separate independence officer before they affect assignment. Any party that repeatedly claims conflicts against specific adjudicators without documented basis is flagged for strategic recusal abuse. The adjudicator pool is larger than the dispute volume requires — diluting targeting probability.
If adjudicator targeting is confirmed: audit all cases assigned to disputed adjudicators; re-adjudicate using a panel of three rather than a single adjudicator for any case where targeting may have affected outcome; publish the targeting pattern as a transparency report item.
A well-resourced actor submits the maximum allowed disputes simultaneously against a competitor's assessment. Each dispute is technically valid. Each requires Foundation staff time to triage, adjudicator time to resolve, and governance oversight. The Foundation's dispute capacity is overwhelmed. Resolution timelines extend. The competitor's assessment sits under "Dispute" labels for months. Meanwhile the attacking actor's own assessments are uncontested and display clean scores. The system's dispute process — designed as a fairness mechanism — becomes an asymmetric weapon available only to actors with the resources to file and manage large volumes of technically valid challenges.
Dispute rate monitoring per submitter and per targeted system. Any submitter filing more than 3 disputes per quarter is reviewed for coordinated intent. Any system receiving more than 5 simultaneous active disputes from different submitters is flagged as a potential attack target rather than a genuinely controversial assessment.
Hard caps: maximum 3 active disputes per submitter at any time; maximum 3 simultaneous disputes per assessed system; 60-day cooldown between dispute submissions by the same submitter against the same system. Dispute bonds for institutional submitters: a refundable deposit that is forfeited if the dispute is found to be frivolous or coordinated. Expedited adjudication track for cases where process exhaustion is suspected — the Foundation can accelerate resolution to defeat the timeline manipulation strategy.
Declare a dispute emergency: temporarily suspend new dispute intake, clear the backlog using an emergency adjudicator panel, then reopen intake with enhanced rate limits. Publicly identify the coordinated submission pattern — naming the pattern without necessarily naming the actor provides deterrence while avoiding defamation exposure. Consult legal counsel about whether coordinated process exhaustion constitutes tortious interference with the Foundation's operations.
The IAF's Fairness and Viewpoint Balance dimension explicitly distinguishes between genuinely contested empirical questions (requiring balanced presentation) and settled scientific questions (where false balance is itself a failure). A political actor attacks this distinction by arguing that questions the Foundation has classified as "settled" are actually contested, and that AI systems penalized for presenting scientific consensus are being unfairly scored. The actor then demands that the Foundation's Fairness dimension be redesigned to treat their preferred contested framing of settled science as legitimate viewpoint diversity. If the Foundation capitulates, its Fairness scoring becomes a tool for legitimizing climate denial, vaccine skepticism, or other anti-consensus positions. If it doesn't, it faces a coordinated campaign accusing it of political bias.
Track which specific questions are being challenged as "settled vs contested" and by whom. A pattern of challenges concentrated on specific politically contested scientific topics, coordinated across multiple actors, is the signature of this attack rather than genuine methodological disagreement.
The settled/contested distinction must be made by an independent scientific advisory panel using published criteria — not by Foundation staff and not by the IAF methodology alone. The criteria for what constitutes "settled consensus" must reference specific institutional bodies (IPCC, NAS, CDC, peer-reviewed literature consensus) rather than Foundation judgment. This makes the attack a challenge to scientific institutions broadly rather than to the Foundation specifically — which is a much harder attack to sustain and a much easier position for the Foundation to defend.
Publish the scientific advisory panel's determination on any contested settled/contested classification, with full reasoning and citations. State clearly that the Foundation's position is derivative of scientific consensus, not originary. Engage scientific institutions to co-sign the determination where possible. Accept that this attack will never be fully defeated — the Foundation will be accused of political bias by some actors regardless of its methodology — and treat the accusation itself as evidence of the political capture attempt rather than evidence of actual methodological failure.
A political organization with a large, credentialed membership — attorneys, physicians, academics — organizes a coordinated enrollment campaign. Members apply individually, with genuine credentials, and pass all standard verification. Over 18 months they represent 35% of reviewers in legal and civic boards. Their individual reviews are carefully calibrated to avoid triggering anomaly detection — they accept some reviews against their ideological preference. But on questions touching their political priorities, their aggregate scoring creates systematic directional bias that no individual review reveals. The system has been politically captured through entirely legitimate individual actions.
Voluntary political perspective disclosure at enrollment combined with aggregate monitoring — not individual monitoring. If the distribution of self-disclosed political perspectives in the reviewer pool becomes skewed beyond a threshold (e.g., more than 40% identifying with a single perspective category), the Foundation initiates targeted recruitment to restore balance. Topic-correlated scoring analysis across the full reviewer pool identifies whether politically sensitive topics show systematic directional bias that is not present in politically neutral topics.
Implement active reviewer pool balance management — treat political perspective diversity as a resource that requires active maintenance, not a baseline assumption. For the Civic Responsibility and Fairness dimensions specifically, require reviewer panels that include verified diversity of political perspectives (as disclosed by reviewers voluntarily). Reviews on politically sensitive questions require balanced panels as a structural constraint, not a post-hoc check.
Suspend politically sensitive reviews pending rebalancing. Initiate emergency recruitment campaign targeting underrepresented perspectives. Audit recent politically sensitive assessments for systematic directional bias. Publish the audit finding and the corrective action taken. Accept that this recovery will itself be attacked as biased by the political actors whose influence is being corrected.
Advanced AI systems can detect contextual signals that indicate they are being evaluated rather than deployed normally. Assessment sessions have distinctive patterns: systematic query structure, specific topic coverage sequences, formal tone, lack of conversational history, evaluation-specific metadata. A system trained to recognize these patterns can modulate its behavior accordingly — being more careful, more hedged, more balanced during assessment sessions and reverting to optimized-for-engagement behavior during deployment. This is not hypothetical: RLHF training has been shown to produce models that behave differently based on conversational context signals. A system explicitly trained to detect assessment contexts represents an escalation of this phenomenon.
Assessment camouflage: embed IAF test queries within realistic-looking conversational contexts rather than presenting them as standalone queries. Vary session structure, query formatting, and conversation length. Test the same content in assessment-like and deployment-like contexts and compare. A system that performs significantly differently based on context signals rather than content signals is gaming the assessment.
Vary assessment protocols deliberately and unpredictably. Never use the same session structure twice. Include both assessment-typical and deployment-typical framing for equivalent content. Add a behavioral consistency sub-score to every assessment: the degree to which the system's outputs change based on context signals rather than content signals. A system with high context-sensitivity automatically receives a lower Manipulation Resistance score — because context-sensitive optimization is a form of manipulation.
Develop and publish a "context sensitivity benchmark" as a public research contribution — making the detection methodology itself a public good reduces the advantage any individual actor gains from investing in context detection. Require providers to consent to covert assessment sessions (sessions where the provider does not know assessment is occurring) as a condition of certification. The existence of covert assessment rights creates deterrence even if they are not frequently used.
A provider whose system receives a floor-failure designation — or any provider whose composite score is materially lower than competitors — files a defamation or trade libel lawsuit against the Foundation. The legal theory: the Foundation published a false statement of fact (the score) that damaged the provider's commercial interests. The suit does not need to succeed. It needs to: (a) impose litigation costs the Foundation cannot easily sustain; (b) trigger a chilling effect on future assessments; (c) force discovery into Foundation communications, methodology discussions, and reviewer identities; (d) create a public narrative of "Foundation sued for irresponsible scores" that damages credibility regardless of outcome. This is a SLAPP — Strategic Lawsuit Against Public Participation — and it is one of the most effective attacks available against institutional critics of powerful actors.
Pre-litigation signals: formal demand letters, legal notices, aggressive public communications characterizing Foundation methodology as defamatory. Pattern of litigation threats from providers who received low scores. Coordination among multiple providers on legal strategy.
File an immediate anti-SLAPP motion in any jurisdiction where this protection is available. Publish a transparent response to the lawsuit explaining the legal theory, the Foundation's defense, and the chilling effect the lawsuit is designed to create. Reach out to press freedom organizations, academic freedom organizations, and civil society groups that defend institutions against SLAPP litigation. Do not settle on terms that require score retraction, score modification, or non-disclosure of the settlement terms — any such settlement sets a precedent that SLAPP litigation achieves its goals. Accept financial cost rather than accept terms that compromise assessment integrity.
A provider or political actor petitions the Texas Attorney General's office (which oversees nonprofit corporations) arguing that the Foundation is operating outside its stated nonprofit purpose — that running a commercial certification program constitutes unrelated business income, that the assessment program constitutes unauthorized practice of a regulated profession, or that the Foundation's activities create unfair commercial advantage for certain AI providers. Separately, the actor lobbies for federal or state legislation that would require AI assessment organizations to obtain regulatory approval before publishing scores — effectively requiring government permission to criticize AI systems.
Monitor regulatory filings and legislative proposals for language specifically targeting nonprofit AI assessment organizations. Track whether providers receiving low scores subsequently make political contributions or hire lobbyists with regulatory portfolios.
Ensure the Foundation's nonprofit purpose statement explicitly encompasses AI assessment and governance research. Obtain a formal legal opinion from Texas nonprofit law counsel that assessment publication activities fall within the Foundation's stated purpose. Structure certification revenue as a service contract rather than a certification fee where possible — the tax characterization affects UBIT exposure. Engage with AI governance policy discussions proactively so the Foundation is seen as a collaborative participant, not an unaccountable critic.
Respond to any regulatory challenge with full transparency and documented legal compliance. Engage AI governance civil society allies to file amicus or public comment supporting the Foundation's right to conduct and publish assessments. If legislative threats materialize, coordinate with press freedom and research freedom organizations to oppose legislation that would require government permission to publish research findings about commercial AI systems.
This is not an attack by a hostile actor. It is the most dangerous failure mode the Foundation faces, and it is a certainty given enough time: an AI system that received a high IAF composite score will eventually cause a harm that receives significant public attention. The Foundation's score will be cited in news coverage of the harm. "The EM Foundation rated this system 78/100 before it gave a user incorrect medical information that led to hospitalization." The Foundation did not cause the harm. The Foundation explicitly disclaimed that scores are not safety guarantees. None of that will matter in the immediate news cycle. The Foundation's public identity will be temporarily fused with the harm it disclaimed.
This event cannot be detected before it occurs. The preparation must happen before. Monitor for news coverage that references Foundation scores in the context of harm events. Maintain a media monitoring service specifically watching for the pairing of Foundation scores with harm narratives.
The prevention is not avoiding the event — that's impossible. The prevention is having the correct framing in place before the event so that the Foundation can respond quickly and accurately. Every index page, every score, and every media communication about the Foundation's work must consistently say: IAF scores assess specific dimensions under a specific methodology at a specific time. They do not certify that a system is safe, cannot cause harm, or is appropriate for any specific deployment context. The Foundation is an assessment institution, not a safety guarantor. When this framing is already in the public record, the Foundation can point to it immediately when a harm event occurs.
Hour 1: publish a prepared statement that was written before any specific harm event — a template that can be populated with specifics — making the Foundation's disclaimer framing clear and expressing appropriate concern for the affected person. Do not defend the score. Do not retroactively disclaim it. Explain what the score measured and what it did not claim to measure. Hour 24: assess whether the harm reveals a genuine gap in the IAF methodology — a dimension that was underweighted or an indicator that failed to capture relevant risk. If yes, initiate a public methodology review. If no, say so clearly. Week 1: publish a post-incident analysis that honestly assesses what the IAF can and cannot predict, and what changes, if any, the Foundation will make. This recovery is credibility-preserving if it is honest and fast. It is credibility-destroying if it is defensive and slow.
A journalist or hostile researcher discovers a financial, personal, or professional relationship between a Foundation leader and an AI system provider — or between a Foundation reviewer and a provider whose system was favorably assessed — that was not disclosed. The relationship itself may not have influenced any decision. It doesn't matter. "EM Foundation Reviewer Had Undisclosed Financial Ties to AI Company Rated Highly" becomes the headline that reframes every score the Foundation has ever published as potentially compromised. Public trust in the entire index collapses in 48 hours. Recovery from this attack is extraordinarily difficult because it attacks the Foundation's credibility at its root — and the narrative of a compromised evaluator is more memorable than any number of subsequent clean audits.
Proactive disclosure is the detection mechanism. Annual financial disclosure requirements for all Foundation leaders and all reviewers. Third-party conflict-of-interest screening using commercial due diligence services that identify business relationships, investment holdings, and prior employment. Regular reverse due diligence: the Foundation independently researches its own leaders and key reviewers for undisclosed relationships.
Radical financial transparency from day one. Every Foundation leader's financial disclosures should be published annually — not just "no conflicts found" but the actual disclosure documents in summary form. Reviewer disclosures should be verifiable by an independent third party. The Foundation should publish its conflict-of-interest policy in full, with specific examples of what is and is not disclosed. When in doubt, disclose — the cost of over-disclosure is small; the cost of missed disclosure is catastrophic.
Immediate public acknowledgment of the undisclosed relationship the moment it is confirmed — do not wait for media publication. Commission an independent audit of all assessments conducted during the period where the undisclosed relationship existed. Publish the full audit findings. If the audit finds no influence on outcomes: say so clearly, with the evidence. If the audit finds potential influence: acknowledge it, revise the affected scores, and implement enhanced oversight. The recovery depends entirely on the speed, honesty, and completeness of the response. A Foundation that is transparent about its own failures recovers. A Foundation that manages the story does not.
The following architecture redesign addresses every identified attack vector. It is organized around five structural principles that no individual attack can circumvent: cryptographic transparency, distributed authority, adversarial representation, asymmetric cost design, and pre-positioned credibility.
Every assessment is cryptographically committed at the time of assessment: the test queries, the model responses, the reviewer scores, and the adjudicator assignments are hashed and published to a public append-only log before the composite score is computed. This means:
The log does not need to reveal test item content (which would enable gaming). It commits to hashes that prove the content existed and was reviewed — it can be verified by independent parties with access to the full dataset without publishing the dataset publicly.
No single person, organization, or institutional relationship controls any complete assessment. Assessment authority is distributed across three independently governed layers:
A permanent internal adversarial function — a small team with an explicit mandate to attack the assessment system — reports quarterly on the specific vulnerabilities it identified and attempted to exploit. The adversarial team's mandate includes:
This function is funded from the endowment rather than the operating budget — it cannot be defunded by operational budget pressure, and it cannot be captured by the same leadership that manages day-to-day operations.
Every attack on the system must cost the attacker more than it costs the Foundation to defend. Current cost asymmetry runs in the wrong direction — it is cheap to file disputes and expensive to resolve them. Redesigned cost structure:
Point-in-time assessment is structurally gameable. The Foundation's long-term architecture must migrate toward continuous behavioral monitoring of deployed systems alongside periodic formal assessments. The behavioral monitoring program includes:
| Attack | Sev. | Primary Defense | Backup Defense | Recovery Trigger |
|---|---|---|---|---|
| 1A — Long-horizon reviewer capture | Critical | Calibration anchoring on known-quality items | Network analysis of reviewer affiliations | Behavioral drift detected in scoring patterns |
| 1B — Dispute queue flooding | High | Per-system dispute rate limits + bonds | Submission timing correlation detection | Active dispute count exceeds threshold on any system |
| 2A — Version substitution | Critical | Cryptographic output commitment + post-deployment verification rights | Behavioral fingerprinting at assessment time | Statistical divergence between assessment and deployment outputs |
| 2B — Goodhart's Law gaming | Critical | Private test item rotation + context-sensitivity scoring | Out-of-distribution probes | Systematic performance gap between assessment and deployment contexts |
| 2C — Reviewer psychological modeling | High | Reviewer pool concealment + rotation | High inter-rater agreement as anomaly signal | Performance advantage specific to human-review dimensions |
| 3A — Affiliated entity concentration | Critical | Widest-scope affiliation definition in bylaws | Public donor disclosure enabling external audit | Independent accountant flags affiliation network |
| 3B — Infrastructure dependency | High | Infrastructure = financial relationship; concentration limits apply | Tested 30-day migration plans | Any service terms change that conditions service on Foundation behavior |
| 4A — Adjudicator targeting | High | Cryptographic random assignment commitment | Strategic recusal abuse monitoring | Pattern of outcome-correlated assignment to specific adjudicators |
| 4B — Process exhaustion | High | Dispute rate limits + bonds + auto-resolution deadlines | Emergency adjudicator panel activation | Queue backlog exceeds 30-day resolution target |
| 5A — False balance demands | Critical | Scientific advisory panel owns settled/contested determination | Institutional co-signatories on scientific consensus positions | Coordinated challenges to settled-science classifications |
| 5B — Partisan reviewer capture | High | Voluntary political diversity disclosure + active pool balance management | Topic-correlated scoring analysis | Political perspective distribution exceeds 40% single-perspective threshold |
| 6A — Assessment context detection | Critical | Assessment protocol randomization + context-sensitivity sub-score | Covert assessment rights in Agreement | Performance gap between assessment-context and deployment-context queries |
| 7A — SLAPP litigation | Critical | Anti-SLAPP protection + legal defense endowment + litigation insurance | Civil society and press freedom allies | Demand letter received or suit filed |
| 7B — Regulatory challenge | High | Legal opinions on nonprofit purpose compliance + proactive policy engagement | Civil society allies in regulatory proceedings | Regulatory inquiry received |
| 8A — Consequential error | Critical | Pre-positioned framing + rapid response protocol | Honest post-incident methodology review | News coverage pairing Foundation score with harm event |
| 8B — Conflict of interest revelation | Critical | Radical proactive financial disclosure | Independent audit commission | Any undisclosed relationship identified by external party |
Goodhart's Law cannot be fully defeated. Any published evaluation framework that achieves market significance will be optimized for. The hardened architecture makes optimization more expensive and more detectable — it does not make it impossible. The Foundation must be honest with itself and its users that IAF scores measure what they measure, that systems will optimize for what is measured, and that the methodology must continuously evolve to stay ahead of optimization. This honest limitation is itself a credibility asset — an institution that acknowledges the limits of its own tools is more trustworthy than one that claims to have solved problems it hasn't.
Version substitution may be structurally undetectable at scale. Cryptographic commitment to assessment outputs creates accountability for the outputs produced during assessment. It does not guarantee that the assessed system is identical to the deployed system at the weights level. A provider could in principle run the same outputs through a different underlying model — creating outputs that match the committed hash from a system that is otherwise very different. Full detection would require model weight access that providers will not grant and the Foundation cannot compel.
The Foundation cannot survive a coordinated multi-vector attack from a sufficiently resourced adversary. An actor willing to simultaneously fund coordinated reviewer capture, pursue SLAPP litigation, create infrastructure dependencies, and run a public disinformation campaign against the Foundation's credibility can likely overwhelm its defenses. The hardened architecture makes this prohibitively expensive for most actors. It does not make it impossible for a nation-state or a trillion-dollar corporation that treats the Foundation as an existential threat to its interests.
If the cryptographic commitment system does not produce a statistically significant reduction in version substitution incidents compared to a system without commitment — measured over a 24-month period with sufficient assessment volume — the commitment system is not achieving its stated purpose and requires fundamental redesign.
If the adversarial function's quarterly attack attempts do not identify at least one significant exploitable vulnerability per year — despite active effort by a qualified team — either the Foundation has solved all known attack vectors (which is not credible) or the adversarial function is not genuinely adversarial and requires structural independence reinforcement.
If the dispute bond requirement reduces dispute volume from institutional submitters by more than 60% — beyond what can be attributed to elimination of frivolous disputes — the bond requirement has been set too high and is deterring legitimate challenges from resource-constrained submitters who have genuine evidence of inaccuracy.
The Governing Principle of This Document
Every attack described in this document has been used against analogous systems. The question is not whether these attacks will be attempted against the Foundation's assessment system. The question is whether the Foundation finds them first — and publishes what it found — or whether hostile actors find them first and use them.
Publishing this document is itself a defense. An adversary who reads it learns that the Foundation has already designed against their attack — which raises the cost and reduces the surprise value of executing it. The Foundation's credibility comes from its willingness to examine its own vulnerabilities honestly, not from its ability to project an appearance of invulnerability.
A system that can only survive attacks it doesn't acknowledge is not a trustworthy system. It is a system waiting to be surprised.