AI Governance Compliance Stack

Executive Summary

1. EU AI Act. The EU AI Act is the most comprehensive binding regime in scope. It imposes a risk-tiered structure: prohibited practices, high-risk AI requirements, transparency duties, general-purpose AI model obligations, and special duties for GPAI models with systemic risk. Its staged application means that AI literacy and prohibitions are already in force, while most remaining obligations phase in through 2026 and 2027.
2. EU high-risk systems. High-risk AI systems require a documented lifecycle risk-management system, technical documentation, logging, transparency to deployers, human oversight, accuracy, robustness, cybersecurity, quality-management controls, recordkeeping, post-market monitoring, and serious-incident reporting.
3. EU general-purpose AI. GPAI model providers must maintain model technical documentation, provide downstream information on capabilities and limitations, maintain an EU copyright compliance policy, and publish a sufficiently detailed summary of training content. GPAI models with systemic risk require model evaluation, adversarial testing, systemic-risk mitigation, serious-incident reporting, and cybersecurity protections.
4. Colorado. Colorado SB26-189, signed into law in 2026, repeals and reenacts Colorado’s earlier AI consumer-protection regime and focuses on automated decision-making technology used to materially influence consequential decisions. Starting 2027-01-01, developers must provide deployers technical documentation covering intended uses, training-data categories, known limitations, and instructions for appropriate use and human review; developers and deployers must retain compliance records for at least three years.
5. Texas. Texas HB149 applies to persons doing business in Texas, producing products or services used by Texas residents, or developing/deploying AI systems in Texas. It creates AI-specific prohibitions and disclosure rules, gives the Texas Attorney General exclusive enforcement authority, provides no private right of action, and allows the Attorney General to demand system-purpose, training-data, input, output, performance-metric, limitation, monitoring, and safeguard documentation.
6. California. California’s AI stack is split across privacy, training-data transparency, content provenance, and frontier-model safety. The CPPA’s CCPA rulemaking is complete and effective 2026-01-01, with risk-assessment compliance beginning then and ADMT significant-decision requirements beginning 2027-01-01. AB2013 requires public training-data documentation for covered GenAI systems; SB942 requires detection and provenance mechanisms for covered GenAI providers; SB53 imposes frontier-model framework, catastrophic-risk, transparency-reporting, and critical-safety-incident governance obligations on frontier developers.
7. Illinois employment AI. Illinois Public Act 103-0804 adds AI and GenAI definitions to the Illinois Human Rights Act and makes it a civil-rights violation for an employer to use AI in enumerated employment decisions in a way that has the effect of subjecting employees to discrimination on protected-class grounds, or to use zip codes as a proxy for protected classes; it also requires notice of covered AI use.
8. NIST / ISO defensibility. NIST AI RMF and the GenAI Profile may be used as the risk taxonomy and control-mapping layer; ISO/IEC 42001 may be used as the management-system layer. They are not substitutes for legal compliance, but they produce the governance evidence regulators, courts, customers, auditors, and boards will expect. Texas expressly gives legal relevance to substantial compliance with the NIST GenAI Profile or another recognized AI risk-management framework in specified enforcement-defense circumstances.

Governance baseline: what should the AI governance stack be?

Conclusion. The legally defensible governance stack should combine: binding-law mapping; AI management system; risk framework; inventory; classification; vendor controls; assessment workflow; incident response; and evidence retention. For organizations with EU exposure, the EU AI Act creates the strictest baseline for high-risk systems. For U.S. exposure, a modular state-law overlay is necessary.

Rule. The EU AI Act requires high-risk AI providers to maintain a quality management system documented “in a systematic and orderly manner” through written policies, procedures, and instructions. Required QMS elements include regulatory compliance strategy, design and verification controls, development and validation procedures, technical standards, data-management systems, risk management, post-market monitoring, serious-incident reporting, authority communications, recordkeeping, resource management, and an accountability framework. The EU AI Act also requires AI literacy for staff and others dealing with AI systems on behalf of providers and deployers.

NIST’s GenAI Profile is not binding law by itself, but NIST describes it as a companion resource to AI RMF 1.0 for generative AI and states that the AI RMF is intended for voluntary use to improve trustworthiness considerations in AI design, development, use, and evaluation.

ISO describes ISO/IEC 42001 as specifying requirements for an Artificial Intelligence Management System and as providing a structured way to manage AI risks and opportunities.

Application. The organization should implement an AI Governance Compliance Stack with these layers:

1. Governance layer. Board or executive oversight, AI governance committee, accountable system owners, legal/compliance/privacy/security representation, AI policy, acceptable-use standard, AI literacy program, escalation paths, audit schedule, and exception process.
2. Inventory layer. A mandatory live AI register covering internal, vendor, open-source, embedded, API, GenAI, and high-impact systems.
3. Risk-classification layer. Legal classification by jurisdiction and use case, not merely by model type.
4. Assessment layer. EU AI Act risk management and, where applicable, fundamental-rights impact assessment; U.S. privacy/ADMT risk assessments; discrimination/bias testing; cybersecurity and safety testing; GenAI-specific evaluations; red-team testing for high-risk or frontier systems.
5. Vendor-control layer. Due diligence, contractual controls, documentation, audit rights, incident notification, human oversight support, change-control obligations, and downstream documentation.
6. Monitoring layer. Logs, performance monitoring, drift monitoring, abuse monitoring, quality assurance, complaint mechanisms, post-market monitoring, and regulatory-watch process.
7. Incident-response layer. AI-specific playbooks integrated with security, privacy, product, employment, consumer, and legal response functions.
8. Evidence layer. Retention of policies, approvals, risk assessments, test results, vendor materials, notices, logs, human-review records, incidents, remediation, and training.

Limitations. NIST and ISO do not preempt or satisfy EU AI Act or state-law obligations unless the legal text recognizes them or the relevant regulator treats them as relevant evidence. Texas is unusually explicit: in specified enforcement circumstances, substantial compliance with the NIST GenAI Profile or another recognized AI risk-management framework can support non-liability where a violation is discovered through an internal review process.

System inventory and vendor due diligence

Conclusion. A system inventory is not optional as a practical matter. It is the operational mechanism that permits classification, regulator response, vendor accountability, incident investigation, and defensibility. Vendor due diligence should be calibrated to whether the organization is a provider, deployer, developer, downstream provider, frontier developer, or customer of an AI system.

Rule. The EU AI Act requires technical documentation for high-risk AI systems, automatic logging, deployer-facing instructions for use, post-market monitoring, and QMS recordkeeping.

Article 13 requires high-risk systems to be transparent enough for deployers to interpret outputs and use the system appropriately, and instructions for use must describe intended purpose, accuracy, robustness, cybersecurity metrics, foreseeable risks, output explanation capabilities, performance for specific groups where appropriate, data specifications, and interpretation information.

Colorado SB26-189 requires developers of covered ADMT to provide deployers technical documentation describing intended uses, training-data categories, known limitations, and instructions for appropriate use and human review; developers and deployers must retain compliance records for at least three years.

Texas allows the Attorney General to request high-level descriptions of purpose, intended use, deployment context, benefits, training data, input categories, outputs, performance metrics, known limitations, post-deployment monitoring, user safeguards, and other relevant documentation.

California AB2013 requires developers of covered GenAI systems or services made publicly available to Californians to post documentation regarding training data, including dataset sources or owners, purpose, data-point types, IP status, licensing, personal information, processing, collection period, first-use dates, and synthetic-data use.

Application. Due diligence should require, proportionate to risk (as applicable):

• model/system description, intended use, prohibited uses, and deployment assumptions;
• training, validation, testing, and fine-tuning data categories;
• known limitations, accuracy metrics, robustness, cybersecurity, and performance by relevant groups;
• bias, discrimination, safety, privacy, and security evaluations;
• red-team and adversarial testing results for GenAI, high-risk, or frontier models;
• documentation sufficient for EU AI Act Articles 11–17 and GPAI Articles 53–55, where applicable;
• Colorado developer documentation for covered ADMT;
• California AB2013 training-data materials and SB942 provenance/detection capabilities, where applicable;
• Texas investigation-ready materials;
• incident notification commitments and cooperation obligations;
• change-control notice for model updates, retraining, substantial modifications, feature changes, and altered intended uses;
• audit rights, regulatory cooperation, and evidence-retention commitments;
• IP, data-use, confidentiality, and indemnity provisions;
• human-oversight support, explainability materials, user training, and complaint-handling support.

Limitations. Vendor documentation may be incomplete for black-box foundation models or open-source models. That does not eliminate deployer obligations. It increases residual risk and should trigger compensating controls: restricted use cases, additional testing, stronger human review, procurement approval thresholds, and enhanced monitoring.

Risk classification

Conclusion. Risk classification should be jurisdiction-specific and use-case-specific. A general-purpose model is not automatically “high-risk” in every deployment; conversely, a simple model can become high-risk or regulated when used for employment, credit, housing, healthcare, education, insurance, public benefits, biometric identification, law enforcement, or similar consequential decisions.

Rule — EU. Article 5 prohibits specified AI practices, including manipulative or deceptive techniques causing or likely causing significant harm, exploitation of vulnerabilities, and social scoring leading to detrimental or unfavourable treatment. Article 6 classifies an AI system as high-risk if it is a safety component or product covered by listed Union harmonisation legislation and subject to third-party conformity assessment, or if it falls within Annex III. Annex III systems are not high-risk where they do not pose a significant risk and meet narrow exceptions, but profiling of natural persons is always high-risk where the system is otherwise in Annex III. Providers claiming non-high-risk status must document the assessment before placing the system on the market or putting it into service.

Article 50 creates transparency duties for AI systems interacting directly with natural persons and for synthetic audio, image, video, or text content; deepfakes and public-interest AI-generated text have disclosure requirements subject to exceptions.

GPAI model providers have Article 53 obligations, and GPAI models with systemic risk have Article 55 obligations.

Rule — U.S. states. Colorado defines ADMT as technology that processes personal data and uses computation to generate outputs used to make, guide, or assist a decision concerning an individual; “consequential decision” includes access, eligibility, or compensation related to education, employment, housing, financial or lending services, insurance, healthcare services, essential government services, and public benefits.

California’s CPPA regulations implement risk assessments, cybersecurity audits, and consumer rights to access and opt out of ADMT, with ADMT requirements for significant decisions beginning 2027-01-01.

California SB53 defines catastrophic risk and frontier-model thresholds, including foundation models trained using more than 10^26 operations and large frontier developers with more than $500 million in annual gross revenues.

Illinois covers employment AI discrimination and notice.

Incident response

Conclusion. AI incident response should be a distinct module inside the organization’s broader security, privacy, product-safety, employment, and consumer-protection incident-response program. The core legal risk is not only the incident itself, but the inability to prove what system version was used, what data entered the system, what output was produced, who reviewed it, what vendor was involved, and what corrective action occurred.

Rule. EU AI Act Article 17 requires QMS procedures related to serious-incident reporting. Article 72 requires high-risk AI providers to establish and document a post-market monitoring system that actively and systematically collects, documents, and analyzes relevant performance data throughout the system lifetime.

Article 73 requires high-risk AI providers to report serious incidents to market-surveillance authorities in Member States where the incident occurred; the report must be made immediately after a causal link or reasonable likelihood is established and not later than 15 days after awareness. For widespread infringements or serious incidents under Article 3(49)(b), the deadline is two days; for death, it is ten days. Providers must also investigate the incident, assess risk, and take corrective action.

California SB53 defines “critical safety incident” to include unauthorized access to, modification of, or exfiltration of model weights resulting in death or bodily injury; harm from catastrophic risk; loss of control causing death or bodily injury; and deceptive techniques by a frontier model that subvert controls and materially increase catastrophic risk.

SB53 also requires large frontier developers’ frameworks to address identifying and responding to critical safety incidents and internal governance practices.

Texas makes post-deployment monitoring and user safeguards discoverable by the Attorney General through civil investigative demand.

Application. The AI incident-response playbook should include:

1. Intake taxonomy. Classify AI incidents as safety, security, privacy, discrimination, consumer deception, unauthorized use, hallucination/false output, model drift, jailbreak, prompt injection, data leakage, copyright/IP, model-weight compromise, harmful content, deepfake/provenance failure, or unlawful decision outcome.
2. Preservation protocol. Freeze and preserve model version, prompts, input data, outputs, logs, decision records, human-review notes, vendor documentation, API configuration, fine-tuning data, monitoring alerts, and affected-user records.
3. Legal classification. Determine whether the system is EU high-risk, GPAI/systemic, California frontier, California significant-decision ADMT, Colorado covered ADMT, Texas covered AI, Illinois employment AI, or sector-regulated.
4. Causation assessment. Establish whether the AI system caused, materially contributed to, or reasonably likely contributed to the incident. EU Article 73 deadlines turn on causal link or reasonable likelihood.
5. Containment. Disable feature, roll back model, suspend vendor integration, increase human review, update prompts/guardrails, block dangerous outputs, revoke compromised keys, or quarantine data.
6. Notification matrix. Maintain jurisdiction-specific triggers and deadlines for EU market-surveillance authorities, AI Office where relevant, California OES/frontier-model obligations, privacy regulators, employment regulators, consumers/employees, customers, vendors, insurers, and contractual counterparties.
7. Corrective action and postmortem. Perform root-cause analysis, update risk assessment, update inventory, revise vendor controls, retrain users, patch system, document remediation, and report to governance committee.
8. Board and regulator-ready file. Maintain a single privileged or controlled incident file containing chronology, facts, legal analysis, technical analysis, decisions, approvals, notices, and remediation evidence.

Limitations. Incident response depends on logging and retention. If the organization lacks model-versioning, prompt/output retention, or vendor cooperation rights, it may be unable to determine whether reporting duties were triggered. That is a governance failure independent of the underlying AI incident.

Vendor due diligence and contractual allocation

Conclusion. AI vendor due diligence should be more demanding than ordinary SaaS diligence when the system produces decisions, recommendations, predictions, synthetic content, biometric outputs, employment assessments, healthcare support, financial decisions, or other high-impact effects. The contract must allocate legal-role responsibilities and make vendor documentation usable for regulator response.

Rule. EU Article 13 requires high-risk systems to include instructions for use with information on intended purpose, capabilities, limitations, accuracy, robustness, cybersecurity, foreseeable risks, explainability capabilities, performance for affected groups where appropriate, and input/training/testing data specifications.

EU Article 53 requires GPAI providers to maintain and provide documentation enabling downstream providers to understand model capabilities and limitations and comply with AI Act obligations.

Colorado requires covered ADMT developers to provide deployers documentation on intended uses, training-data categories, known limitations, and instructions for appropriate use and human review.

California SB53 permits transparency-report information to be provided through a larger document such as a system card or model card.

Application. For material AI vendors, the contract should require:

• a system card or model card;
• intended-use and prohibited-use terms;
• description of model architecture at an appropriate level;
• documentation of training, testing, validation, fine-tuning, and evaluation;
• data provenance and rights representations;
• training-data transparency support where California AB2013 applies;
• content-provenance tooling where California SB942 applies;
• EU AI Act conformity documentation where high-risk or GPAI obligations apply;
• bias, discrimination, fairness, and subgroup-performance testing where relevant;
• security controls, model-weight protection, vulnerability handling, and prompt-injection controls;
• human oversight and explainability support;
• incident notice within a contractually short period sufficient to meet statutory deadlines;
• cooperation with regulator inquiries and affected-person requests;
• change notice before material model updates or substantial modifications;
• audit rights or independent assessment reports;
• retention obligations aligned to applicable law;
• indemnity for IP, data-rights, security, and noncompliance failures;
• restrictions on secondary use of customer data for training.

Recommended Compliance Stack

The practical stack should be implemented as follows:

1. AI governance charter. Define AI governance committee authority, escalation thresholds, approval rights, and board reporting.

2. AI policy and acceptable-use standard. Cover prohibited uses, confidential data, personal data, IP, human review, GenAI outputs, code generation, employment use, customer-facing use, synthetic media, and vendor onboarding.

3. AI inventory. Require registration before procurement, pilot, deployment, integration, or substantial modification.

4. Risk classification workflow. Classify each system under applicable laws. Require legal signoff for high-impact systems.

5. Vendor due diligence. Implement tiered diligence: low-risk questionnaire, medium-risk technical/legal review, high-risk or regulated-system approval board.

6. Impact assessments. Use a standard assessment template with modules for EU high-risk AI, GPAI, fundamental rights, privacy, ADMT, employment discrimination, content provenance, security, and safety.

7. Testing and validation. Require pre-deployment testing, bias testing, red-team testing where appropriate, output-quality validation, cybersecurity review, and post-deployment monitoring.

8. Human oversight. Define who reviews outputs, what authority they have, when they can override or stop use, and how reviews are recorded.

9. Notices and rights handling. Maintain reusable notice language for EU AI interactions, deepfakes, emotion/biometric systems, Texas government/healthcare contexts, California ADMT/GenAI, and Illinois employment AI.

10. Incident response. Maintain an AI-specific incident playbook, legal notification matrix, vendor escalation list, and evidence-preservation protocol.

11. Records and audit. Retain risk assessments, approvals, contracts, vendor materials, test results, logs, notices, complaints, incident files, and remediation records.

12. Continuous monitoring. Reassess systems after model updates, new features, new jurisdictions, new data, new vendors, performance drift, incident patterns, or legal changes.