SOC 2 for AI Systems: What Your Auditor Will Actually Ask

SOC 2 was written in a world where software was deterministic. You put data in, you got predictable results out. The Trust Service Criteria made sense for databases, web applications, and payment systems where "processing integrity" meant the system did exactly what it was supposed to do, every time.

Then organizations started deploying AI agents that generate different outputs for the same input, that drift in accuracy over time, and that occasionally produce confident-sounding nonsense. The SOC 2 framework did not get a rewrite to accommodate any of that.

Your auditor is still going to ask about it, though. They will ask how you version your models, how you test for bias, what happens when the model hallucinates, and how you prove that a probabilistic system processes data with "integrity." This guide covers what actually comes up during a SOC 2 audit when AI systems are in scope.

Key Takeaways

• SOC 2 Trust Service Criteria were not designed for AI, but auditors are applying them to AI systems with increasing specificity
• Processing Integrity is the most difficult criterion because of non-deterministic outputs, model drift, and hallucinations
• Auditors will ask about model versioning, bias testing, drift monitoring, PII handling in prompts, and incident response for AI failures
• You need AI-specific controls layered onto your existing SOC 2 control matrix, not a separate framework
• Evidence collection is the hardest part - start building documentation months before the audit

SOC 2 Was Not Designed for AI (But It Still Applies)

SOC 2 is built around five Trust Service Criteria (TSC) defined by the AICPA: Security, Availability, Processing Integrity, Confidentiality, and Privacy. These criteria were designed to evaluate controls around any system that processes customer data. They are intentionally broad, which is both a strength and a problem for AI.

The strength is that the criteria are flexible enough to cover AI systems without a formal update. The problem is that flexibility leaves room for interpretation. Two auditors can look at the same AI system and have very different expectations about what controls are adequate.

This is why early communication with your audit firm matters more for AI systems than for traditional software. If you wait until the audit to discover your auditor expects model cards, bias testing reports, and drift monitoring dashboards, you will have a bad time.

How the TSC Maps to AI-Specific Risks

Security (CC6, CC7). AI systems expand the attack surface. Model endpoints can be exploited through adversarial inputs. Training data pipelines introduce vectors for data poisoning. Model artifacts need the same access controls as source code. Availability (A1). GPU availability, inference latency spikes, model loading times, and third-party API rate limits all affect system availability in ways your auditor will want to understand. Processing Integrity (PI1). A probabilistic system cannot guarantee deterministic outputs. Auditors need to understand how you define and measure "correct" processing for a non-deterministic system. Confidentiality (C1). Training data may contain confidential information. Prompts may include sensitive context. Fine-tuned models can memorize and reproduce training examples. Privacy (P1-P8). Privacy controls must extend through the entire AI pipeline, from training data collection to inference-time processing to output logging. For healthcare-specific requirements, see our guide to HIPAA-compliant AI agents.

The Five Trust Service Criteria Applied to AI Systems

Security

For AI systems, security extends beyond application and infrastructure security:

• Model endpoint security. Authentication, authorization, rate limiting, plus protection against prompt injection, jailbreaking, and data extraction attacks.
• Training infrastructure access control. Permissions for training pipelines, training data modification, and production model deployment need the same rigor as production database access.
• Model artifact security. Trained models should be stored in versioned registries with access controls, integrity verification, and audit logging.
• Supply chain security. AI frameworks, pre-trained models, and third-party APIs each represent supply chain risk. Your auditor may ask how you vet these dependencies.

Availability

• Model serving reliability. GPU failures, memory exhaustion, and cold start latency all affect availability. Controls should include capacity planning, auto-scaling, inference-specific health checks, and failover mechanisms.
• Graceful degradation. What happens when your AI model is unavailable? Does the system queue requests, fall back to rules, or route to a human? Your auditor will want a defined degradation strategy.
• Third-party dependencies. If your system calls an external LLM API, document SLAs, implement timeout and retry logic, and have contingency plans for outages.

Processing Integrity

This criterion generates the most friction. We cover it in depth below.

Confidentiality

• Training data confidentiality. Data used for training does not become less sensitive because it has been transformed into embeddings. Models can memorize and reproduce training data.
• Prompt confidentiality. How are user prompts stored? Who has access to prompt logs? Are prompts used for model improvement, and have customers consented?
• Output confidentiality. An AI agent with access to multiple customer accounts needs controls to prevent cross-contamination of information.

Privacy

• PII in training data. Your auditor will ask how PII is identified, minimized, or anonymized throughout the training pipeline.
• PII in prompts and context. Users often include personal information in prompts. Controls must address how PII in prompts is handled, logged, and retained.
• Data subject rights. Can you remove someone's data from training sets upon request? You need a documented approach even if the answer is "we do not use personal data for training."

For how privacy, compliance, and governance intersect, see our AI governance framework guide.

Processing Integrity Is Where AI Gets Tricky

Processing Integrity (PI1) requires that "system processing is complete, valid, accurate, timely, and authorized." For an AI system, defining what "accurate" means requires nuance.

The Non-Determinism Problem

Traditional software is deterministic. AI models are not. Even with temperature set to zero, floating-point differences across hardware can produce different outputs. Your auditor will ask: how do you validate processing integrity for a system where outputs vary?

The answer is to define what "correct" means in probabilistic terms and demonstrate that you measure against that definition:

• Documented accuracy thresholds. For classification, you might define processing integrity as "at least 95% accuracy on validation and 92% on production data, measured weekly." For generative AI, "outputs are factually grounded in provided context at least 98% of the time."
• Defined acceptable output boundaries. What outputs are never acceptable? A medical AI agent should never recommend a medication the patient is allergic to. Document these boundaries and enforce them through guardrails.
• Statistical process control. Define control limits for key metrics. When metrics drift outside those limits, that triggers investigation.

Model Drift

Models degrade over time as production data distributions shift from training distributions. Drift is a processing integrity issue. Your auditor will expect:

• Drift monitoring in production. Track input feature distributions and output distributions. Statistical tests (PSI, KS tests) can detect drift before accuracy degrades visibly.
• Defined drift thresholds. "If PSI exceeds 0.2 for any monitored feature, a model review is triggered within 5 business days" is the kind of specific control auditors want.
• Retraining and revalidation procedures. Document who approves retraining, what data is used, how the retrained model is validated, and how it is deployed.

Hallucinations and Fabricated Outputs

From a SOC 2 perspective, a hallucinated output is a processing integrity failure. Your controls should include:

• Retrieval-augmented generation (RAG) to ground outputs in verified source data.
• Output validation through automated checks and human review for high-stakes decisions.
• Confidence scoring and thresholds. Low-confidence outputs should be routed for human review rather than delivered as answers.
• Monitoring hallucination rates through periodic sampling and evaluation.

What Auditors Actually Ask About AI Systems

These are drawn from actual audit experiences with organizations deploying AI agents in production.

Model Versioning and Change Management

"How do you version your models?" Your auditor wants every production model to have a unique version identifier traceable to its training data, configuration, and deployment approval. They expect a model registry (MLflow, Weights & Biases, SageMaker, or similar) with version history and access controls. "What is your change management process for model updates?" Model deployment should go through your change management process with documented request, risk assessment, testing (including bias and accuracy evaluation), approval, and post-deployment validation. "Can you roll back to a previous model version?" Document rollback procedures, test them, and retain previous model artifacts.

Bias and Fairness

"How do you test for bias?" Auditors expect bias testing reports showing model performance across demographic categories, with defined fairness criteria and remediation procedures. "What happens when you discover bias?" They expect a documented remediation procedure and ideally evidence that it has been followed.

Error Handling and Incident Response

"What happens when the model is wrong?" Your auditor wants to know how users report errors, how errors are investigated, and how you determine if an error is isolated or systemic. "Do you have AI-specific incident response?" Generic incident response plans often do not cover AI scenarios like months of biased outputs, prompt injection attacks, or drift below acceptable thresholds. You need documented procedures for AI-specific failure modes.

Data Handling

"How do you handle PII in prompts?" Auditors expect a data flow diagram showing how PII moves through the AI pipeline, with controls at each stage. "How do you ensure training data quality?" They expect data lineage documentation, quality checks, and provenance records. "Is customer data used for training?" Document data usage policies, customer consent mechanisms, and technical controls preventing data leakage.

Controls You Need to Have in Place

Access Control for Model Endpoints and Infrastructure

• Role-based access control for all model-related systems
• Least privilege access for model deployment
• API authentication and authorization with rate limiting
• Audit logging for every access to model artifacts and training data

Change Management for Model Updates

• Formal approval process with documented sign-off
• Pre-deployment testing including accuracy benchmarks, bias evaluations, and security assessments
• Canary deployments or staged rollouts with automated rollback triggers
• Post-deployment validation within a defined observation window

Monitoring for Drift and Anomalies

• Real-time performance monitoring (latency, error rates, throughput)
• Model quality monitoring (accuracy, precision, recall)
• Data drift detection using statistical tests
• Output distribution monitoring
• Alerting and escalation procedures when metrics exceed thresholds

Incident Response for AI Failures

• AI-specific incident classification with severity levels
• Response playbooks for each failure category
• Post-incident review with root cause analysis
• Regular tabletop exercises testing AI incident response

Data Lineage and Governance

• End-to-end data lineage from source through inference
• Data quality validation at each pipeline stage
• PII detection and handling with masking, encryption, or anonymization
• Data retention and deletion policies

For implementing these controls systematically, our AI governance services can help you build the framework before audit season.

Evidence Collection Checklist

Many organizations fail their first AI-inclusive SOC 2 audit not because they lack controls, but because they cannot produce evidence. Start collecting these artifacts now.

Model Documentation

• Model cards for every production model documenting purpose, training data, performance metrics, known limitations, bias results, and intended use cases
• Architecture diagrams showing how the model fits into the broader system
• Training configuration records

Testing and Evaluation Reports

• Pre-deployment test results for every model version (accuracy, bias, regression)
• Ongoing evaluation results from production monitoring
• Red team or adversarial testing reports

Access and Change Logs

• Access logs for model registries, training infrastructure, and inference endpoints
• Change logs for model deployments with linked approval records
• Complete deployment history with timestamps and approvers

Data Flow Documentation

• Data flow diagrams from ingestion through output delivery
• PII mapping identifying personal information at each pipeline stage
• Third-party data processing inventory with agreements and security assessments

Incident Records

• Incident logs for AI-specific incidents with full lifecycle documentation
• Remediation tracking showing previous issues have been addressed

For a real-world example, see how we implemented governance controls for an AI loan verification system in our CRFG case study.

Common Pitfalls to Avoid

Treating AI systems as regular software. Standard SOC 2 controls are a starting point but do not address drift, bias, non-determinism, and training data governance. You need AI-specific controls layered on top. Waiting until audit season. SOC 2 Type II examines a 6-12 month window. If you implement controls two months before the audit, you will not have enough evidence. Start at least 6 months early. Confusing SOC 2 with AI ethics. Your auditor is evaluating whether you have defined what "fair" means, whether you test against that definition, and whether you respond when the system falls short. Keep the conversation grounded in controls and evidence. Not involving your auditor early. Auditors have significant discretion in interpreting the TSC for AI. Have a planning conversation before the audit period begins. Overlooking third-party AI services. If you use OpenAI, Anthropic, or other third-party AI services, your auditor will want their SOC 2 reports, your vendor management procedures, and your controls for data sent to those services.

We build AI systems that pass compliance audits. ISO 27001 certified. Learn about our governance practice or get in touch.