Soc2 mapping

Executive Summary¶

The Agent Governance Toolkit provides runtime governance infrastructure that addresses SOC 2 Type II controls across Security, Availability, and Processing Integrity criteria. The toolkit's strongest coverage is in Security (CC1–CC9), where the policy engine, RBAC, cryptographic identity, execution rings, and audit logging provide a defense-in-depth enforcement stack. Availability (A1) is well-supported through circuit breakers, SLO enforcement, and chaos testing primitives. Processing Integrity (PI1) benefits from deterministic policy evaluation, Merkle audit chains, and input validation — though several audit chain implementations have integrity defects.

Confidentiality (C1) has partial coverage through egress controls, PII pattern detection, cryptographic identity, and credential redaction on the MCP gateway audit path — but still lacks comprehensive audit-log PII minimization, at-rest encryption, and key rotation. Privacy (P1–P8) is the largest gap area: the toolkit detects only 2 built-in PII patterns (SSN, credit card) on tool inputs, has no consent management, no data subject access request support, and no retention enforcement. Organizations deploying this toolkit in SOC 2 scope must supplement Privacy controls with external tooling.

Important: This mapping documents what the toolkit provides as infrastructure. SOC 2 Type II requires evidence of operating effectiveness over a review period — policies followed, controls monitored, exceptions investigated. The toolkit provides the enforcement mechanisms; the operating procedures, organizational policies, and evidence collection are the deployer's responsibility. "Partial" coverage means the toolkit provides building blocks but does not satisfy the control independently.

Coverage Summary¶

0 of 5 criteria fully covered. 4 partially addressed. 1 gap.

Security (CC1–CC9)¶

The system is protected against unauthorized access, unauthorized disclosure of information, and damage to systems that could compromise the availability, integrity, confidentiality, and privacy of information or systems and affect the entity's ability to meet its objectives.

What Exists¶

CC1: Control Environment¶

CC5: Control Activities¶

CC6: Logical and Physical Access Controls¶

CC7: System Operations¶

CC8: Change Management¶

CC9: Risk Mitigation¶

# CC6.1 in action: Role-Based Access Control
from agent_os.integrations.rbac import RBACManager, Role

rbac = RBACManager()
rbac.assign_role("data-analyst", Role.READER)
rbac.assign_role("ops-agent", Role.ADMIN)

# Reader cannot write — denied by permission check
assert not rbac.has_permission("data-analyst", "write")   # False
assert rbac.has_permission("data-analyst", "read")         # True

# CC7.1 in action: Governance Audit Logging
from agent_os.audit_logger import GovernanceAuditLogger, JsonlFileBackend

audit = GovernanceAuditLogger()
audit.add_backend(JsonlFileBackend("governance_audit.jsonl"))
audit.log_decision(
    agent_id="finance-bot",
    action="transfer",
    decision="deny",
    reason="Policy violation: amount exceeds role limit",
)

Reference implementation: The finance-soc2 example demonstrates CC6.1, CC6.3, CC7.1, CC7.2, CC7.3, and CC8.1 using real Agent OS governance APIs with role-based separation of duties, approval workflows, and immutable audit trails.

Security Gaps¶

• [x] ~~Kill switch is placeholder~~ (CC7.4): Resolved. KillSwitch now registers agents and substitutes, creates StepHandoff records, and increments handoff_success_count during saga orchestration. See kill_switch.py:69-178.
• [ ] Detection modules not wired to enforcement (CC6.8): PromptInjectionDetector, RateLimiter, BoundedSemaphore, ScopeGuard, SupplyChainGuard, and MCPSecurityScanner exist as standalone utilities but are not auto-wired into the BaseIntegration enforcement lifecycle. 6 of 10 OWASP risks share this structural gap.
• [ ] MCP scanner acknowledges incompleteness (CC7.3): Line 287 of mcp_security.py warns it "uses built-in sample rules that may not cover all MCP tool poisoning techniques."
• [ ] Regex-only prompt injection detection (CC6.8): No semantic or multilingual detection. English-only regex patterns can be bypassed via paraphrasing.
• [ ] No network-level security enforcement (CC6.6): TLS enforcement and certificate pinning are deferred to deployment configuration.
• [ ] Organizational controls not addressed (CC1–CC4): Board oversight, personnel policies, risk assessment governance, and monitoring activities are organizational obligations outside the toolkit's scope.

Recommended Controls¶

1. Wire detection modules into BaseIntegration.pre_execute() via GovernancePolicy flags (closes CC6.8 gaps across multiple OWASP risks).
2. Implement actual process termination in KillSwitch (CC7.4).
3. Deploy each agent in a separate container with governance middleware inside for defense-in-depth (CC6.7).
4. Add network policies for cross-agent communication control (CC6.6).
5. Integrate LlamaFirewall for semantic prompt injection detection (CC6.8).

Availability (A1)¶

The system is available for operation and use as committed or agreed.

What Exists¶

Performance Benchmarks¶

These numbers are relevant to A1.1 (system capacity) and demonstrate that the governance layer does not meaningfully impact availability:

Source: BENCHMARKS.md. Measured with time.perf_counter(), 10,000 iterations, on a development workstation.

# A1.2 in action: SLO with Error Budget
from agent_sre import SLO, ErrorBudget
from agent_sre.slo.indicators import TaskSuccessRate, HallucinationRate

slo = SLO(
    name="production-agent",
    description="Production reliability targets",
    indicators=[
        TaskSuccessRate(target=0.95, window="24h"),
        HallucinationRate(target=0.05, window="24h"),
    ],
    error_budget=ErrorBudget(total=0.05),
)

Availability Gaps¶

• [ ] Chaos engine is framework-only (A1.2): ChaosExperiment.inject_fault() records that a fault was injected but does not modify system behavior. Callers must implement actual fault injection externally.
• [ ] RateLimiter not wired (A1.2): RateLimiter at rate_limiter.py:93-101 has a correct token-bucket algorithm but is not imported by any adapter or interceptor. BoundedSemaphore for concurrency limiting is similarly unwired.
• [ ] No health check endpoints (A1.1): No liveness or readiness probes exposed for container orchestration.
• [ ] No disaster recovery automation (A1.3): Replay engine is designed for debugging and failure reproduction, not automated recovery. Saga compensation in the kill switch is placeholder-only.
• [ ] No backup/restore for audit data (A1.3): Audit backends write to JSONL files or in-memory stores with no backup, replication, or archival mechanism.

Recommended Controls¶

1. Wire RateLimiter and BoundedSemaphore into BaseIntegration with blocking behavior controlled by policy flags.
2. Implement health check endpoints for Kubernetes liveness/readiness probes.
3. Add pluggable fault injection hooks in the chaos engine for real resilience testing.
4. Deploy audit logs to an external append-only sink (Azure Monitor, write-once storage) for durability.
5. Implement automated backup and retention for audit data stores.

Processing Integrity (PI1)¶

System processing is complete, valid, accurate, timely, and authorized.

What Exists¶

# PI1.5 in action: Merkle Audit Chain
from agentmesh.governance.audit import AuditEntry, AuditLog

log = AuditLog()
entry = AuditEntry(
    event_type="governance_decision",
    agent_did="did:agentmesh:finance-bot:abc123",
    action="transfer",
    outcome="denied",
    policy_decision="DENY",
    matched_rule="max_transfer_limit",
)
log.add_entry(entry)

# Entry gets automatic SHA-256 hash chaining
assert entry.entry_hash != ""
assert entry.previous_hash != "" or log.entries.index(entry) == 0

Processing Integrity Gaps¶

• [x] ~~DeltaEngine chain verification is a stub~~ (PI1.5): Resolved. verify_chain() now computes SHA-256 hashes and verifies parent linkage across entries. See delta.py:67-127.
• [ ] FlightRecorder hash covers INSERT-time state (PI1.5): Hash is computed at insert time with policy_verdict='pending', but the verdict is later updated to 'allowed'/'blocked'. Tampering of the verdict field is undetectable by integrity verification.
• [ ] Anomaly detections outside tamper-evident chain (PI1.5): RogueAgentDetector stores assessments in an in-memory list, not in the integrity-protected audit chain.
• [ ] post_execute() never blocks (PI1.3): base.py:977-1038 computes drift scores and emits DRIFT_DETECTED events but always returns (True, None) — advisory only, no enforcement on output integrity.
• [ ] Python-only code validation (PI1.3): CodeSecurityValidator raises ValueError for any language other than Python at secure_codegen.py:193.
• [ ] No output text sanitization (PI1.4): Tool argument scanning exists; LLM response text is not scanned for dangerous content, PII, or secrets.

Recommended Controls¶

1. ~~Fix DeltaEngine verify_chain() stub~~ — Done. Now performs real SHA-256 chain verification.
2. Fix FlightRecorder hash — compute hash over final state including resolved verdict, not INSERT-time state.
3. Wire anomaly detections into the tamper-evident audit chain.
4. Add GovernancePolicy.block_on_drift flag to enable enforcement in post_execute().
5. Use only MerkleAuditChain (the sound implementation) for SOC 2 audit evidence until other implementations are fixed.

Confidentiality (C1)¶

Information designated as confidential is protected as committed or agreed.

What Exists¶

# C1.2 in action: Egress Policy with Default-Deny
from agent_os.egress_policy import EgressPolicy

policy = EgressPolicy(default_action="deny")
policy.add_rule("*.internal.corp.com", action="allow")
policy.add_rule("api.openai.com", action="allow")

# All other domains blocked — prevents data exfiltration
assert not policy.is_allowed("evil-exfil-server.com")  # Denied
assert policy.is_allowed("api.openai.com")              # Allowed

Confidentiality Gaps¶

• [ ] HMAC uses symmetric keys (C1.2): Any insider with the HMAC key can forge the entire audit chain. No external commitment (Merkle root anchoring to a timestamping service) or asymmetric signing prevents full chain rewrite.
• [ ] No at-rest encryption (C1.1): Audit logs, policy documents, and configuration files are stored in plaintext. No encryption for data at rest.
• [ ] No key rotation mechanism (C1.2): No mechanism for rotating Ed25519 keys, HMAC secrets, or SPIFFE certificates on a schedule.
• [ ] Audit redaction is partial, not comprehensive (C1.1): mcp_gateway.py now redacts credential-like secrets before persisted audit storage via CredentialRedactor, but broad PII minimization remains incomplete. Built-in PII detection still covers only SSN and credit card patterns on tool inputs, so non-credential PII can still flow into logs on some paths.
• [ ] Only 2 PII patterns (C1.1): SSN and credit card number. No email, phone, IP address, JWT token, or other sensitive data patterns.
• [ ] retention_days not enforced (C1.3): The schema field exists but no code preserves or deletes logs based on this value. A deployer can set retention_days: 1 without validation error.
• [ ] No TLS enforcement (C1.2): Network encryption deferred entirely to deployment configuration.

Recommended Controls¶

1. Expand audit redaction beyond credentials so persisted audit payloads also minimize non-credential PII before storage.
2. Expand PII patterns to cover the OWASP-recommended set (email, phone, IP address, JWT tokens).
3. Implement asymmetric signing for audit entries to prevent insider forgery.
4. Add key rotation tooling for Ed25519 and HMAC credentials.
5. Enforce retention_days at runtime with actual log deletion and archival.
6. Deploy audit logs to encrypted storage (e.g., Azure Blob with SSE, S3 with KMS).

Privacy (P1–P8)¶

Personal information is collected, used, retained, disclosed, and disposed to meet the entity's objectives.

⚠️ Privacy is the largest gap area. The toolkit is a runtime governance framework for AI agent actions. It was not designed as a privacy management platform. Organizations in SOC 2 scope with Privacy criteria must supplement with dedicated privacy tooling.

What Exists¶

Privacy Gaps¶

• [ ] No consent management (P2): No opt-in/opt-out mechanism, consent tracking, purpose limitation, or consent withdrawal support. This is a fundamental Privacy criteria requirement.
• [ ] No data subject access requests (P5): No DSAR workflow, data export mechanism, or right-to-erasure support.
• [ ] No data minimization (P3): No mechanism to limit data collection to what is necessary for a specific purpose. blocked_patterns is a negative control (block known-bad) rather than a positive control (allow only known-good).
• [ ] No retention enforcement (P4): retention_days field exists in the policy schema but no code preserves or deletes data based on this value. Default is 90 days with minimum 1 — there is no floor enforcement.
• [ ] Only 2 PII patterns (P6): SSN (\b\d{3}-\d{2}-\d{4}\b) and credit card number regex in mcp_gateway.py:34-42. No detection for email addresses, phone numbers, IP addresses, physical addresses, dates of birth, or other PII categories.
• [ ] No output PII scanning (P6): PII patterns check tool input arguments only. LLM response text is not scanned — an agent can freely output personal data in its responses.
• [ ] Audit-log PII minimization is incomplete (P6): The MCP gateway redacts credential-like secrets before persisted audit storage, but broader PII classes are not comprehensively minimized across audit paths. PII in tool arguments can still propagate into logs when it does not match the built-in credential patterns.
• [ ] No privacy notice mechanism (P1): No feature generates or delivers privacy notices to end users interacting with governed agents.
• [ ] No privacy impact assessment tooling (P8): No DPIA/PIA workflow or template generation.

Recommended Controls¶

1. Broaden audit parameter minimization — extend current credential redaction to cover a wider PII set before persistence. This remains the highest-leverage single fix.
2. Expand PII detection from 2 patterns to the OWASP-recommended set (email, phone, IP, JWT, passport, driver's license numbers).
3. Apply PII scanning to LLM outputs via post_execute() or a dedicated output interceptor.
4. Deploy dedicated privacy management tooling (e.g., OneTrust, BigID, Transcend) for consent, DSAR, and data mapping.
5. Enforce retention_days at runtime with automated log deletion.
6. Add GovernancePolicy.data_classification metadata to categorize agents by data sensitivity.
7. Document the scope boundary: the toolkit governs agent actions, not personal data lifecycle management.

Evidence Sources¶

All file paths referenced in this document, organized by package:

Agent OS (agent-governance-python/agent-os/)¶

AgentMesh (agent-governance-python/agent-mesh/)¶

Agent Hypervisor (agent-governance-python/agent-hypervisor/)¶

Agent SRE (agent-governance-python/agent-sre/)¶

Other¶

Gaps Summary¶

All gaps consolidated and rated by severity for remediation prioritization.

Critical¶

Resolved (formerly Critical/High)¶

High¶

Medium¶

Alignment with Other Compliance Mappings¶