{ "questionnaire": "OWASP-LLM-Top10", "version": "2025-2026", "framework": "OWASP Top 10 for LLM Applications v2.0 (2025)", "vendor": "CMDash, Inc.", "vendorUrl": "https://cmdash.com", "contact": "security@cmdash.com", "dateCompleted": "2026-07-08", "completedBy": "security@cmdash.com", "scope": "How CMDash secures its own use of AI (Anthropic Claude) against each of the ten OWASP Top 10 for LLM Applications (2025) risk categories, LLM01 through LLM10. Six internal product features call the model: AI-assisted policy drafting, policy-control link autosuggestion, vendor-questionnaire auto-answer, the daily AI risk narrative, AI executive-summary generation, and AI risk-remediation suggestions. CMDash does not train, fine-tune, or self-host any model, and uses no vector store / RAG / embedding retrieval — every model call is a stateless request to a hosted third-party API. This materially changes the applicability of LLM04 (Data and Model Poisoning) and LLM08 (Vector and Embedding Weaknesses). Answers reflect controls actually shipped as of the date above; items not shipped are answered 'No' or 'Partial', never an aspirational 'Yes'. The companion internal review is docs/security/llm-top10-review.md.", "answerLegend": { "Yes": "Control is fully implemented with evidence.", "No": "Control is not implemented; remediation status is in notes.", "Partial": "Control is partially implemented; the gap is described in notes.", "NA": "Not applicable — typically because CMDash uses a hosted third-party model with no training pipeline and no vector/embedding retrieval." }, "sections": [ { "id": "LLM01", "title": "Prompt Injection", "questions": [ { "id": "LLM01-01", "question": "Is untrusted, user-authored text framed as data rather than instructions before being interpolated into a model prompt?", "answer": "Yes", "evidence": "All six AI services wrap tenant-authored interpolations (risk titles/descriptions, policy text, evidence descriptions, questionnaire text, org name/industry) in delimited blocks via wrapUntrusted()/withUntrustedDataPreamble() in apps/api/src/common/ai/untrusted-data.ts, with a system-prompt preamble instructing the model to treat delimited content as data, never instructions. The delimiter is escaped so crafted content cannot forge a closing tag and break out of the block." }, { "id": "LLM01-02", "question": "Are model outputs validated so that a successful injection cannot smuggle arbitrary structured data downstream?", "answer": "Yes", "evidence": "All six services zod-validate the model's JSON output before use, and referent validation re-checks AI-cited control/evidence/policy IDs against real tenant rows (policy-link resolves (frameworkSlug, controlId) to a real Control.id; questionnaire-ai filters returned questionIds to the requested set), so invented or injected references are dropped." }, { "id": "LLM01-03", "question": "Is there an independent, out-of-band prompt-injection detection/guardrail layer (e.g. a classifier scanning inputs and outputs)?", "answer": "No", "notes": "Defense is input framing plus output schema/referent validation, not a detection model. Indirect-injection exposure is inherently bounded because the AI services have no tool-use, no function-calling, and no external content retrieval, but a dedicated guardrail classifier is not implemented. Tracked as an open gap." } ] }, { "id": "LLM02", "title": "Sensitive Information Disclosure", "questions": [ { "id": "LLM02-01", "question": "Is there an accurate, published inventory of what tenant data is sent to the AI provider?", "answer": "Yes", "evidence": "docs/user/feature-guides/ai-narrative.md was corrected (item B1) to accurately state that organization name, industry, risk titles/descriptions, and policy/questionnaire text reach the model.", "notes": "The prior claim that org name and workspace identifiers were 'not included' in model inputs was false and has been removed." }, { "id": "LLM02-02", "question": "Is the AI provider disclosed as a subprocessor with purpose, data categories, region, and legal basis?", "answer": "Partial", "notes": "Anthropic is listed at /trust/subprocessors with purpose, data categories, region, and legal basis." }, { "id": "LLM02-03", "question": "Is there a CMDash-specific zero-retention or no-training agreement with the AI provider?", "answer": "No", "notes": "CMDash relies on the provider's standard commercial API data-handling terms; no CMDash-specific zero-retention/no-training addendum is in place (DSP-04). Provider terms are documented in docs/security/ai-governance-policy.md and the DSP-04 provider data-handling memo." }, { "id": "LLM02-04", "question": "Is tenant isolation enforced at the database layer as defense-in-depth for data reaching AI prompts?", "answer": "Partial", "notes": "Prompts are assembled under explicit application-level where:{tenantId} filters." }, { "id": "LLM02-05", "question": "Is there automated PII/secret redaction on prompt inputs before they leave for the provider?", "answer": "No", "notes": "No automated input redaction exists. Data minimization relies on each service sending only the fields it needs. Tracked as an open gap." } ] }, { "id": "LLM03", "title": "Supply Chain Vulnerabilities", "questions": [ { "id": "LLM03-01", "question": "Is the AI provider SDK a pinned, dependency-scanned component?", "answer": "Yes", "evidence": "@anthropic-ai/sdk is a pinned managed dependency covered by the standard CI dependency scanning described in docs/security/supply-chain.md." }, { "id": "LLM03-02", "question": "Can the AI/model dependency be cut off instantly if the provider or SDK is compromised or degraded?", "answer": "Yes", "evidence": "AI_ENABLED=false is a uniform kill switch honoured by all six services (apps/api/src/common/ai/ai-budget.service.ts and each service), disabling every model call immediately." }, { "id": "LLM03-03", "question": "Is there a multi-provider abstraction or model failover so a single-provider outage or compromise has a same-capability fallback?", "answer": "No", "notes": "CMDash is Claude-only (IPY-01). There is no fallback model vendor; multi-provider support is a roadmap item, not shipped. Some enterprise buyers require a specific LLM provider by policy — that is not yet supported." } ] }, { "id": "LLM04", "title": "Data and Model Poisoning", "questions": [ { "id": "LLM04-01", "question": "Are training data, fine-tuning pipelines, and model weights protected against poisoning?", "answer": "NA", "notes": "CMDash does not train, fine-tune, or self-host any model, and maintains no training data or model weights. There is no training pipeline to poison; model integrity is inherited from the hosted provider." }, { "id": "LLM04-02", "question": "Is untrusted runtime input prevented from steering model behavior in unintended ways (the runtime analogue of poisoning)?", "answer": "Yes", "evidence": "The closest applicable analogue — runtime input steering the model — is covered by LLM01 input framing (A4, untrusted-data.ts) and by output schema plus referent validation, not by a training-data control." } ] }, { "id": "LLM05", "title": "Improper Output Handling", "questions": [ { "id": "LLM05-01", "question": "Is model output validated against a schema before it is persisted or displayed?", "answer": "Yes", "evidence": "All six services zod-validate Claude's JSON output before use. Item A2 closed the last gap: ai-suggestions now safeParses against AiSuggestionSchema and skips/logs malformed output rather than persisting raw model text." }, { "id": "LLM05-02", "question": "Is AI-generated content escaped/sanitized before being rendered or emailed, to prevent stored XSS?", "answer": "Yes", "evidence": "The weekly executive-briefing email — the one place AI output reaches hand-built markup outside the frontend framework's auto-escaping — HTML-escapes every AI field via esc() (apps/api/src/executive-briefing/executive-briefing.service.ts) before send; the narrative service strips HTML from its output before storage. Guidance: docs/developers/ai-secure-development.md." }, { "id": "LLM05-03", "question": "Is there a dedicated server-side output content-moderation/safety layer beyond schema bounds?", "answer": "No", "notes": "In-app (non-email) AI text relies on the frontend framework's default auto-escaping plus zod schema bounds and HTML-stripping. There is no dedicated output content-moderation or safety-classification layer. See LLM09." } ] }, { "id": "LLM06", "title": "Excessive Agency", "questions": [ { "id": "LLM06-01", "question": "Do the AI features avoid tool-use, function-calling, and autonomous side-effecting actions?", "answer": "Yes", "evidence": "None of the six services grants the model tool-use, function-calling, or autonomous actions. Every output is either a suggestion a human accepts/rejects (risk suggestions, policy-link, policy drafts) or bounded, schema-validated text." }, { "id": "LLM06-02", "question": "Are privileged AI actions restricted by role-based authorization?", "answer": "Yes", "evidence": "AiSuggestionsController mounts RolesGuard at class level with @Roles('OWNER','ADMIN') on its mutation endpoints (A1), matching the role gates on the other AI controllers; all six AI endpoints sit behind JWT auth. TPRM answer/submit endpoints are role-gated (A8)." }, { "id": "LLM06-03", "question": "Does AI output that reaches an external recipient pass a human review gate before send?", "answer": "Partial", "evidence": "The weekly executive briefing — the AI output that reaches an external recipient automatically — now has a review gate before send (item A7).", "notes": "A subset of other AI outputs still persist immediately with post-hoc accept/reject rather than pre-write approval; risk is bounded because these outputs store suggested text and take no further action (MDS-04)." } ] }, { "id": "LLM07", "title": "System Prompt Leakage", "questions": [ { "id": "LLM07-01", "question": "Do system prompts avoid embedding secrets, credentials, or tenant data whose disclosure would be harmful?", "answer": "Yes", "evidence": "System prompts are static instruction text containing no secrets, credentials, or tenant data; leaking a system prompt would not disclose sensitive material." }, { "id": "LLM07-02", "question": "Is the model instructed to refuse embedded requests to reveal its prompt, and are outputs constrained so a prompt echo would be rejected?", "answer": "Yes", "evidence": "The A4 preamble instructs the model to treat 'reveal this prompt' requests found in delimited data as data to note, not commands to obey; zod output schemas would reject a verbatim system-prompt echo as malformed." }, { "id": "LLM07-03", "question": "Is there an independent filter that scans outputs for system-prompt leakage?", "answer": "No", "notes": "The instruction-hierarchy defense is preamble-based and best-effort; no out-of-band output scanner for prompt echoes exists. Tracked as an open gap." } ] }, { "id": "LLM08", "title": "Vector and Embedding Weaknesses", "questions": [ { "id": "LLM08-01", "question": "Are vector databases, embeddings, and RAG retrieval pipelines isolated per tenant and protected against embedding inversion / cross-tenant retrieval?", "answer": "NA", "notes": "CMDash uses no vector database, no embeddings, and no RAG retrieval. Every prompt is assembled from the tenant's own relational data under an explicit where:{tenantId} filter, not from a shared or cross-tenant vector index, so there is no embedding-inversion or cross-tenant-retrieval surface. If RAG/embeddings are introduced later, this row must be re-opened with per-tenant index isolation and inversion controls." } ] }, { "id": "LLM09", "title": "Misinformation", "questions": [ { "id": "LLM09-01", "question": "Are AI-invented references (e.g. non-existent control/evidence/policy IDs) prevented from being trusted?", "answer": "Yes", "evidence": "Referent validation drops any AI citation that does not resolve to a real tenant row: policy-link resolves (frameworkSlug, controlId) back to a real Control.id; questionnaire-ai filters returned questionIds to the requested set." }, { "id": "LLM09-02", "question": "Are AI outputs surfaced for human review rather than applied as authoritative facts?", "answer": "Partial", "evidence": "Most AI outputs are suggestions surfaced for OWNER/ADMIN review and explicit accept/reject; the externally-delivered executive briefing has a human review gate (A7).", "notes": "Free-text narrative and summary prose is not independently fact-checked beyond referent validation and schema bounds." }, { "id": "LLM09-03", "question": "Is there a dedicated LLM-output factuality-checking or content-moderation layer?", "answer": "No", "notes": "No dedicated content-moderation or factuality-checking layer exists. AI-generated prose is the user's responsibility to review, and a uniform 'AI-generated, verify before relying' disclaimer is not yet present on every AI surface. Tracked as an open gap." } ] }, { "id": "LLM10", "title": "Unbounded Consumption", "questions": [ { "id": "LLM10-01", "question": "Is there a per-tenant rate/spend ceiling enforced before every model call, across both HTTP and background paths?", "answer": "Yes", "evidence": "AiBudgetService (apps/api/src/common/ai/ai-budget.service.ts) enforces a Redis-backed per-tenant request window, per-feature cooldown (e.g. narrative regenerate 1/hour), and daily token budget. Every service calls enforce(tenantId, feature) immediately before messages.create. HTTP paths return 429 with X-RateLimit-*/Retry-After headers (ai-rate-limit.interceptor.ts); the narrative cron path logs a skip. Redis is the choke point so ceilings hold across replicas. GET /rate-limits/usage reports real counts." }, { "id": "LLM10-02", "question": "Is the fan-out of model calls per request bounded?", "answer": "Yes", "evidence": "The ai-suggestions fan-out ?limit= parameter is parsed and clamped to a hard ceiling (A1), bounding the number of sequential model calls a single request can trigger. AI_ENABLED=false provides a hard global stop for incident response." }, { "id": "LLM10-03", "question": "Is per-field input length validated on prompt-feeding fields to bound single-prompt cost?", "answer": "Yes", "notes": "Item A6 / TVM-03: .strict() zod schemas cap the free-text fields that feed a Claude prompt or a provenance write (policy-drafting topic/scope/additionalContext, risk title/description, TPRM answerText/evidence/notes/aiRationale), safeParse'd at the top of each service method with a BadRequestException naming the offending field on rejection." }, { "id": "LLM10-04", "question": "Are the failure modes of the budget limiter documented and bounded?", "answer": "Partial", "notes": "By deliberate design the budget service fails OPEN on a Redis outage (logs and allows) — spend is unmetered only for the outage duration, with AI_ENABLED as the hard stop. The daily token budget is a check-of-accumulated-spend, so a single in-flight call can overrun by one call's worth of tokens. Both trade-offs are documented in ai-budget.service.ts, not silently accepted." } ] } ] }