Legal AI Vendor Security Checklist

Definitions

Field definitions

Controlled vocabulary guidance

data_class

Examples: Public, Internal, Confidential, Client-confidential, Privileged or restricted, Personal data, Sensitive personal data, Security-sensitive

Governance: Apply the organization and client-approved classification before enabling a feature. Do not infer a lower classification because a vendor calls a field metadata or telemetry.

flow_status

Examples: Proposed, Verified, Approved, Exception, Blocked, Retired

Governance: A flow is production-approved only when its recipient, purpose, region, retention, access, security controls, and contract basis are verified.

training_use

Examples: Prohibited, Contractually prohibited, Opt-out available, Opt-in only, Allowed for defined purpose, Unknown

Governance: Use Unknown until the vendor answers data types, feature exceptions, human review, derived data, retention, and controlling terms.

tenant_test_result

Examples: Pass, Pass with finding, Fail, Not tested, Not applicable

Governance: Record the test population, configuration, feature, model route, date, evidence, and remediation. A Pass is not a guarantee outside the tested boundary.

evidence_status

Examples: Current and configuration-specific, Current but scope-limited, Expired, Pending validation, Remediation open, Unavailable

Governance: Record scope, period, exclusions, control mapping, reviewer, and open findings for every attestation, certification, test, or vendor answer.

incident_severity

Examples: Critical, High, Medium, Low, Unknown

Governance: Use organization-designed severity anchors based on data exposure, tenant scope, legal or operational impact, ongoing risk, evidence integrity, and urgency. Severity is not a legal conclusion.

lifecycle_state

Examples: Active, Archived, Deletion requested, Deletion verified, Held, Restored, Terminated

Governance: Do not mark deletion verified without evidence from the relevant live, backup, index, log, support, and subprocessor paths or a documented exception.

Practical workflow

1. Define the use case and risk boundary

   Name the legal workflow, users, entities, clients, matters, jurisdictions, data classes, output decisions, human reviewers, prohibited uses, and business consequence of failure. Separate assistive drafting or retrieval from actions that can disclose information, alter records, communicate externally, set deadlines, approve transactions, or influence a legal determination. Record the intended model, features, integrations, environment, and accountable owner.

2. Draw the complete data-flow map

   Trace prompts, files, metadata, embeddings, retrieved passages, outputs, feedback, telemetry, support tickets, abuse signals, audit logs, backups, exports, caches, and deletion requests from the customer boundary through the vendor, model providers, cloud regions, subprocessors, administrators, and return paths. Record data type, purpose, direction, protocol, storage location, retention, access role, encryption state, and whether the flow is optional or required. Ask the vendor to validate the map for the exact plan and configuration.

3. Identify every model and provider dependency

   List each foundation model, hosted model, fine-tuned model, embedding model, reranker, classifier, OCR engine, speech or translation service, vector store, cloud platform, observability service, support tool, and subprocessors used by each feature. Capture provider legal name, service role, version or model family, region, change notice, fallback route, data-use terms, access boundary, and failure behavior. Require a process for notifying customers before material provider or model changes.

4. Set the training and improvement rule

   Obtain an explicit answer for whether customer prompts, files, outputs, feedback, usage metadata, identifiers, support content, or derived representations are used for training, fine-tuning, evaluation, abuse detection, product improvement, or human review. Confirm default and opt-in settings, account-level controls, feature exceptions, de-identification claims, retention of opt-out data, and the contract language that prevails. Do not accept a marketing statement such as "not used for training" without defining every data type and processing purpose.

5. Verify tenant and matter isolation

   Ask how the service separates tenants, workspaces, matters, clients, users, administrator roles, retrieval indexes, caches, embeddings, logs, backups, exports, and support tooling. Request architecture diagrams, authorization rules, negative test results, cross-tenant test evidence, incident history, and controls for shared infrastructure. Test that search, previews, citations, APIs, notifications, bulk exports, integrations, and support access cannot cross an approved matter or tenant boundary.

6. Review regions, residency, and transfer paths

   Record processing and storage regions for live data, temporary files, logs, backups, disaster-recovery copies, model services, support access, telemetry, and subprocessors. Ask whether routing can change by feature, incident, load, or provider fallback. Confirm the vendor can identify where a specific customer record was processed and can support the organization's privacy, client, contractual, records, and cross-border review. Residency is a deployment and contract question, not an inference from the vendor headquarters.

7. Inspect encryption and key management

   Document encryption in transit, at rest, backups, queues, temporary storage, exports, and administrative channels; protocol and algorithm choices; key ownership; key generation, rotation, access, separation, recovery, and revocation; and the treatment of model inputs, outputs, embeddings, and logs. Ask whether customer-managed keys, tenant-specific keys, bring-your-own-key, or regional key controls are supported and what data or features are excluded. Require evidence for the purchased service, not only a corporate policy.

8. Check identity, authorization, and privileged access

   Verify SSO, federation, MFA, SCIM or lifecycle automation, role-based and attribute-based authorization, matter or workspace restrictions, service accounts, API keys, session controls, support impersonation, break-glass access, administrator separation, least privilege, and joiner-mover-leaver handling. Ask how the system evaluates authorization for retrieval, prompts, outputs, exports, citations, integrations, and background jobs. Require attributable access and review evidence for vendor personnel and privileged operations.

9. Specify logging and audit evidence

   Define which events are logged for sign-in, authorization, prompt receipt, source retrieval, model route, output, citation, export, integration, configuration, administrator action, support access, deletion, policy change, and security event. Ask for timestamps, actor and tenant IDs, correlation IDs, model or feature version, region, outcome, tamper protection, customer access, export format, retention, and clock handling. Confirm that logging does not silently create a new sensitive-data copy outside the approved boundary.

10. Test retention, deletion, and legal holds

    Map default and configurable retention for prompts, files, outputs, embeddings, indexes, feedback, logs, backups, snapshots, support records, abuse-monitoring data, and derived data. Define deletion triggers, user and administrator controls, API behavior, backup expiration, subprocessor propagation, verification evidence, exceptions for security or legal obligations, and handling of legal holds. Test a representative record through active, archived, deleted, restored, and backup-expiry states and retain the result.

11. Evaluate incident and vulnerability response

    Obtain the vendor's incident definitions, detection and escalation process, customer notification commitments, contact channels, evidence preservation, containment, root-cause reporting, remediation tracking, vulnerability disclosure process, patch timelines, dependency monitoring, and cooperation with customer investigation. Ask how the vendor handles prompt injection, data exfiltration, model abuse, credential compromise, cross-tenant exposure, poisoned retrieval content, unsafe integrations, and provider incidents. Contractual notice periods should be confirmed against the organization's obligations and response needs.

12. Review testing and model assurance

    Request security testing scope and dates, penetration tests, vulnerability scans, code review, dependency and supply-chain controls, red-team or adversarial testing, privacy testing, authorization tests, data-leakage tests, prompt-injection tests, retrieval poisoning tests, abuse controls, availability tests, and regression evidence. Require the test population, version, configuration, exclusions, severity method, unresolved findings, retest results, and independent reviewer. For legal workflows, also test citation support, source boundaries, hallucination handling, human review, and output labeling.

13. Assess integrations and downstream exposure

    Inventory browsers, email, document stores, case and contract systems, identity providers, APIs, webhooks, plugins, connectors, exports, search indexes, storage targets, and automation actions. For each integration, document permissions, scopes, token storage, data minimization, synchronization direction, cache and failure behavior, retry and replay risk, approval gates, tenant mapping, logging, revocation, and downstream retention. Disable actions that can send, change, delete, or share records unless the authorization, preview, confirmation, and audit path is explicit.

14. Check business continuity and exit

    Review availability commitments, regions and redundancy, recovery time and recovery point objectives, backup protection, dependency failure modes, model or provider fallback, degraded mode, support coverage, disaster exercises, customer communications, portability, export format, deletion on termination, transition assistance, and orderly service shutdown. Test that the organization can retrieve source records, prompts, outputs, citations, audit evidence, configurations, permissions, and deletion attestations without relying on a vendor dashboard that may no longer be available.

15. Convert findings into contract controls

    Attach the approved data-flow map, service and model scope, training-use rule, region list, subprocessor process, security controls, access model, logging, retention and deletion schedule, incident and vulnerability duties, testing evidence, continuity terms, audit rights, cooperation obligations, confidentiality, intellectual-property treatment, data return, deletion certification, change notice, suspension rights, and termination assistance to the procurement record. Assign each requirement an owner, evidence source, review date, exception, and acceptance decision.

Comparison

Limitations and exceptions

• This is an organization-designed procurement and third-party-risk checklist, not a universal security standard, legal opinion, privilege determination, privacy impact assessment, or certification of any vendor.
• A certification, attestation, audit report, penetration test, encryption statement, or security questionnaire describes a defined scope, period, method, and evidence set. It does not guarantee the security, confidentiality, availability, accuracy, legal suitability, privilege, or compliance of a particular deployment.
• Vendor answers can be incomplete, stale, feature-specific, plan-specific, or inconsistent with the actual tenant configuration. Validate material claims with architecture, testing, contract language, operational evidence, and customer-controlled settings.
• No checklist proves that a model will avoid hallucinations, prompt injection, data leakage, unsafe outputs, biased behavior, unauthorized retrieval, or every future vulnerability. Use human review, restricted actions, monitoring, testing, incident response, and change control.
• Residency, deletion, retention, confidentiality, privilege, records, and notification requirements depend on the data, entity, client, contract, jurisdiction, system, and facts. Qualified legal, privacy, security, records, and business owners must make the applicable decisions.
• Provider, model, subprocessor, feature, region, and integration changes can alter risk after approval. Reassess on material change, incident, vulnerability, contract change, new data class, new use case, or failed control and preserve the prior decision.

Primary sources

Methodology

Use this checklist as a versioned third-party-risk record. Start with the approved use case, data classes, matters, entities, jurisdictions, users, prohibited actions, human-review points, and consequence of failure. Build a complete data-flow register for prompts, files, metadata, retrieval, embeddings, outputs, feedback, telemetry, logs, support, backups, exports, deletion, models, cloud regions, and subprocessors. Record each model and provider dependency, training-use rule, tenant boundary, region, encryption and key control, identity path, privileged access, logging and audit events, retention and deletion rule, incident and vulnerability duty, testing result, integration scope, continuity plan, exit evidence, and contract term. Treat organization-designed thresholds as internal controls, not universal NIST or CISA requirements. For each requirement, record owner, evidence, scope, period, configuration, model or feature version, exception, remediation, approval, review date, and change trigger. Validate negative cases for cross-tenant retrieval, unauthorized exports, prompt injection, poisoned sources, integration overreach, support access, deletion, restore, and provider failure. Certifications and attestations can support assurance, but they are not guarantees; compare their scope and exclusions to the actual service and supplement them with configuration-specific evidence, testing, contract rights, monitoring, and human review.

FAQs

Related CaseDocker capabilities

Related guides and resources

• Legal AI data residency and privacy requirements
• Legal AI prompt and output audit logs
• Legal RAG knowledge base governance
• Legal AI pilot success metrics