Legal RAG Knowledge Base Governance Guide

Definitions

Field definitions

Controlled vocabulary guidance

Authority class

Examples: Official primary source; executed agreement; approved internal policy; verified matter record; expert or research reference; user-provided context; unverified content.

Governance: Treat values as organization-designed controls. Record permitted use, jurisdiction, reviewer, source owner, and whether the class may support a legal conclusion or only contextual research.

Source lifecycle

Examples: Proposed; under review; approved; restricted; stale; conflicted; superseded; rejected; retired; held.

Governance: Keep lifecycle state separate from source authority, access permission, and legal applicability. A source can be authoritative but superseded for a historical question.

Chunk context state

Examples: Complete context; heading retained; table-linked; footnote-linked; cross-reference-linked; OCR-warning; context incomplete; not applicable.

Governance: Block or label retrieval when a chunk loses a condition, exception, definition, actor, date, or table relationship that changes meaning.

Retrieval review result

Examples: Expected source found; acceptable alternative; incomplete; wrong scope; forbidden source; stale source; permission failure; no result; under review.

Governance: Record question, expected result, index and policy versions, reviewer, failure reason, disposition, and regression status. Do not turn a retrieval result into a legal conclusion.

Human reliance decision

Examples: Research aid only; edit required; approved for internal workflow; approved for supervised draft; not approved; escalated; withdrawn.

Governance: State the use, reviewer authority, source and output scope, limitations, edits, and expiration or review trigger. Never imply that model output is self-authenticating.

Retention and hold state

Examples: Routine retention; deletion eligible; deletion approved; legal hold; regulatory hold; hold released; deletion completed; deletion blocked.

Governance: Map each value to an approved schedule or hold authority and include derived chunks, embeddings, caches, exports, backups, and audit evidence in scope.

Practical workflow

1. Define the use case and decision boundary

   State whether the knowledge base supports search, issue spotting, matter preparation, contract review, compliance research, drafting assistance, or another bounded use. Identify the audience, jurisdictions, entities, source classes, prohibited uses, output types, and review authority. Do not present a retrieval answer as a legal opinion, filing, notice, client advice, or approved business decision without the required human process.

2. Create the source authority policy

   Classify sources by authority and permitted use: official law or regulator material, executed agreement, approved internal policy, matter record, verified expert material, research reference, user-provided context, or unverified content. Define acceptance criteria, evidence of provenance, reviewer role, jurisdiction, language, scope, and source owner. A source may be useful context without being authoritative for a legal conclusion.

3. Assign ownership and accountability

   Name the knowledge-base owner, source owner, legal or subject-matter approver, information owner, privacy or security reviewer, platform owner, and operational support contact. Assign responsibility for approval, ingestion, metadata quality, permissions, version changes, conflict disposition, test-set maintenance, monitoring, deletion, holds, incident response, and human review. Record decision authority rather than inferring it from system access.

4. Register and qualify each source

   Create a source record with stable ID, title, authority, issuer, jurisdiction, entity or matter scope, source type, official location, custodian, provenance, language, publication date, effective date, expiry or supersession state, retrieval date, access class, and review status. Preserve the original source or an approved reference where policy permits. Do not ingest a file merely because it has a plausible name or matches a keyword.

5. Separate source text from interpretation

   Preserve source wording, page or section location, definitions, tables, footnotes, exceptions, and cross-references separately from summaries, translations, annotations, prompts, extracted facts, and organization-designed interpretations. Mark generated or analyst-created text as derived. Require qualified review when a transformation could change scope, condition, actor, timing, exception, or legal effect.

6. Design chunking and metadata together

   Choose boundaries that preserve headings, definitions, clauses, table context, footnotes, page references, paragraph relationships, and document order. Store source ID, chunk ID, parent record, location, page or section, version, effective interval, jurisdiction, matter or entity scope, access class, confidentiality, language, extraction method, parser version, and checksum or change marker. Avoid chunks that lose the condition or exception needed to interpret a statement.

7. Enforce permissions before retrieval

   Apply source and matter permissions at ingestion, indexing, retrieval, citation, export, and answer display. Test role, entity, matter, client, ethical-wall, privilege, personal-data, and need-to-know rules. A model prompt or post-generation filter is not a substitute for retrieval-time authorization. Log denied and allowed access decisions with actor, scope, source, time, policy version, and result.

8. Preserve versions and effective intervals

   Give each source and indexed representation a stable identity plus version, predecessor, successor, publication date, effective start, effective end, supersession reason, approval, and index status. Retrieve the version applicable to the declared question date and show when no applicable version is known. Never overwrite a prior source or silently mix current and historical text in one answer.

9. Build citation-ready retrieval

   Return source title, authority, stable identifier, precise page or section, version, effective date, and access context with each material passage. Preserve enough neighboring context for the reviewer to understand conditions and exceptions. Distinguish a citation to a source from a claim that the source proves an answer. If a response lacks adequate citations, route it to clarification or human review.

10. Detect stale, duplicate, and conflicting content

    Run source refresh and change checks using approved feeds, repositories, owner attestations, hashes, version comparisons, or scheduled review. Mark stale, withdrawn, expired, inaccessible, duplicate, translated, superseded, and conflicting records without deleting their history prematurely. Route conflicts to an accountable reviewer, record the competing passages and scopes, and block or label retrieval when the conflict could change the answer.

11. Create representative retrieval tests

    Version a test set covering direct lookup, definitions, exceptions, tables, cross-references, historical dates, permissions, multilingual content, OCR, duplicates, conflicts, no-result questions, and adversarially similar sources. Record expected source IDs or passages, acceptable alternatives, forbidden sources, query date, reviewer, test environment, model or index version, and failure disposition. Measure retrieval behavior separately from answer wording.

12. Run human review for material use

    Require an authorized reviewer to inspect the question, retrieved passages, citations, source versions, permissions, uncertainty, conflicts, and generated output before relying on it for advice, filing, negotiation, client communication, legal position, privilege-sensitive work, or consequential business action. Record the decision, edits, dissent, reviewer, time, and source or model versions. A review gate must be meaningful rather than a checkbox after automatic publication.

13. Manage deletion, retention, and legal holds

    Map sources, chunks, embeddings, caches, exports, audit records, backups, and derived summaries to approved retention and deletion rules. When a legal, regulatory, investigation, client, or records hold applies, suspend conflicting disposition and record the authority, scope, start date, custodian, release decision, and affected derived artifacts. On deletion, verify propagation, access revocation, index removal, cache handling, and evidence of completion without erasing required hold history.

14. Monitor quality, access, and drift

    Monitor source freshness, owner review age, citation coverage, retrieval success, expected-source recall, forbidden-source retrieval, permission denials, access anomalies, conflict age, no-result rate, OCR or parsing failures, test regressions, human-review completion, correction rate, and incident response. Report denominators, scope, version, and limitations. A high retrieval score does not establish legal accuracy or eliminate the need for review.

15. Control changes and incidents

    Version changes to source policy, chunking, metadata, permissions, embeddings, retrievers, prompts, reranking, models, answer templates, test sets, monitoring thresholds, and review gates. Require impact assessment, approver, rollout scope, rollback plan, regression tests, effective date, and communication. Investigate unauthorized access, stale citations, missing sources, corrupted extraction, deletion failures, and misleading outputs with preserved evidence and corrective actions.

16. Review governance and recalibrate

    Set a scheduled and event-triggered governance review for source classes, owners, access policies, test coverage, retention, conflicts, metrics, incidents, and human-review outcomes. Recalibrate thresholds and workflows using documented evidence and approved risk decisions. Preserve prior configurations and decisions so a later reviewer can reconstruct which source, index, policy, and review state produced an output.

Comparison

Limitations and exceptions

• RAG can improve source traceability and retrieval workflows, but it does not prevent hallucinations, establish legal correctness, or replace qualified legal judgment.
• There is no universal chunk size, overlap, metadata schema, authority taxonomy, freshness threshold, retrieval metric, or human-review threshold. Calibrate the framework to the use case, sources, jurisdictions, and risk.
• A citation can point to a real passage while the answer still misreads scope, exceptions, definitions, dates, or relationships. Review the cited context rather than treating citation presence as proof.
• Permission-aware retrieval depends on complete and current source, matter, client, entity, privilege, and identity data. A technically correct filter can still be unsafe when the underlying authorization record is wrong.
• Automated ingestion, OCR, parsing, chunking, embedding, ranking, and summarization can omit or transform content. Preserve warnings, failed sections, versions, and human review decisions.
• Source conflicts and legal change may require interpretation that software cannot resolve. Preserve disagreement and route material ambiguity to an authorized reviewer instead of selecting a convenient answer.
• Deletion and hold controls may be limited by caches, backups, vendor services, exports, logs, and derived artifacts. Define scope and evidence of completion with records and legal stakeholders.
• Retrieval tests are samples of behavior under declared conditions. Passing a test set does not prove production completeness, answer accuracy, or suitability for every matter or question.

Primary sources

Methodology

Use a versioned source-and-chunk register with these standard fields: source_id; source_title; authority_class; issuer; official_source_url; source_type; jurisdiction; entity_scope; matter_scope; client_scope; language; source_owner; custodian; provenance; acquisition_method; publication_date; effective_start; effective_end; expiry_or_supersession_state; retrieved_at; source_review_state; source_access_class; source_text_reference; source_checksum_or_change_marker; source_version; chunk_id; parent_source_id; parent_section; page_or_location; document_order; chunk_text; chunk_context; chunk_boundary_rule; extraction_method; parser_or_ocr_version; extraction_warnings; embedding_or_index_version; retrieval_metadata; confidentiality; privilege_state; access_policy_version; citation_trace; conflict_record; retrieval_test_case; human_review_decision; retention_class; hold_state; deletion_state; derived_artifact_scope; monitoring_record; change_record; and last_reviewed_at. Keep source authority, source scope, source text, internal interpretation, and generated output separate. Use one stable source_id across versions and a separate source_version and chunk_id for each approved representation. Effective dates must be distinct from publication, upload, retrieval, and index-build dates. Enforce access before retrieval and carry the same scope into citation, export, cache, backup, and derived-summary handling. For each material question, preserve the query, user or role scope, question date, retriever and index versions, retrieved chunk IDs, ranking or filter context where relevant, citations, output version, reviewer, and reliance decision. Organization-designed starting states are proposed, under review, approved, restricted, stale, conflicted, superseded, rejected, retired, and held for sources; complete context, context warning, permission filtered, stale, no result, and under review for chunks or retrieval; and research aid only, edit required, approved for supervised workflow, not approved, escalated, or withdrawn for human reliance. Suggested measures are source freshness coverage = sources with a current owner, review date, effective interval, and approved state / sources expected to be current x 100; citation trace coverage = material outputs with source, chunk, location, version, effective date, and retrieval event / material outputs sampled x 100; expected-source retrieval rate = test questions returning an expected source or approved alternative / eligible test questions x 100; forbidden-source retrieval rate = test questions returning a forbidden, unauthorized, stale, or out-of-scope source / eligible test questions x 100; permission enforcement rate = sampled retrieval decisions matching the approved access result / sampled authorized and denied decisions x 100; conflict disposition coverage = conflicts with owner, scope, decision, and review state / detected material conflicts x 100; human-review completion = defined material uses with a recorded authorized review decision / defined material uses sampled x 100; and deletion propagation coverage = approved deletion or hold-release actions with verified treatment across indexed and derived artifacts / approved actions sampled x 100. Publish numerator, denominator, population, time period, source and system versions, exclusions, and known failures. These measures are local governance indicators, not guarantees of legal accuracy, completeness, confidentiality, or hallucination prevention. Require qualified professionals to approve authority, applicability, access, conflicts, retention, holds, and material reliance.

FAQs

Related CaseDocker capabilities

Related guides and resources

• Legal AI citation verification workflow
• Legal AI vendor security checklist
• AI legal chronology generation controls
• Legal document OCR and search requirements