In June 2023, attorneys Steven Schwartz and Peter LoDuca made international headlines for the wrong reason. They had submitted a legal brief to the Southern District of New York that cited six cases -- none of which existed. The cases had been generated by ChatGPT, and neither attorney had verified them before filing. Judge P. Kevin Castel sanctioned both attorneys and described the filing as an "unprecedented circumstance" of submitting "bogus judicial decisions with bogus quotes and bogus internal citations."

The Mata v. Avianca debacle was not an isolated incident. It was the most visible symptom of a structural problem: general-purpose language models generate text that looks authoritative but is not grounded in verifiable sources. In most applications, this is an inconvenience. In legal practice, it is a professional and ethical catastrophe.

Provenance tracking is Themis's answer to this problem, and we believe it is the single most important feature that separates responsible legal AI from dangerous legal AI.

What Provenance Means in Legal Work

Provenance, in the context of legal analysis, means that every assertion in a work product can be traced back to a specific, verifiable source. This includes:

  • Factual assertions traced to specific documents in the case file -- an incident report, a medical record, a deposition transcript, a contract provision.
  • Legal rules traced to specific authorities -- statutes, regulations, case holdings -- with proper Bluebook citations that a reader can independently verify.
  • Analytical conclusions traced to both the factual predicates and the legal rules that support them, forming a complete chain of reasoning.
  • Strategic recommendations traced to the risk factors and legal landscape from which they derive.

Without provenance, a legal document is an assertion without support. With provenance, it is an argument that can be evaluated, challenged, and relied upon.

How Themis Traces Every Assertion

In the Themis architecture, provenance is not an afterthought bolted onto the output. It is a structural requirement enforced at every layer of the system.

Every agent in the Themis system -- the Legal Data Analyst, the Doctrinal Expert, the Legal Strategy Agent, and the Document Drafting Agent -- is required to include provenance metadata in its output. When the LDA extracts a fact from a case document, the extraction record includes the source document identifier, the relevant text span, and the extraction confidence. When the DEA identifies a legal authority, the citation record includes the full Bluebook citation, the relevant holding, and the relationship of the authority to the client's position (supporting, distinguishable, or adverse).

The orchestrator validates provenance at phase boundaries. When the DEA produces legal analysis that references a factual assertion, the orchestrator verifies that the assertion appears in the LDA's fact extraction output. When the DDA produces a draft document that cites an authority, the orchestrator verifies that the authority appears in the DEA's research output. This cross-validation prevents the most dangerous failure mode of language models in legal work: the generation of plausible-sounding but fictitious citations during the drafting phase.

Bluebook Citation Verification

The Bluebook: A Uniform System of Citation is the standard reference for legal citation format in the United States. Proper citation is not merely a formatting preference -- it is a professional obligation. An improperly cited case is, at best, difficult for the reader to locate and verify. At worst, it suggests carelessness that may undermine the court's confidence in the substance of the argument.

Themis enforces Bluebook citation format throughout the system. The DEA produces citations in proper Bluebook format during the research phase. The DDA validates citation format when incorporating authorities into draft documents. The orchestrator's reflection pass checks citation consistency across the final work product.

This is not a spell-checker for legal citations. It is a structural guarantee that every authority referenced in a Themis-generated document has been identified during the research phase, cited in proper format, and cross-referenced against the factual and legal assertions it supports.

Jurisdiction-Aware Outputs

Legal authority is not universal. A California Supreme Court opinion is binding in California state courts but merely persuasive in New York. Federal circuit court decisions bind only the districts within that circuit. State statutes vary dramatically in their treatment of identical legal issues -- comparative negligence rules, statutes of limitations, discovery obligations, and filing requirements all differ by jurisdiction.

Themis is jurisdiction-aware at every level. The DEA distinguishes between controlling authority (binding precedent from the relevant jurisdiction) and persuasive authority (decisions from other jurisdictions or lower courts). The DDA formats documents according to jurisdiction-specific filing requirements. The LDA applies jurisdiction-specific rules when computing damages and tracking statutes of limitations.

This jurisdictional awareness is baked into the system architecture, not layered on as a post-processing step. When the orchestrator builds a task graph for a matter filed in the Southern District of New York, every agent in the pipeline receives the jurisdictional context and constrains its output accordingly.

The Trust Gap

There is a fundamental trust gap between AI-generated output and documents that lawyers are willing to file with a court, send to opposing counsel, or present to a client. This gap exists because most AI systems provide no mechanism for a lawyer to verify how a conclusion was reached, what sources support it, or whether the system considered adverse authority.

Provenance tracking closes this gap. When a Themis-generated document states that "the defendant owed a duty of care to the plaintiff under Palsgraf v. Long Island Railroad Co., 248 N.Y. 339 (1928)," the attorney reviewing the document can verify:

  1. The citation is a real case with the correct reporter and year.
  2. The case was identified during the research phase, not generated during drafting.
  3. The case is controlling authority in the relevant jurisdiction.
  4. The holding attributed to the case is accurate.
  5. The system also identified and disclosed any adverse authority on the same point.

This is the difference between a document that a lawyer hopes is correct and a document that a lawyer can verify is correct. The former is a liability. The latter is a tool.

From Output to Work Product

The legal profession has a standard for work product that predates AI by centuries. A brief filed with a court represents a lawyer's professional judgment that the facts are accurate, the law is correctly stated, and the arguments are made in good faith. Rule 11 of the Federal Rules of Civil Procedure requires that every filing be supported by reasonable inquiry -- that the factual contentions have evidentiary support and that the legal contentions are warranted by existing law or a nonfrivolous argument for its modification.

AI-generated text, by itself, does not meet this standard. There is no "reasonable inquiry" when the system that generated the text cannot identify what sources it consulted or whether those sources exist. Provenance tracking is what transforms AI output from generated text into a foundation for professional work product. It provides the audit trail that makes reasonable inquiry possible.

Every factual assertion must include provenance metadata. When uncertain about a legal principle, flag it as an unresolved issue rather than guessing.

The core principle is simple: a system that knows what it does not know is far more valuable than a system that confidently fabricates answers. Themis tracks provenance not because it makes the output look more credible, but because it makes the output actually verifiable. For lawyers, that distinction is everything.