Inside the Method: From Intelligence Tradecraft to the Courtroom

How the Structured Evidence Analysis Matrix works — Part 2 of 3

 

If you haven’t read the previous article, please read this one first. It introduces the case study this series is built around: a contested court matter with cross-allegations, a documentary record running to thousands of pages, and a client who needed the court to see the wood for the trees.

In Part 1, I described the outcome: seventeen evidence items, three competing accounts, and a one-page result showing that one account was contradicted nine times by the contemporaneous record, the neutral account three times, and one account not at all.

This article explains the machinery that produced that page — the Analysis of Competing Hypotheses (ACH) at its core, and what I changed to make it fit for legal work. The result of those changes is what I call the Structured Evidence Analysis Matrix, or SEAM.

The problem ACH was built to solve

 

ACH was developed by Richards J. Heuer Jr, a veteran of the CIA, and published in his Psychology of Intelligence Analysis (1999). Heuer’s central observation was uncomfortable: intelligence failures were rarely failures of information. Analysts had the data. They failed because of how minds handle data — settling early on a favoured explanation and then unconsciously weighing everything that arrived against it. Supporting facts registered; contradicting facts were explained away.

Lawyers will recognise the pattern instantly, because litigation manufactures it. Each party assembles the evidence that supports its case — that is, quite literally, the job — and by the time a matter reaches hearing, two teams have spent months curating confirmation. The decision-maker is then asked to choose between two professionally constructed exercises in confirmation bias.

ACH inverts the process. In summary, the analyst must:

1. Identify a set of competing hypotheses that is as complete as reasonably practicable, and mutually exclusive.  

2. Assemble the relevant evidence — all of it, not just the convenient parts.

3. Test each item of evidence against each hypothesis, item by item, asking whether it is consistent or inconsistent with that hypothesis.

4. Focus on diagnosticity — the capacity of an item to discriminate between hypotheses — and on disconfirmation rather than confirmation.

5. Provisionally prefer the hypothesis with the least inconsistency — not the one with the most consistent items.

Step 5 is the one that startles people. We are wired to ask “what supports my theory?” ACH insists that the question is analytically worthless.

Diagnosticity: the concept that changes everything

 

Recall the accused car thief from Part 1. Licence, knowledge of the car’s location, access to a key: all consistent with guilt, all consistent with innocence. Because those facts fit every hypothesis, they discriminate between none. They have zero diagnostic value, no matter how many of them you pile up.

Being verifiably in another country on the day has enormous diagnostic value — not because it supports the innocence hypothesis, but because it is nearly impossible to reconcile with the guilt hypothesis. One well-established inconsistency can be decisive where a hundred consistencies prove nothing.

This is why a SEAM is read down the columns rather than across the rows, and why the matrix expressly labels evidence that fits every account as “equivocal — fits, but does not discriminate.” In the case study, the very first item in the matrix was exactly that kind of fact — included deliberately, scored equivocal against all three accounts, to show the court that the method gives no credit for mere consistency. An analysis that shows its own discipline earns trust.

From ACH to SEAM: what a legal setting demands

 

Classic ACH was designed for intelligence assessments read by other analysts. Courts are a different audience with different rules, and this is where SEAM departs from its parent. Five adaptations matter most.

1. Accounts, not abstract hypotheses. In litigation the hypotheses are not invented by the analyst; they already exist as the parties’ formal positions. A SEAM frames them that way — Account A, the applicant’s account; Account C, the respondent’s account — and adds something neither party pleads: Account B, the genuine null, in which neither side’s allegations are made out and events reflect ordinary conflict, mistake, or poor coordination. The set must include an account adverse to whoever prepares the matrix. That is the structural safeguard against the straw-man objection: you cannot be accused of building a set designed to produce a predetermined result when one of the accounts is “I am wrong.”

2. A four-level scale with a materiality threshold. Where classic ACH typically scores consistent/inconsistent, SEAM uses four levels: Consistent, Equivocal, Inconsistent, and Materially inconsistent — difficult to reconcile. The distinction between an inconsistency and a material one mirrors how courts already think: a contradiction on a peripheral detail is noise; a contradiction on a material point — a sworn claim refuted by the parties’ own contemporaneous correspondence, say — is not cured by any volume of other facts that happen to fit.

3. Built for a judicial reader. Every SEAM opens with a plain-language “How to read this matrix” preamble: the principle, a worked illustration, and the instruction to read down columns rather than across rows. No assumed background in analytic tradecraft. If the reader needs training to use your analysis, your analysis has failed.

4. Total traceability. Every evidence item in the matrix is tied to a dated primary document in the material actually filed with the court — not to assertions, summaries, or recollections. In the case study, a dedicated sources annexure mapped each of the seventeen items to the specific exhibits supporting it. The analysis “shows its working,” which means it can be examined, challenged, or re-scored by the other side, or by the court itself. That transparency is not a vulnerability. It is the entire point.

5. Express limitations. A SEAM states on its face what it is not. It does not determine credibility, motive, or the ultimate facts; those are matters for the court. The scoring reflects one party’s analysis; inclusion of an item is not an admission. Overclaiming is the fastest way to have a good analysis discounted — and the discipline of stating limits is itself evidence that the method has been used honestly.

What this looked like in practice

 

In the engagement from Part 1, the seventeen items were not seventeen accusations. They were categories of observable fact: what the contemporaneous messages showed around the dates key allegations were later said to have arisen; whether conduct matched the asserted state of affairs; what the financial records showed against sworn claims about them; the timing and outcomes of earlier proceedings between the parties; and the internal features — corroborated or not, dated or not, probable or not — of the allegations themselves.

Scored honestly, cell by cell, the matrix produced the tally I described in Part 1: nine inconsistencies against one account, including one material; three against the neutral account; none against the third.

One further check matters before anyone relies on such a result: robustness. Would the ranking change if the most contested items were discounted or re-scored? In this case it would not — the conclusion did not depend on any single item. An analysis that survives its own stress test is one you can put your name to.

What the matrix cannot do — and who says so

 

Everything above describes a rigorous analysis. But rigour claimed by the person who prepared the analysis is just assertion. The question a court will ask — and should ask — is: who says this method was applied correctly?

That question is answered by the most distinctive element of the SEAM framework: independent expert validation of the methodology, delivered under the expert witness rules of the court, on a deliberately narrow question. That is Part 3.

If your matter involves a contested record that has outgrown narrative argument, you can read about my independent analysis services here.

Julian Talbot is the author of the Security Risk Management Body of Knowledge (SRMBOK) and has more than 35 years’ experience in risk management and analysis across government and regulated industries. He provides independent analytical reports for law firms, advocacy organisations, and parties to complex disputes. This series describes analytical methodology; it is not legal advice.

Subscribe to see the next in the series.