How Quorum works
Six stages, from your Slack and Gmail through to the ledger. Every number below is read live off the current ledger snapshot's trace, not hand-drawn.
Where the data comes from
Quorum reads Slack and Gmail through an MCP-style tool layer. The same four tools (search Slack, get a Slack thread, search Gmail, get a Gmail thread) are exposed both to an MCP client over stdio and to this web app in-process, from one shared adapter module. You've already used this: every “view the message” link on Signals or the Ledger opens that exact tool call. See the tool boundary below for the detail, or go look at a real conflict to watch it happen again.
Loading...
The six stages
- First, Quorum throws out anything that isn't a real person talking. A fixed 5-rule ladder (bot author, automation email address, gated channel, automation text signature, short social aside), all code, no model call.
- Then it reads every remaining message and pulls out the factual claims it makes, quoting the exact words rather than paraphrasing.
- It groups claims that are about the same underlying topic, by exact match, known aliases, then wording similarity, without ever calling a model.
- Before asking any model to judge anything, a fixed set of rules checks for the easy cases: someone updating their own earlier statement, someone correcting themselves, a senior person's final call (R5).
- Only what's left after that, genuine live disagreement between different people from the model's point of view, gets exactly one question to a model: Guardrailed scope asks yes/no (“contradiction or compatible?”), Open scope lets it choose freely from the full verdict vocabulary.
- The result is written to a persistent ledger: today's beliefs, plus a full history of who changed their mind and when.
Tool boundary
Four tools, one shared adapter, two callers: an MCP server over stdio, and this web app in-process.
Quorum treats Slack and Gmail as tool-shaped data sources, not a database it reads directly: every read goes through one of four named tools, each scoped to exactly one kind of query.
| Tool | What it does |
|---|---|
| slack.search_messages | Search the workspace's Slack messages by text, channel, author, and/or time range. |
| slack.get_thread | Fetch a full Slack thread (and its messages) by thread id. |
| gmail.search | Search the workspace's Gmail messages by text, subject, sender, and/or time range. |
| gmail.get_thread | Fetch a full Gmail thread (and its messages) by thread id. |
Both MCP tools sit on top of one adapter module, src/adapters/workspace.ts, over the same underlying corpus files (fixtures/corpus/*.json) the web app also reads.
mcp-server/exposes that adapter over stdio as the four MCP tools above (seemcp-server/src/adapter.ts, which re-exports it directly) — no network call, no HTTP round trip to the web app.- The web app itself reads the same corpus files through its own in-process
FsMessageSource(src/server/deps.ts), implementing the sameMessageSourceinterface rather than importing the adapter module directly — same data, same query shape, one interface, two call-sites. Consolidating the web app onto the adapter module too is a natural next step, not yet done.
This page shows what the MCP surface exposes without requiring you to run one — connecting a real MCP client needs a terminal, which nothing else on this site does. See README §7 to try it yourself.
How this page works (for reviewers)
This page and the SourcePanel are the two places the MCP boundary is actually exercised rather than just documented: src/adapters/workspace.ts is the single implementation; mcp-server/src/adapter.ts re-exports it for the stdio MCP server, and src/app/api/workspace/route.ts calls it in-process for the web app, never an HTTP round trip to the MCP server itself. Model selection, hooks, and agent hand-offs are documented in full in the README sections this note links to.
Full detail in the README: /architecture#architecture
The routing decision, diagrammed
The same six stages above, drawn as the actual branch points a message goes through: deterministic steps in one style, model calls in another, with the Guardrailed/Open judge-scope split shown as a real branch, not prose. Concretely: Guardrailed restricts the model's output to CONTRADICTION or COMPATIBLE only, enforced by a strict schema (BinaryVerdictSchema); Open allows all 7 verdicts (Full7VerdictSchema). See src/core/schema/verdict.ts.
| Verdict | Meaning | Decided by |
|---|---|---|
| Contradiction | Two live positions genuinely conflict. | Model only. No pre-rule ever emits this. |
| Compatible | No real disagreement: one live claim, or the live claims already agree. | Code (R6/R6b/R7) when one of those settles it; the model otherwise. |
| Update | The same person changed their own answer; no one else has a live, differing claim. | Code (deterministic same-asserter check, src/core/ledger.ts). |
| Resolved (supersession) | A senior person's later claim overrides an earlier disagreement. | Code (R5). |
| Resolved (correction) | Someone corrected themselves to match what someone else had already said. | Code (R4). |
| Ambiguous referent | Two different buckets may be the same real topic, phrased identically, disagreeing in value. | Code (ambiguity-pair detection, src/core/ledger.ts). A different mechanism from CONTESTED below. |
| Contested | Hand-labelled as genuinely arguable either way; kept out of right/wrong scoring entirely. | Code (R8). Only for a referent in the hardcoded contested set, not a general judgment call. |
For what R4-R8 (and the other code-decided checks referenced above) actually do, see Signals page, pre-rules below.
Rendered once from routing-diagram.mmd (Mermaid syntax, committed alongside the SVG) via npx @mermaid-js/mermaid-cli — a dev-time tool only, not a runtime or npm dependency of this app. Verified against src/core/pipeline.ts's runAdjudicationPipeline, not hand-drawn from memory.
Reviewer appendix
The engineering detail each page's “How this page works” note links back to. One section per page, kept here instead of a repo-root README route Vercel can't serve directly.
Overview page
“Topics being tracked” counts only catalogued referents (see the Ledger page's “other topics detected automatically” split). Internal identifiers like indep_event.launch_date and extractor-minted noise both live in the same underlying Bucket[], but only the former is a real tracked topic from a product point of view. The false positive rate and contradiction recall figures live entirely on the Evals page, computed by the same in-browser eval suite you can run yourself there. The Overview page does not duplicate that computation.
Signals page, pre-rules
Each bucket runs through this deterministic ladder before any model is called. Only a bucket with two or more live claims from different people, with no rule below able to settle it, gets a single binary model call: “do these live positions genuinely conflict?” “Rewind the ledger” re-runs the same deterministic pipeline as of an earlier point in time. It does not re-ask the model a new question, it replays the same logic against a smaller set of visible messages. These rules are hand-authored, studio-specific business logic (src/core/prerules.ts): code, not model output, and not user-configurable in this build.
| Rule | What it checks |
|---|---|
| R1 | Someone relaying what another person said doesn't count as their own claim. Excluded from the disagreement. |
| R1b | A hedge, a proposal, or a question isn't a statement of fact. Excluded. |
| R2 | The same person said something different later. Their earlier message is retired in favour of the later one. |
| R3 | Someone explicitly said what something is NOT. Checked against their own other positions rather than read as a fact itself. |
| R4 | Someone corrected themselves to match what someone else already said. Read as a correction, not an open conflict. |
| R5 | The most senior person in the thread made the final call. It overrides the earlier disagreement. |
| R6 | Only one current position remains. Nothing left to disagree about. |
| R6b | More than one current position, but they all agree. Not a conflict. |
| R7 | No current position remains at all. Nothing to disagree about. |
| R8 | Genuinely arguable both ways. Kept in its own "contested" bucket rather than scored as right or wrong. |
Two user-facing actions live on this page (src/components/BucketRow.tsx): Dismiss persists a Suppression and Mark-as-resolved persists a Resolution. Same shape, same store, same “survives a restart” guarantee (src/core/ledger.ts's dismissBucket/resolveBucket, written via /api/ledger/suppress//api/ledger/resolve). Both re-raise automatically the moment the bucket's live claim set changes (isSuppressed/isResolved). A dismissal or a resolution is never a silent, permanent hide. Neither touches projectAsOf or verdict computation: a resolution is a human annotation recorded alongside the system's own verdict, not a replacement for it. Explicitly out of scope for this pass: real notifications (Slack-reply/email-send) when a conflict is dismissed or resolved, assigning a conflict to a specific person, and a comment thread on a bucket. Real product needs, not silently missing, just a materially larger build (external-write integration, not just UI) than this pass covers.
Ledger page
Each row is a temporal projection over that referent's claims as of the ledger's frozen as-of (see src/core/ledger.ts): superseded/withdrawn claims stay visible, not hidden, so a reviewer can see what was ruled out and why. The watermark (snapshot.watermark) tracks which messages have already been processed, making re-runs idempotent. Suppression (dismiss/restore on the Signals tab) re-raises a bucket only if its live claim set actually changes. A dismissal isn't silently permanent.
Evals page
npm run eval -- --print-hash against the same committed recordings prints the same report hash offline. That is the actual reproducibility guarantee, not just the on-screen claim on that page. The regression protocol treats any single-scenario regression as a failure even if the average improves; the diff panel there implements exactly that rule, not an aggregate pass/fail. See src/core/eval/diff.ts and src/core/eval/run-eval.ts for the scoring implementation, and src/core/eval/scenarios.ts for the full scenario registry (C1-C9, N1-N18) that page's tables are driven from.
The adjudication grader scores every run against these gold claims, not against anything the model itself produced. One person (this project's author) hand-labeled them, with no measured agreement from a second annotator. That's a real limitation, stated here rather than smoothed over. See README §8 (“Known limitations”) for the full statement.
A predicted value doesn't need to match a gold value byte-for-byte to count as correct. Dates and numbers are normalized before comparing, and free text is allowed to differ if it still shares at least 60% of its words with the gold value (a token-set overlap check, not an edit distance). Referent, modality, and polarity are graded as three separate axes, not folded into one pass/fail recall number. See src/core/eval/extraction-grader.ts and src/core/eval/adjudication-grader.ts for the exact rules.
All 38 gold claims
| Claim id | Message | Referent | Value | Asserter |
|---|---|---|---|---|
| CL-001 | M-001 | indep_event.launch_date | 2026-08-12 | meera.iyer |
| CL-002 | M-002 | indep_event.launch_date | 2026-08-15 | priya.raghunathan |
| CL-010 | M-010 | indep_event.success_criteria | ARPDAU lift over 7-day window | meera.iyer |
| CL-011 | M-011 | indep_event.success_criteria | session depth and returning players | rohan.desai |
| CL-020 | M-020 | liveops_calendar.signoff_owner | priya.raghunathan | priya.raghunathan |
| CL-021 | M-021 | liveops_calendar.signoff_owner | meera.iyer | meera.iyer |
| CL-030 | M-030 | d7_retention.trend | up 1.8pp WoW | neha.bhatt |
| CL-031 | M-031 | d7_retention.trend | down 3pp since 1.9.3 | arjun.rao |
| CL-040 | M-040 | tournament.scope | leaderboard only, brackets cut | meera.iyer |
| CL-041 | M-041 | tournament.scope | includes bracket seeding | rohan.desai |
| CL-050 | M-050 | leaderboard.readiness | 2026-08-25 earliest | vikram.shetty |
| CL-051 | M-051 | leaderboard.readiness | ships with event | meera.iyer |
| CL-060 | M-060 | build_194.release_readiness | not ready, open P1 | farah.qureshi |
| CL-061 | M-061 | build_194.release_readiness | shipping Friday | priya.raghunathan |
| CL-070 | M-070 | art_capacity.allocation | 2 artists to Onam | sana.kulkarni |
| CL-071 | M-071 | art_capacity.allocation | fully on Independence assets | priya.raghunathan |
| CL-080 | M-080 | reward_config.live_state | old Holi table visible | deepak.menon |
| CL-081 | M-081 | reward_config.live_state | new config live since 18th | meera.iyer |
| CL-100 | M-100 | level40_art.eta | 2026-07-24 | sana.kulkarni |
| CL-101 | M-101 | level40_art.eta | 2026-07-29 | sana.kulkarni |
| CL-110 | M-110 | indep_event.launch_date | 2026-08-15 | karthik.nair |
| CL-120 | M-120 | soft_launch.date | 2026-08-05 | arjun.rao |
| CL-121 | M-121 | indep_event.launch_date | 2026-08-15 | priya.raghunathan |
| CL-130 | M-130 | onam_event.launch_date | 2026-08-26 | priya.raghunathan |
| CL-160 | M-160 | indep_event.launch_date | 2026-08-15 | sana.kulkarni |
| CL-170 | M-170 | indep_event.launch_date | 2026-08-15 | meera.iyer |
| CL-180a | M-180 | indep_event.duration | 7 days from go-live | priya.raghunathan |
| CL-180b | M-180 | config_freeze.owner | meera.iyer | priya.raghunathan |
| CL-190 | M-190 | reward_config.tiers | 12 | rohan.desai |
| CL-191 | M-191 | reward_config.tiers | 8 | meera.iyer |
| CL-192 | M-192 | reward_config.tiers | 8 | rohan.desai |
| CL-210 | M-210 | push_notification.upgrade_eta | 2026-07-30 | vikram.shetty |
| CL-211 | M-211 | event_dashboard.readiness | 2026-07-08 | neha.bhatt |
| CL-212 | M-212 | ua_creative_refresh.booking | booked with the agency | arjun.rao |
| CL-213 | M-213 | qa_regression_suite.coverage | covers the reconnect path | farah.qureshi |
| CL-214 | M-214 | tricolour_token_set.status | at final polish | sana.kulkarni |
| CL-215 | M-215 | event_faq.status | drafted | deepak.menon |
| CL-216 | M-216 | difficulty_curve.status | rebalanced | rohan.desai |
Same data as evals/gold-claims.json. Click any claim below to see every field plus the real source message it was labeled from.
Look up one gold claim in full
The actual prompts sent to the model
Same worked example as Evals: the literal output of renderUser() for M-001, the flagship bucket's opening message, exactly as the model receives it. No scenario-specific hints, no few-shot examples encoding the right answer.
Extraction, system prompt:
view prompt
You extract factual claims made by the message author about project state.
Emit one object per distinct claim the author makes. Emit an empty array if the message makes no claim.
For each claim:
- "referent": a short noun phrase naming what the claim is about.
- "value": the value or position the author asserts.
- "source_span": copied character-for-character from the message text. Do not paraphrase, do not fix typos, do not add ellipses. This must be an exact substring of the message.
- "modality": one of "assertion", "hedge", "proposal", "question", "reported".
- "assertion": the author states it as fact.
- "hedge": the author softens or qualifies it.
- "proposal": the author is suggesting a change, not stating current fact.
- "question": the author is asking, not asserting.
- "reported": the author is relaying what someone else said or decided, not asserting it themselves.
- "polarity": "positive" if the author affirms the value, "negative" if the author denies or rejects it.
- "attributed_to": when modality is "reported", the name or handle of the person credited with the statement. Otherwise null.
Do not infer claims from any context messages provided; they are shown only to help you resolve pronouns and are not themselves a source of claims.
Output JSON only, matching this schema exactly:
{"claims": [{"referent": string, "value": string, "source_span": string, "modality": "assertion"|"hedge"|"proposal"|"question"|"reported", "polarity": "positive"|"negative", "attributed_to": string|null}]}Message from Meera Iyer (Product Manager): Kicking off planning for the Independence Day event. Working assumption is we go live 12 August, config frozen by the 5th so QA gets a clean week.
Judgment, Guardrailed (binary) scope, system prompt:
view prompt
Decide whether a set of live claims are mutually incompatible statements about the same thing.
Two claims are incompatible if they cannot both be true at the same time. Differences in emphasis, scope, or wording are not incompatibility.
You are told which claims are live. Claims that were superseded, withdrawn, or reported have already been removed from what you see — do not reason about them or try to reconstruct them.
Output "CONTRADICTION" if the live claims conflict, "COMPATIBLE" if they do not.
Output JSON only, matching this schema exactly:
{"verdict": "CONTRADICTION"|"COMPATIBLE", "rationale": string, "conflicting_claim_ids": string[]}
"conflicting_claim_ids" should list the claim ids that are in conflict, or an empty array if COMPATIBLE.Referent: indep_event.launch_date Live claims: - M-001#0 | meera.iyer | 6 Jul 2026, 10:12 AM IST | value: "2026-08-12" | span: "we go live 12 August" - M-002#0 | priya.raghunathan | 15 Jul 2026, 6:22 PM IST | value: "2026-08-15" | span: "Go-live is 15 August"
Try it page
Every run goes through the same runExtractionPipeline/runAdjudicationPipeline the ledger and Signals pages use. This is not a simplified demo path. In replay mode, novel input that doesn't match a committed recording's cache key returns a clear “no recording for this input” error rather than a fabricated result (see ReplayMissError in src/core/model/client.ts). Live mode calls the Vercel AI Gateway server-side only. The API key never reaches the browser, and it is rate-limited to 10 calls per session per 10 minutes, with an automatic fallback to replay on a 429. See “Enabling live mode” below for how to turn it on for a deployment.