Discovery.Ai

Documentation Launch App →
// technical reference

How Discovery.Ai actually works

This is a straight technical account of the running system — the real agent loop, the real evidence sources, the real decisions it makes about identity and relationships, and the real limits it hits. Nothing here is aspirational; where something isn't built yet, it says so.

01 — Overview

A traceable investigation, not a finished answer

Discovery.Ai takes a question and recursively investigates it: an agent decides whether to answer directly, break the question into one narrower sub-question, or admit it's hit a genuine boundary. Each decision is made one step at a time, informed by everything resolved so far — not planned out in a single batch up front. The result is rendered as a live graph you can navigate: zoom into any entity, go deeper to trigger a fresh investigation, or switch the lens (economic, technical, historical, or any dimension you name) and watch the same subject get re-explored through a different frame.

It is deliberately not a knowledge base and doesn't claim to be one. It's a trace of one investigation, backed by real retrieved evidence with honest confidence scores — see Reliability & limits for exactly what that does and doesn't guarantee.

02 — Theory

The knowledge model underneath the code

Before any of this was code, it was a conceptual model of how knowledge itself is structured — the graph, the identity resolver, and the dimension-based lenses all exist to implement this model, not the other way around.

A network, not a taxonomy

The foundation is a network of domains — Physics, Economics, Law, Technology, Psychology, and so on — connected to each other rather than filed into one rigid tree. A domain can sit in many different networks at once; there is no single "correct" parent for anything.

Abstraction: a boundary, not a dimension

An abstraction is a boundary drawn around the part of the network you're currently choosing to study — "for this investigation, this is the portion of the network we're considering." It can be a subject (a bounded region — "Quantum Computing" spans Physics, Mathematics, Information Theory, and Computer Science at once) or an entity (a concrete thing — PayPal, Mastercard, an individual) that exists inside a boundary, across dimensions, with its own internal structure and history. Crucially, an abstraction is cheap to redraw — it's a lens you're holding up to the network right now, not a permanent structural commitment.

Zoom: changing the boundary, not moving through space

Zooming doesn't mean approaching an object — it means changing which boundary is currently the object of study. Zoom in and one entity unfolds into a network of smaller components; zoom out and a whole network collapses into a single node inside a larger one. The underlying reality doesn't change — only which boundary you're currently reading it through.

flowchart TB
  E["PayPal
(one entity)"] -->|"zoom in — unfolds into"| N["Payment Processing · Transaction Processing
Authorization · Fraud Detection"] N -->|"zoom out — collapses into"| E E -->|"zoom out again"| P["Payment Platforms
(PayPal, Mastercard, Stripe, Visa, banks)"] P -->|"zoom out again"| F["Financial System"]

This is also why the structure is a pyramid you can enter from any layer, not a fixed tree the user is forced to descend top-down — PayPal sits under "Payment Platforms" today, but could just as validly sit under "Fintech Case Studies" under a different abstraction drawn around it later. Discovery.Ai's resolve_entity (see Identity resolution) is what makes that non-strict hierarchy actually safe to build without duplicating or colliding entities.

Dimensions: ways of interrogating, not nodes

A dimension is not a thing in the graph — it's a lens you apply to whatever abstraction you're studying, and applying one is what actually generates a question: Abstraction + Dimension → Question. Three dimensions are useful across almost any abstraction; the rest are specific to whatever system you're actually looking at:

Scale — physical → individual → group → organization → society → global
Perspective — economic, psychological, legal, computational, historical…
Time — origin, development, current state, failure, transformation
Custom, per-abstraction — Risk/Settlement/Trust for payments; Architecture/Reliability for systems; Jurisdiction/Precedent for law

The same entity produces a genuinely different question under each lens — this is real, not a template swap:

DimensionQuestion generated for "PayPal"
EconomicHow does PayPal create and capture value?
ComputationalHow does PayPal process transactions?
PsychologicalWhy do users trust PayPal?
LegalWhat legal structures govern its operations?
HistoricalWhy did PayPal emerge when it did?

And the reverse holds too: the same dimension asked at a different scale produces a different question — Economic + Individual asks about income and constraints; Economic + Organization asks about revenue and competition; Economic + Society asks about markets and distribution. This is why the Question Engine is level-aware rather than a flat dimension-to-question lookup — Question = f(Abstraction, Network, Entity, Dimension, Level, Context).

One consequence worth stating explicitly: hierarchy itself is dimension-relative, not intrinsic to the entities. Order the same set of organizations by Scale and you get Individual → Local → Regional → National → Global; order the identical set by Revenue, or Technical dependency, and they line up completely differently. A hierarchy is an emergent structure produced by choosing a dimension, not a fact about the entities themselves.

The full loop, and the four operations it's built from

flowchart LR
  NET["Network"] --> ABS["Abstraction"]
  ABS --> DIM["Dimension"]
  DIM --> Q["Question"]
  Q --> RES["Resource"]
  RES --> K["Knowledge"]
  K -.->|generates| Q
        

Everything a user actually does reduces to four operations over that loop:

OperationIn theoryIn the app today
NavigateMove between connected nodes along existing relationshipsClick a node or edge in the graph view
ZoomChange the abstraction boundary"Zoom into X" / "go deeper into X"
InterrogateApply a dimension to generate a question"Look at this economically" / switch lens
LearnAttach a resource that answers the questionThe Evidence Engine's six real retrievers

Theory vs. what's actually live

The conceptual model above is the original design in full — the running system implements a real, working slice of it, not a simplification pretending to be the whole thing. Dimensions today are freeform named lenses an investigation can carry (not a fixed Scale/Perspective/Time selector UI), decomposition and question-generation are fully live and level-aware, and identity resolution is what makes the non-strict pyramid safe in practice. A dedicated Roadmap feature — turning an explored sub-graph into an ordered, readable learning path rather than free-form exploration — is designed but not yet built.

Two gaps worth naming precisely rather than glossing over. First, drawing and naming a Subject or Entity boundary isn't yet a decision the agent deliberately makes — today's abstraction label is picked incidentally by the intent classifier, not reasoned about the way decompose/answer/boundary_hit are. Second, "zoom in" today returns a one-line summary of what's known about a node rather than opening that node's own internal graph of domains and sub-entities — the real target, already designed (a bounded "tile" read as a View, never persisted as new structure), just not built end-to-end yet. Comparing two entities currently also persists a node for the comparison itself, which it shouldn't — a comparison is a way of looking at two existing things, not a new thing in the world. Full internal detail in docs/Architecture.md §0.6-§0.21.

03 — Architecture

Six layers, each with one job

There's no single monolith making decisions — each layer owns exactly one responsibility, and only the Graph Interface is allowed to talk to Neo4j directly.

graph TD
  FE["Frontend
HTML / CSS / JS + Cytoscape.js"] -->|REST| API["API Layer
FastAPI"] API --> QE["Question Engine
intent · decisions · relation extraction"] API --> GI["Graph Interface
the only code that touches Neo4j"] QE --> LLM["LLM fallback chain
Groq / Gemini / Cerebras"] QE --> AR["Agent Runtime
recursive Ground Agent"] AR --> EE["Evidence Engine
Tavily · Semantic Scholar · arXiv · Open Library · YouTube · Wikipedia"] AR --> GI GI --> NEO[("Neo4j")] API --> PG[("Postgres / Supabase
sessions + API keys")] AR --> SQ[("SQLite
agent checkpoints")]
Solid arrows are real, currently-wired call paths — not a target architecture.

What each layer owns

  • Graph Interface (backend/graph) — find_or_create_entity, resolve_entity, attach_question, attach_claim, create_relationship. Mechanically simple by design; no hierarchy or navigation logic lives here.
  • Agent Runtime (backend/agents, backend/runtime) — one recursive agent class, not a fixed tier of specialized agent types. A "master-level" decision and a "ground-level" decision are the same code path at different recursion depths, using a heavier model only where the decision is structural.
  • Question Engine (backend/questions) — intent classification, the per-step decompose/answer/boundary_hit decision, and relation extraction. Every LLM call in the system funnels through here.
  • Evidence Engine (backend/evidence) — six real retrievers, fanned out concurrently per question, each result synthesized into a typed claim with its own confidence score.
  • API layer (backend/api) — plain REST endpoints (no WebSocket, no push channel). The frontend re-fetches and re-renders after each action.
  • Frontend (frontend/) — a single-page vanilla JS/HTML/CSS app, Cytoscape.js for the graph view. No framework, no build step.
04 — The investigation loop

One decision at a time, not a plan made upfront

Before any investigation starts, a single classification call decides what the message even is — a new question, a navigation request, small talk, or something else — and can just reply conversationally itself rather than forcing every message into a rigid action. Once an investigation actually starts, this is the loop it runs:

flowchart TD
  Q["Question"] --> D{"decide_next_step"}
  D -->|answer| A["Gather evidence + finish"]
  D -->|decompose| C["Spawn ONE child question
at depth + 1"] C --> D2{"decide_next_step
on the child"} D2 -->|answer| A D2 -->|decompose| C D2 -->|boundary_hit| B["Escalate up the parent chain"] D -->|boundary_hit or budget exhausted| B A --> R["Extract relationships
a separate decision from decompose"] R --> ID["resolve_entity on both endpoints"] ID -->|REUSE or CREATE, both sides| W["Write the relationship"] ID -->|AMBIGUOUS or CONFLICT, either side| SK["Skip — never guess"]
"decompose" only ever proposes ONE narrower sub-question at a time — the result folds back in before the next decision is made.

The budget for how many sub-questions one investigation can spawn is a hard ceiling, not a target — a narrow question typically answers directly in one or two steps; a genuinely structural question (data structures, protocols, multi-party systems) uses more of the budget because the model judges there's real, separable structure worth representing as distinct nodes.

05 — Evidence & confidence

Real sources, scored honestly — including when they're wrong

Every answered question can gather evidence from six real, named retrievers: Tavily (general web), Semantic Scholar and arXiv (papers), Open Library (books), the YouTube Data API, and Wikipedia. Each retrieved resource is synthesized into a claim with its own evidence, reasoning, and a confidence score describing how well that specific source actually answers the question — not how confident the model is in general.

The important part: a source that doesn't actually answer the question is still attached to the graph, scored honestly low, instead of being silently dropped. This is a real result from an actual investigation into "Card Payment," inspected directly through the app's node-detail panel:

Real claims attached to one question
Payment processor (Wikipedia)high confidence
Centurion Card (Wikipedia)high confidence
Expected Performance of the ATLAS Experiment (arXiv)~10% confidence
Observation of a rare B⁰ₛ decay (arXiv)~10% confidence

The two arXiv results are CERN particle-physics papers that surfaced from a keyword-level search match, not a topical one — the synthesis step correctly recognized they don't address card payments at all and scored them accordingly, rather than presenting them as if they were equally good evidence.

06 — Identity resolution

Four outcomes, not two

When the system discovers something that might be an entity already in the graph, it can't just do a name match — "Transmission" could mean the electrical-grid concept or the car part, and two different investigations might both legitimately create something called "Router." resolve_entity is a frozen, deterministic contract with four outcomes, not a binary reuse-or-create guess:

flowchart LR
  N["New candidate name"] --> S["Score existing candidates
by real token overlap with context"] S -->|one strong match| REUSE["REUSE
same entity"] S -->|no matching candidates| CREATE["CREATE
genuinely new"] S -->|no evidence either way| AMB["AMBIGUOUS
don't guess"] S -->|multiple real, competing matches| CONF["CONFLICT
don't guess"]

AMBIGUOUS and CONFLICT are kept distinct on purpose — "there's no evidence anywhere" and "there's real, comparable evidence for more than one candidate" are different failure modes, and collapsing them into one shrug would hide which one actually happened. Both are "don't guess" outcomes: a relationship is only ever written to the graph when both its endpoints resolve to REUSE or CREATE.

07 — Typed relationships

Relationships are a separate decision from decomposition

Early on, relationship type was extracted as a rider on the same "decompose" decision — and that framing biased the model toward plain decomposes_into edges even for clearly non-structural facts (an intrusion-detection system spotting a privilege escalation attempt is an actor relationship, not composition). Asking "what decomposes into what" and "how do these things relate" turned out to need genuinely separate prompts, so relation extraction now runs as its own decision after an answer is produced, followed by a canonicalization pass that normalizes active/passive voice inconsistency in what gets extracted.

A more subtle finding from testing this: the same two claims, in identical wording, were correctly classified with three different relationship types across three sessions — sequential, complementary, and alternative-explanation — purely because the question framing each was investigated under changed each time. A relationship isn't a fixed property of two facts; it's a function of the facts and the question they're being read in service of.

08 — Graph topology

Proving the renderer doesn't secretly think in trees

A recurring worry, raised while actually using the graph: does everything eventually collapse back into a decomposition tree, no matter what the real relationships look like? That's testable, and it was tested in two separate passes rather than patched on sight.

Pass one — the renderer alone. Ten synthetic graphs, spanning every topology that matters for a knowledge model, were fed directly into the live renderer with no LLM, no Neo4j, and no navigation logic in the loop — pure "given this exact node/edge data, what gets drawn." All ten passed: composition edges nest into boxes, every other relation type crosses box boundaries as an ordinary edge, convergent and cyclic structure survive without inventing a root or duplicating a node.

TreePass

Composition only — nests into boxes, no stray edges.

flowchart TD
  Payment --> A
  Payment --> B
  Payment --> C
  B --> D
  B --> E
          
NetworkPass

Pure interaction — zero boxes invented.

flowchart LR
  PayPal -->|uses| Mastercard
  PayPal -->|serves| Merchant
  Mastercard -->|routes_to| Acquirer
  Merchant -->|receives| Payment
  PayPal -->|processes| Payment
          
DAGPass

Branch converges on one node, not a duplicate.

flowchart TD
  Request[Payment Request] --> Fraud[Fraud Check]
  Request --> Auth[Authorization]
  Fraud --> Capture
  Auth --> Capture
          
CyclePass

Full loop preserved, no root invented.

flowchart LR
  Controller -->|causes| Plant
  Plant -->|causes| Sensor
  Sensor -->|causes| Controller
          
Nested boxesPass

A space inside a space, independent internal graph.

flowchart TD
  subgraph Payment
    direction TB
    subgraph Stages["Payment Stages"]
      direction TB
      Client --> ClientBank["Client Bank"] --> Network --> MerchantBank["Merchant Bank"] --> Merchant
    end
    PayPal -->|uses| Mastercard
  end
          
Cross-space edgesPass

Relations crossing two boxes stay real edges.

flowchart TD
  subgraph A
    X --> Y
  end
  subgraph B
    Z --> W
  end
  X -->|uses| Z
  Y -->|depends_on| W
          
Workflow + retryPass

Branch, plus a backward retry edge — still a graph.

flowchart TD
  Request --> Validation
  Validation --> Approved
  Validation --> Rejected
  Approved --> Payment
  Payment --> Settlement --> Reconciliation --> Audit --> Exception
  Exception -.->|retry| Payment
          
Nested workflowPass

A space's own internal edge crossing back out.

flowchart TD
  subgraph PP["Payment Process"]
    direction TB
    subgraph Auth["Authorization"]
      direction TB
      Req[Request] --> Val[Validate] --> Dec[Decide]
      Dec --> App[Approve]
      Dec --> Rej[Reject]
    end
    App --> Capture --> Settlement
  end
          
HubPass

High out-degree ≠ a parent — no box formed.

flowchart TD
  A -->|notifies| B
  A -->|queries| C
  A -->|authorizes| D
  A -->|serves| E
          
MeshPass

Dense cross-links, no hierarchy invented.

flowchart LR
  A --- B
  A --- C
  A --- X
  B --- D
  B --- X
  C --- D
  C --- X
  D --- X
          

The renderer's box-vs-edge logic passed 10/10 — renderGraph derives shape from each edge's actual relation family, never from investigation order. But a second, narrower limitation surfaced here too: the focus/zoom viewport (a separate 1-hop "children + parent + siblings" window, built for readability) can silently drop real structure more than one hop from whatever node is focused — a cycle's back-edge, or most of a mesh, depending on which node happens to get clicked. Documented and left alone for a later pass; it's a windowing limitation, not a modeling one.

Pass two — the real pipeline. The harder, more honest question: does the live system, investigating a genuine natural-language question end to end, actually construct these topologies, or only render them when handed one by hand? Three fresh chats, three real investigations, no manual graph editing. A hierarchy question produced a real tree. A relational question — "how do PayPal, Mastercard, Visa, banks, and merchants interact" — correctly produced a network, and incidentally regression-tested the exact PayPal/Mastercard containment bug this project hit early on, this time with fresh model output instead of a manual reproduction:

flowchart TD
  subgraph PayPal
    direction TB
    RiskEngine["risk engine"]
    ProcEngine["Payment Processing Engine"]
    BizModel["Business Model"]
  end
  PayPal -->|USES| Mastercard
  PayPal -->|competes_with| Stripe
  Mastercard -->|CONNECTS| CardBanks["card-issuing banks"]
        
Real output, unedited: Mastercard stayed a peer, not a child, of PayPal — exactly the distinction the earlier fix was for.

A sequence question broke it, and the failure was traced to its actual source instead of patched at the symptom:

Before the fixFlattened

"...lifecycle from request through risk checks, authorization, capture, clearing and settlement" produced this — zero ordering, two stages silently dropped.

flowchart TD
  Q["Payment Lifecycle"] --> RiskChecks["Risk checks"]
  Q --> Authorization
  Q --> Capture
  Q --> CS["Capture & Settlement Process"]
          
After the fixPass

Same question, one prompt sentence added telling extraction that ordering is in scope. Verified directly against Neo4j.

flowchart LR
  RiskChecks["Risk checks"] -->|PRECEDES| Authorization
  Authorization -->|PRECEDES| Capture["Payment Capture"]
  Capture -->|PRECEDES| Clearing
          

The root cause wasn't the renderer, decomposition, or the relation registry — all three passed every test aimed at them. It was one paragraph: the relation-extraction prompt asked for actor, causal, and functional relationships, and simply never mentioned temporal ones, even though precedes/follows already existed as real entries in the relation-type registry. Naming the gap fixed it. Full trace in docs/Memory.md's §0.28 entry.

09 — Reliability & limits

What this doesn't promise

Being honest about limits is part of the design, not an afterthought:

Not a verified knowledge base

This is a trace of what one investigation found, backed by real evidence — not a claim that any subject is now fully or correctly modeled.

Free-tier LLM quotas are real

Groq's daily token cap, Gemini's free-tier daily request limit, and Cerebras needing billing enabled have all been hit simultaneously during real use — an investigation can fail or slow down for reasons that have nothing to do with the question asked. Bringing your own API key (in Settings) runs your investigations on your own quota instead.

Evidence relevance isn't perfect

Keyword-based retrieval sometimes returns off-topic sources (see the arXiv example above). They're scored honestly low, not filtered out — the confidence score is the signal, not a guarantee every attached source is useful.

Synthesis can over-smooth disagreement

When claims genuinely represent competing explanations rather than additive facts, synthesis has been observed presenting them as uniformly complementary. Worth reading the underlying claims yourself when a topic is genuinely contested.

10 — Tech stack

What it actually runs on

API
FastAPI
Plain REST, async throughout.
Graph store
Neo4j
Entities, questions, claims, typed relationships.
Sessions
Postgres (Supabase)
Chat history, per-account API keys.
Agent state
SQLite
Resumable checkpoints per agent step.
LLMs
Groq · Gemini · Cerebras
Free-tier fallback chain via Instructor.
Evidence
6 real retrievers
Tavily, Semantic Scholar, arXiv, Open Library, YouTube, Wikipedia.
Graph UI
Cytoscape.js
Interactive, click-to-inspect nodes.
Frontend
Vanilla JS/HTML/CSS
No framework, no build step.
Auth
Supabase (Google OAuth)
Per-account isolation, no shared state.
11 — Reports

Every verification pass, in order

Claims on this page are backed by tests that actually ran, not asserted from design intent. This is the log: every real verification pass conducted against this system, what it checked, how, and what it found — including the ones that failed on the first attempt. Newest first.

Bounded reachability — Focus/Enter Space unification Implemented & verified

Method: rebuilt Focus and Enter Space on one shared primitive (bounded search + a strictly separate edge-inclusion pass), specified and prototype-tested first, then implemented with the same radius Focus always had — and regression-tested against the full topology matrix, live, after deploying.

Result: a cycle now fully survives Focus with zero information loss; a mesh or DAG that genuinely can't fit in a small view now says so — a visible marker on exactly the node with more beyond it — instead of silently pretending to be complete. Verified deterministic: the same graph and parameters always produce the same view.

Abstraction levels — model validation No new schema needed

Question: can one entity have different valid relational structures at different abstraction levels, in one world model, without a new primitive?

Method: replayed the real payment-lifecycle Neo4j data (below) through the existing Graph Space mechanism — zero new code, zero LLM calls.

Result: entering Authorization correctly exposed its own internal interaction graph while the coarse temporal chain stayed visible as context — same nodes, same edges, confirmed byte-identical before and after. Abstraction reduces to scope in the current model; topology stays a derived read, never stored.

Topology-preserving extraction Bug found & fixed

Method: 10 synthetic topologies fed directly into the renderer, then 3 fresh end-to-end investigations from real natural-language questions — no manual graph editing either time.

Result: renderer 10/10. Live pipeline: tree and network questions correct outright (the network test doubled as a regression test of the containment bug below); a sequence question flattened to a tree with zero ordering. Root-caused to one missing category in the extraction prompt, fixed, and re-verified directly against Neo4j.

Full diagrams and the before/after in Graph topology, above.

Semantic Graph Projections Bug found & fixed

Method: 8-case check of the view-switch invariant — a projection must never write to the graph, never call an LLM, and must report an honest gap rather than invent structure.

Result: world model confirmed byte-identical across every switch. One real bug caught: the reply and the render disagreed about scope when a space was entered — fixed by giving both the same reachability rule, re-verified in both Python and the browser.

Relation evidence & confidence Pass

Method: attach claims to the same relation repeatedly against real Neo4j, checking the native edge never duplicates and confidence tracks a stated formula exactly.

Result: edge count stayed at exactly 1 regardless of claim count; confidence matched the formula to two decimal places for every supports/contradicts ratio tested; an unevidenced relation correctly reported "unknown," never a fabricated default.

Relation-type / family registry Pass

Method: consolidate three independently hand-maintained lists of "which relations mean containment" — one each in extraction, the API, and the frontend — into one registry, grounded in real prior art (part-whole meronymy taxonomy, OWL property characteristics) rather than invented ad hoc.

Result: single source of truth adopted everywhere; this is the registry every later projection and topology test above depends on.

Focus vs. Enter Space acceptance matrix Pass

Method: 8-case live check that focusing on an entity (context stays visible) and entering its own space (context drops, its own subgraph becomes the root) are genuinely different operations, not the same zoom with two names.

Result: all 8 cases passed live, including "Back" correctly restoring prior context and the same node keeping one identity across both views.

Composition vs. interaction containment Bug found & fixed

Method: live use surfaced PayPal rendering as if Mastercard were one of its parts, because an interaction edge (PayPal USES Mastercard) was being treated the same as a structural one.

Result: fixed by checking the relation's family before assigning box containment. This exact bug is the one the network topology test above deliberately re-runs, live, with fresh model output.

Relation-extraction schema stabilization Bug found & fixed

Method: the smaller free-tier model intermittently drifted onto its own field names (subject/predicate/object) instead of the real schema, discarding otherwise-correct extractions.

Result: fixed not by forcing a stricter decoding mode (tried, didn't help) but by renaming the schema to match the vocabulary the model already wanted to use — verified against live Neo4j, drift eliminated.