How Discovery.Ai actually works
This is a straight technical account of the running system — the real agent loop, the real evidence sources, the real decisions it makes about identity and relationships, and the real limits it hits. Nothing here is aspirational; where something isn't built yet, it says so.
A traceable investigation, not a finished answer
Discovery.Ai takes a question and recursively investigates it: an agent decides whether to answer directly, break the question into one narrower sub-question, or admit it's hit a genuine boundary. Each decision is made one step at a time, informed by everything resolved so far — not planned out in a single batch up front. The result is rendered as a live graph you can navigate: zoom into any entity, go deeper to trigger a fresh investigation, or switch the lens (economic, technical, historical, or any dimension you name) and watch the same subject get re-explored through a different frame.
It is deliberately not a knowledge base and doesn't claim to be one. It's a trace of one investigation, backed by real retrieved evidence with honest confidence scores — see Reliability & limits for exactly what that does and doesn't guarantee.
The knowledge model underneath the code
Before any of this was code, it was a conceptual model of how knowledge itself is structured — the graph, the identity resolver, and the dimension-based lenses all exist to implement this model, not the other way around.
A network, not a taxonomy
The foundation is a network of domains — Physics, Economics, Law, Technology, Psychology, and so on — connected to each other rather than filed into one rigid tree. A domain can sit in many different networks at once; there is no single "correct" parent for anything.
Abstraction: a boundary, not a dimension
An abstraction is a boundary drawn around the part of the network you're currently choosing to study — "for this investigation, this is the portion of the network we're considering." It can be a subject (a bounded region — "Quantum Computing" spans Physics, Mathematics, Information Theory, and Computer Science at once) or an entity (a concrete thing — PayPal, Mastercard, an individual) that exists inside a boundary, across dimensions, with its own internal structure and history. Crucially, an abstraction is cheap to redraw — it's a lens you're holding up to the network right now, not a permanent structural commitment.
Zoom: changing the boundary, not moving through space
Zooming doesn't mean approaching an object — it means changing which boundary is currently the object of study. Zoom in and one entity unfolds into a network of smaller components; zoom out and a whole network collapses into a single node inside a larger one. The underlying reality doesn't change — only which boundary you're currently reading it through.
flowchart TB E["PayPal
(one entity)"] -->|"zoom in — unfolds into"| N["Payment Processing · Transaction Processing
Authorization · Fraud Detection"] N -->|"zoom out — collapses into"| E E -->|"zoom out again"| P["Payment Platforms
(PayPal, Mastercard, Stripe, Visa, banks)"] P -->|"zoom out again"| F["Financial System"]
This is also why the structure is a pyramid you can enter from any layer,
not a fixed tree the user is forced to descend top-down — PayPal sits under "Payment
Platforms" today, but could just as validly sit under "Fintech Case Studies" under a
different abstraction drawn around it later. Discovery.Ai's resolve_entity
(see Identity resolution) is what makes that
non-strict hierarchy actually safe to build without duplicating or colliding entities.
Dimensions: ways of interrogating, not nodes
A dimension is not a thing in the graph — it's a lens you apply to whatever abstraction you're studying, and applying one is what actually generates a question: Abstraction + Dimension → Question. Three dimensions are useful across almost any abstraction; the rest are specific to whatever system you're actually looking at:
The same entity produces a genuinely different question under each lens — this is real, not a template swap:
| Dimension | Question generated for "PayPal" |
|---|---|
| Economic | How does PayPal create and capture value? |
| Computational | How does PayPal process transactions? |
| Psychological | Why do users trust PayPal? |
| Legal | What legal structures govern its operations? |
| Historical | Why did PayPal emerge when it did? |
And the reverse holds too: the same dimension asked at a different scale produces
a different question — Economic + Individual asks about income and constraints; Economic +
Organization asks about revenue and competition; Economic + Society asks about markets and
distribution. This is why the Question Engine is level-aware rather than a flat
dimension-to-question lookup — Question = f(Abstraction, Network, Entity, Dimension,
Level, Context).
One consequence worth stating explicitly: hierarchy itself is dimension-relative, not intrinsic to the entities. Order the same set of organizations by Scale and you get Individual → Local → Regional → National → Global; order the identical set by Revenue, or Technical dependency, and they line up completely differently. A hierarchy is an emergent structure produced by choosing a dimension, not a fact about the entities themselves.
The full loop, and the four operations it's built from
flowchart LR
NET["Network"] --> ABS["Abstraction"]
ABS --> DIM["Dimension"]
DIM --> Q["Question"]
Q --> RES["Resource"]
RES --> K["Knowledge"]
K -.->|generates| Q
Everything a user actually does reduces to four operations over that loop:
| Operation | In theory | In the app today |
|---|---|---|
| Navigate | Move between connected nodes along existing relationships | Click a node or edge in the graph view |
| Zoom | Change the abstraction boundary | "Zoom into X" / "go deeper into X" |
| Interrogate | Apply a dimension to generate a question | "Look at this economically" / switch lens |
| Learn | Attach a resource that answers the question | The Evidence Engine's six real retrievers |
Theory vs. what's actually live
The conceptual model above is the original design in full — the running system implements a real, working slice of it, not a simplification pretending to be the whole thing. Dimensions today are freeform named lenses an investigation can carry (not a fixed Scale/Perspective/Time selector UI), decomposition and question-generation are fully live and level-aware, and identity resolution is what makes the non-strict pyramid safe in practice. A dedicated Roadmap feature — turning an explored sub-graph into an ordered, readable learning path rather than free-form exploration — is designed but not yet built.
Two gaps worth naming precisely rather than glossing over. First, drawing and naming a
Subject or Entity boundary isn't yet a decision the agent deliberately makes — today's
abstraction label is picked incidentally by the intent classifier, not reasoned about the
way decompose/answer/boundary_hit are. Second, "zoom
in" today returns a one-line summary of what's known about a node rather than opening that
node's own internal graph of domains and sub-entities — the real target, already designed
(a bounded "tile" read as a View, never persisted as new structure), just not built
end-to-end yet. Comparing two entities currently also persists a node for the comparison
itself, which it shouldn't — a comparison is a way of looking at two existing things, not a
new thing in the world. Full internal detail in docs/Architecture.md §0.6-§0.21.
Six layers, each with one job
There's no single monolith making decisions — each layer owns exactly one responsibility, and only the Graph Interface is allowed to talk to Neo4j directly.
graph TD FE["Frontend
HTML / CSS / JS + Cytoscape.js"] -->|REST| API["API Layer
FastAPI"] API --> QE["Question Engine
intent · decisions · relation extraction"] API --> GI["Graph Interface
the only code that touches Neo4j"] QE --> LLM["LLM fallback chain
Groq / Gemini / Cerebras"] QE --> AR["Agent Runtime
recursive Ground Agent"] AR --> EE["Evidence Engine
Tavily · Semantic Scholar · arXiv · Open Library · YouTube · Wikipedia"] AR --> GI GI --> NEO[("Neo4j")] API --> PG[("Postgres / Supabase
sessions + API keys")] AR --> SQ[("SQLite
agent checkpoints")]
What each layer owns
- Graph Interface (
backend/graph) —find_or_create_entity,resolve_entity,attach_question,attach_claim,create_relationship. Mechanically simple by design; no hierarchy or navigation logic lives here. - Agent Runtime (
backend/agents,backend/runtime) — one recursive agent class, not a fixed tier of specialized agent types. A "master-level" decision and a "ground-level" decision are the same code path at different recursion depths, using a heavier model only where the decision is structural. - Question Engine (
backend/questions) — intent classification, the per-step decompose/answer/boundary_hit decision, and relation extraction. Every LLM call in the system funnels through here. - Evidence Engine (
backend/evidence) — six real retrievers, fanned out concurrently per question, each result synthesized into a typed claim with its own confidence score. - API layer (
backend/api) — plain REST endpoints (no WebSocket, no push channel). The frontend re-fetches and re-renders after each action. - Frontend (
frontend/) — a single-page vanilla JS/HTML/CSS app, Cytoscape.js for the graph view. No framework, no build step.
One decision at a time, not a plan made upfront
Before any investigation starts, a single classification call decides what the message even is — a new question, a navigation request, small talk, or something else — and can just reply conversationally itself rather than forcing every message into a rigid action. Once an investigation actually starts, this is the loop it runs:
flowchart TD
Q["Question"] --> D{"decide_next_step"}
D -->|answer| A["Gather evidence + finish"]
D -->|decompose| C["Spawn ONE child question
at depth + 1"]
C --> D2{"decide_next_step
on the child"}
D2 -->|answer| A
D2 -->|decompose| C
D2 -->|boundary_hit| B["Escalate up the parent chain"]
D -->|boundary_hit or budget exhausted| B
A --> R["Extract relationships
a separate decision from decompose"]
R --> ID["resolve_entity on both endpoints"]
ID -->|REUSE or CREATE, both sides| W["Write the relationship"]
ID -->|AMBIGUOUS or CONFLICT, either side| SK["Skip — never guess"]
The budget for how many sub-questions one investigation can spawn is a hard ceiling, not a target — a narrow question typically answers directly in one or two steps; a genuinely structural question (data structures, protocols, multi-party systems) uses more of the budget because the model judges there's real, separable structure worth representing as distinct nodes.
Real sources, scored honestly — including when they're wrong
Every answered question can gather evidence from six real, named retrievers: Tavily
(general web), Semantic Scholar and arXiv (papers),
Open Library (books), the YouTube Data API, and Wikipedia.
Each retrieved resource is synthesized into a claim with its own evidence,
reasoning, and a confidence score describing how well that
specific source actually answers the question — not how confident the model is in
general.
The important part: a source that doesn't actually answer the question is still attached to the graph, scored honestly low, instead of being silently dropped. This is a real result from an actual investigation into "Card Payment," inspected directly through the app's node-detail panel:
The two arXiv results are CERN particle-physics papers that surfaced from a keyword-level search match, not a topical one — the synthesis step correctly recognized they don't address card payments at all and scored them accordingly, rather than presenting them as if they were equally good evidence.
Four outcomes, not two
When the system discovers something that might be an entity already in the graph, it
can't just do a name match — "Transmission" could mean the electrical-grid concept or the
car part, and two different investigations might both legitimately create something
called "Router." resolve_entity is a frozen, deterministic contract with four
outcomes, not a binary reuse-or-create guess:
flowchart LR N["New candidate name"] --> S["Score existing candidates
by real token overlap with context"] S -->|one strong match| REUSE["REUSE
same entity"] S -->|no matching candidates| CREATE["CREATE
genuinely new"] S -->|no evidence either way| AMB["AMBIGUOUS
don't guess"] S -->|multiple real, competing matches| CONF["CONFLICT
don't guess"]
AMBIGUOUS and CONFLICT are kept distinct on purpose — "there's no evidence anywhere" and "there's real, comparable evidence for more than one candidate" are different failure modes, and collapsing them into one shrug would hide which one actually happened. Both are "don't guess" outcomes: a relationship is only ever written to the graph when both its endpoints resolve to REUSE or CREATE.
Relationships are a separate decision from decomposition
Early on, relationship type was extracted as a rider on the same "decompose" decision —
and that framing biased the model toward plain decomposes_into edges even for
clearly non-structural facts (an intrusion-detection system spotting a privilege
escalation attempt is an actor relationship, not composition). Asking "what decomposes into
what" and "how do these things relate" turned out to need genuinely separate prompts, so
relation extraction now runs as its own decision after an answer is produced, followed by a
canonicalization pass that normalizes active/passive voice inconsistency in what gets
extracted.
A more subtle finding from testing this: the same two claims, in identical wording, were correctly classified with three different relationship types across three sessions — sequential, complementary, and alternative-explanation — purely because the question framing each was investigated under changed each time. A relationship isn't a fixed property of two facts; it's a function of the facts and the question they're being read in service of.
Proving the renderer doesn't secretly think in trees
A recurring worry, raised while actually using the graph: does everything eventually collapse back into a decomposition tree, no matter what the real relationships look like? That's testable, and it was tested in two separate passes rather than patched on sight.
Pass one — the renderer alone. Ten synthetic graphs, spanning every topology that matters for a knowledge model, were fed directly into the live renderer with no LLM, no Neo4j, and no navigation logic in the loop — pure "given this exact node/edge data, what gets drawn." All ten passed: composition edges nest into boxes, every other relation type crosses box boundaries as an ordinary edge, convergent and cyclic structure survive without inventing a root or duplicating a node.
Composition only — nests into boxes, no stray edges.
flowchart TD
Payment --> A
Payment --> B
Payment --> C
B --> D
B --> E
Pure interaction — zero boxes invented.
flowchart LR
PayPal -->|uses| Mastercard
PayPal -->|serves| Merchant
Mastercard -->|routes_to| Acquirer
Merchant -->|receives| Payment
PayPal -->|processes| Payment
Branch converges on one node, not a duplicate.
flowchart TD
Request[Payment Request] --> Fraud[Fraud Check]
Request --> Auth[Authorization]
Fraud --> Capture
Auth --> Capture
Full loop preserved, no root invented.
flowchart LR
Controller -->|causes| Plant
Plant -->|causes| Sensor
Sensor -->|causes| Controller
A space inside a space, independent internal graph.
flowchart TD
subgraph Payment
direction TB
subgraph Stages["Payment Stages"]
direction TB
Client --> ClientBank["Client Bank"] --> Network --> MerchantBank["Merchant Bank"] --> Merchant
end
PayPal -->|uses| Mastercard
end
Relations crossing two boxes stay real edges.
flowchart TD
subgraph A
X --> Y
end
subgraph B
Z --> W
end
X -->|uses| Z
Y -->|depends_on| W
Branch, plus a backward retry edge — still a graph.
flowchart TD
Request --> Validation
Validation --> Approved
Validation --> Rejected
Approved --> Payment
Payment --> Settlement --> Reconciliation --> Audit --> Exception
Exception -.->|retry| Payment
A space's own internal edge crossing back out.
flowchart TD
subgraph PP["Payment Process"]
direction TB
subgraph Auth["Authorization"]
direction TB
Req[Request] --> Val[Validate] --> Dec[Decide]
Dec --> App[Approve]
Dec --> Rej[Reject]
end
App --> Capture --> Settlement
end
High out-degree ≠ a parent — no box formed.
flowchart TD
A -->|notifies| B
A -->|queries| C
A -->|authorizes| D
A -->|serves| E
Dense cross-links, no hierarchy invented.
flowchart LR
A --- B
A --- C
A --- X
B --- D
B --- X
C --- D
C --- X
D --- X
The renderer's box-vs-edge logic passed 10/10 — renderGraph derives shape from each
edge's actual relation family, never from investigation order. But a second, narrower limitation
surfaced here too: the focus/zoom viewport (a separate 1-hop "children + parent + siblings" window,
built for readability) can silently drop real structure more than one hop from whatever node is
focused — a cycle's back-edge, or most of a mesh, depending on which node happens to get clicked.
Documented and left alone for a later pass; it's a windowing limitation, not a modeling one.
Pass two — the real pipeline. The harder, more honest question: does the live system, investigating a genuine natural-language question end to end, actually construct these topologies, or only render them when handed one by hand? Three fresh chats, three real investigations, no manual graph editing. A hierarchy question produced a real tree. A relational question — "how do PayPal, Mastercard, Visa, banks, and merchants interact" — correctly produced a network, and incidentally regression-tested the exact PayPal/Mastercard containment bug this project hit early on, this time with fresh model output instead of a manual reproduction:
flowchart TD
subgraph PayPal
direction TB
RiskEngine["risk engine"]
ProcEngine["Payment Processing Engine"]
BizModel["Business Model"]
end
PayPal -->|USES| Mastercard
PayPal -->|competes_with| Stripe
Mastercard -->|CONNECTS| CardBanks["card-issuing banks"]
A sequence question broke it, and the failure was traced to its actual source instead of patched at the symptom:
"...lifecycle from request through risk checks, authorization, capture, clearing and settlement" produced this — zero ordering, two stages silently dropped.
flowchart TD
Q["Payment Lifecycle"] --> RiskChecks["Risk checks"]
Q --> Authorization
Q --> Capture
Q --> CS["Capture & Settlement Process"]
Same question, one prompt sentence added telling extraction that ordering is in scope. Verified directly against Neo4j.
flowchart LR
RiskChecks["Risk checks"] -->|PRECEDES| Authorization
Authorization -->|PRECEDES| Capture["Payment Capture"]
Capture -->|PRECEDES| Clearing
The root cause wasn't the renderer, decomposition, or the relation registry — all three passed
every test aimed at them. It was one paragraph: the relation-extraction prompt asked for actor,
causal, and functional relationships, and simply never mentioned temporal ones, even though
precedes/follows already existed as real entries in the relation-type
registry. Naming the gap fixed it. Full trace in docs/Memory.md's §0.28 entry.
What this doesn't promise
Being honest about limits is part of the design, not an afterthought:
This is a trace of what one investigation found, backed by real evidence — not a claim that any subject is now fully or correctly modeled.
Groq's daily token cap, Gemini's free-tier daily request limit, and Cerebras needing billing enabled have all been hit simultaneously during real use — an investigation can fail or slow down for reasons that have nothing to do with the question asked. Bringing your own API key (in Settings) runs your investigations on your own quota instead.
Keyword-based retrieval sometimes returns off-topic sources (see the arXiv example above). They're scored honestly low, not filtered out — the confidence score is the signal, not a guarantee every attached source is useful.
When claims genuinely represent competing explanations rather than additive facts, synthesis has been observed presenting them as uniformly complementary. Worth reading the underlying claims yourself when a topic is genuinely contested.
What it actually runs on
Every verification pass, in order
Claims on this page are backed by tests that actually ran, not asserted from design intent. This is the log: every real verification pass conducted against this system, what it checked, how, and what it found — including the ones that failed on the first attempt. Newest first.
Method: rebuilt Focus and Enter Space on one shared primitive (bounded search + a strictly separate edge-inclusion pass), specified and prototype-tested first, then implemented with the same radius Focus always had — and regression-tested against the full topology matrix, live, after deploying.
Result: a cycle now fully survives Focus with zero information loss; a mesh or DAG that genuinely can't fit in a small view now says so — a visible marker on exactly the node with more beyond it — instead of silently pretending to be complete. Verified deterministic: the same graph and parameters always produce the same view.
Question: can one entity have different valid relational structures at different abstraction levels, in one world model, without a new primitive?
Method: replayed the real payment-lifecycle Neo4j data (below) through the existing Graph Space mechanism — zero new code, zero LLM calls.
Result: entering Authorization correctly exposed its own
internal interaction graph while the coarse temporal chain stayed visible as context — same
nodes, same edges, confirmed byte-identical before and after. Abstraction reduces to scope in the
current model; topology stays a derived read, never stored.
Method: 10 synthetic topologies fed directly into the renderer, then 3 fresh end-to-end investigations from real natural-language questions — no manual graph editing either time.
Result: renderer 10/10. Live pipeline: tree and network questions correct outright (the network test doubled as a regression test of the containment bug below); a sequence question flattened to a tree with zero ordering. Root-caused to one missing category in the extraction prompt, fixed, and re-verified directly against Neo4j.
Full diagrams and the before/after in Graph topology, above.
Method: 8-case check of the view-switch invariant — a projection must never write to the graph, never call an LLM, and must report an honest gap rather than invent structure.
Result: world model confirmed byte-identical across every switch. One real bug caught: the reply and the render disagreed about scope when a space was entered — fixed by giving both the same reachability rule, re-verified in both Python and the browser.
Method: attach claims to the same relation repeatedly against real Neo4j, checking the native edge never duplicates and confidence tracks a stated formula exactly.
Result: edge count stayed at exactly 1 regardless of claim count; confidence matched the formula to two decimal places for every supports/contradicts ratio tested; an unevidenced relation correctly reported "unknown," never a fabricated default.
Method: consolidate three independently hand-maintained lists of "which relations mean containment" — one each in extraction, the API, and the frontend — into one registry, grounded in real prior art (part-whole meronymy taxonomy, OWL property characteristics) rather than invented ad hoc.
Result: single source of truth adopted everywhere; this is the registry every later projection and topology test above depends on.
Method: 8-case live check that focusing on an entity (context stays visible) and entering its own space (context drops, its own subgraph becomes the root) are genuinely different operations, not the same zoom with two names.
Result: all 8 cases passed live, including "Back" correctly restoring prior context and the same node keeping one identity across both views.
Method: live use surfaced PayPal rendering as if Mastercard were one of
its parts, because an interaction edge (PayPal USES Mastercard) was being treated the
same as a structural one.
Result: fixed by checking the relation's family before assigning box containment. This exact bug is the one the network topology test above deliberately re-runs, live, with fresh model output.
Method: the smaller free-tier model intermittently drifted onto its
own field names (subject/predicate/object) instead of the
real schema, discarding otherwise-correct extractions.
Result: fixed not by forcing a stricter decoding mode (tried, didn't help) but by renaming the schema to match the vocabulary the model already wanted to use — verified against live Neo4j, drift eliminated.