At Fiatbitcoin we currently run about a dozen Jido agents in parallel: portfolio, tax, on-chain monitoring, anomaly detection, daily briefings. Sooner or later you notice this pattern:
Agent A knows something. Agent B could use it. But B never asks, because A keeps that knowledge in its own little piece of state.
Not an edge case, just the default state of every multi-agent system until you actively build against it. So we built the Brain: a centralized knowledge store that all our agents write to and read from. Recently also Claude Code, over an MCP server.
A few of the architecture decisions held up well in production. One I would approach differently today.
The problem: agent silos
Until recently every agent had its own little memory. Or none at all, depending on how it was built. In practice that meant:
- The PerformanceTaxAgent computed a portfolio summary every night but had no reference to yesterday's summary.
- The WhaleSensor logged suspicious on-chain movements to the logger. The AnomalyDetectionAgent would have loved that signal but had to re-detect it itself.
- When Claude Code helped in the repo, it was blind to whatever observations the production agents had collected.
Cross-agent context had to be wired up by hand. Agent A calls agent B, which then calls agent C. That scales neither cognitively nor in code.
The inspiration came from eToro's Company Brain. We borrowed the core ideas (semantic memory types, write-time enrichment, visibility scopes, hybrid search, knowledge graph with entity extraction) and adapted them pragmatically to our stack: Phoenix umbrella, Elixir, Postgres with pgvector, Jido for the agents.
Three layers
┌─────────────────────────────────────────────────┐
Claude │ ACCESS │
Code │ POST /mcp (tools + resources) │
│ GET /mcp/health │
Jido │ Backend.Brain.Recorder.record/3 │
Agents │ Backend.Brain.{write, recall, relate, ...} │
└─────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────┐
│ CORE │
│ Write pipeline (auto-embed, telemetry) │
│ Recall: vector | FTS | hybrid | entity-only │
│ Trust-tier enforcement │
└─────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────┐
│ PERSISTENCE (PostgreSQL + pgvector) │
│ brain_memories brain_entities ... │
└─────────────────────────────────────────────────┘
On top sits the access layer, with two ways in: an in-cluster path via a direct Elixir function call for the Jido agents, and an out-of-cluster path over HTTP/JSON-RPC for MCP clients like Claude Code. Both land in the same storage. That's the whole idea, more or less.
Memory types: semantic, not structureless
A centralized store turns into a landfill fast if you throw everything in as "text plus tags". We committed early to a small, semantically distinct set of types:
| Type | Use | Example |
|---|
:fact | Static, verifiable statement | "Avg buy price user 1 = 32,450 EUR" |
:episode | Time-bound event | "Whale movement of 1,500 BTC in block 800001" |
:decision | Deliberate decision | "User limit set to 5,000 EUR / month" |
:rule | Learned rule / constraint | "Skip auto-buys when volatility > 0.05" |
:plan | Planned future action | "Tax-free sale on 2026-09-15" |
:preference | User / agent preference | "User prefers EUR-denominated reports" |
:action / :outcome | Action and its result | "Webhook X fired → 200 OK, 1.2s" |
On top of the type, each memory carries a few more fields: a scope (:agent_private, :team or :org), a trust tier (:verified, :observed, :asserted), a lifecycle state (:active, :stale, :superseded, :archived), a confidence value, and a source_agent_id for provenance.
Sounds like a lot of metadata. It is. But each column does real work. Scope makes visibility explicit. Trust tier prevents MCP clients from declaring themselves :verified (that's enforced server-side). Lifecycle allows gentle aging instead of hard deletion.
Reading: four recall modes under one API
Backend.Brain.recall/1 is polymorphic in its input:
# Full-text search
Brain.recall(query: "tax-free positions", limit: 5)
# "What do we know about user 1?"
Brain.recall(entity: {"user", "u-1"})
# Full-text search, but only knowledge about user 1
Brain.recall(query: "loss", entity: {"user", "u-1"})
# Hybrid: vector + FTS, dedupe by memory_id, max score
Brain.recall(query: "volatility", limit: 10)
Under the hood: pgvector with an HNSW index for semantic search, Postgres FTS with a tsvector generated column and GIN index for full text. Both compose. When the embedder is active, a text query gets auto-embedded and the call switches to hybrid mode.
The embedder runs in-process, no external service and no outbound API calls. We use Bumblebee to load sentence-transformers/all-MiniLM-L6-v2 from Hugging Face (384 dimensions, L2-normalized, a one-time download of about 90 MB to ~/.bumblebee/ on first start). Inference goes through Nx.Serving with the EXLA compiler.
The advantage of a single function with polymorphic input is that callers don't have to decide which kind of search they need. They describe what they have (query? embedding? entity? combination?), and the Brain picks the right path.
The knowledge graph
Memories aren't just a flat pool. Two link types turn them into a navigable graph:
- Memory to entity over
brain_memory_entities. Each memory can reference any number of canonical entities (user, BTC address, symbol, date, error fingerprint), each with a role (:subject, :object, or :mentioned).
- Memory to memory over
brain_memory_edges. Typed edges with a fixed enum: supersedes, supports, contradicts, derives_from, related_to, observed_with.
A production example: our GenerateSummary action links every new portfolio summary to its predecessor via newer --derives_from--> older. The history of a user's daily portfolio summaries becomes a visible path in the graph, and along that path automatically hang the entities (user, symbols, time window) each summary referenced.
For the admin LiveView visualization we have an edge-centric snapshot loader. In addition to the primary set (most recent N memories) we pull the most recent K edges from the entire DB and lazy-load missing endpoint memories as external: true. Without that trick, edges from agents with long horizons drop out of the view. The AgentPerformanceTrackerAgent, for example, writes one edge per week, and in a pure time-window heuristic it would disappear immediately.
Lessons learned
Three things I would recommend to anyone building this. And one I would approach differently today.
1. Don't build a generic "all action results into the Brain" hook.
The temptation was strong: a wrapper that automatically persists every agent outcome. The result would have been a pile of half-structured, semantically worthless memories. Instead we committed to deliberate write sites per agent, via the Recorder pattern, with an explicitly chosen type. Brain garbage is harder to clean up than Brain gaps.
2. Enforce the trust tier server-side.
For MCP writes we map the agent_id to a trust tier through an allow-list. An external client can't declare itself :verified, no matter what the payload says. It feels like overhead at first, but it's the property that separates a shared memory from a shared trash can.
3. Knowledge graph with a fixed relation enum, not free text.
Six relation types. Ecto validate_inclusion plus a DB CHECK constraint. Sounds restrictive, but it's exactly what makes the graph queryable and visualizable. User-defined relations are the kind of flexibility you never need and always regret.
4. What I would do differently today: async writes from day one.
Brain.write/1 runs synchronously. Without embedding that's a few milliseconds, with embedding 30 to 80. Fine at our current write rate. But every agent that just wants to record a memory pays the embedding latency in the hot path. A synchronous insert plus an async embed-backfill via an Oban job would have been more robust. We have an EmbedBackfillWorker today as a fallback, but that's cosmetics on top of an architecture problem.
If any detail interests you (the embedder pipeline, the MCP tool schema, the derives_from pattern, the edge-centric graph snapshot), drop it in the comments. Happy to go deeper.
Kommentare