Skip to main content

​
One graph, two granularities

Memories and ingested documents live in one graph, connected through the entities they mention. Your saved belief “the ingest worker restarts via systemd” and the ops runbook you ingested both mention the entity ingest worker, so the graph puts them two hops apart, and recall’s entity arm follows the same edges. The two granularities:
  • Storage links at chunk granularity. Entity edges point at individual chunks, so the engine knows which section of a document mentions what.
  • Display projects at document granularity. The graph API rolls chunk-level edges up into one weighted document → entity edge (weight = how many chunks mention the entity), so a 300-page PDF shows as one node instead of 300.

​
Node and edge types

Only active memories and edges appear; stale versions and reverted resolutions drop out.

​
How indexing works

By default, indexing runs automatically: a few seconds after you save a memory or add a file, the daemon extracts entities in the background (debounced, so a folder ingest becomes one run), and the runs show up in the Console like any other. Saves and ingests stay fast because the LLM work never blocks them. Turn it off with memloom auto-index off (or the Console’s toggle, or MEMLOOM_AUTO_INDEX=off as the config default) to make indexing explicit again; offline mode has no auto-indexing since extraction needs the LLM. Extraction is schema-constrained and precision-first: it would rather miss an entity than add a wrong one.
  1. Every unindexed memory and context chunk goes to the LLM with an extraction prompt built from the graph schema: a closed vocabulary of 8 entity types (person, organization, project, tool, technology, place, event, and concept, which is explicitly a last resort). The prompt instructs the model to abstain when in doubt and caps output at the 5 most salient entities per text.
  2. A deterministic validator checks every extraction: it drops anything with an out-of-vocab type, anything shaped like a math expression (f(x) = x^3 - 2x^2), sentence fragments (over 40 characters or 5 words), and names with fewer than two letters.
  3. Formula-dominated chunks skip the LLM entirely: a cheap math-density check keeps math worksheets from becoming graphs of equations (those runs log skipped: math-dense).
  4. Surviving entities resolve to a node by name key: casefolded, trimmed, a leading @ stripped, whitespace collapsed. @memloom/core in one document and memloom/core in another land on the same node, and the first-seen spelling and type stay (the node’s identity is its name key alone; type is an attribute, so an extractor that says technology today and project tomorrow cannot fork the node). The prompt also lists the names already in your graph, so the model reuses canonical spellings instead of minting variants. A mention edge links the source to each resolved entity.
  5. The same call also extracts typed relationships between entities, classified against a closed predicate vocabulary: uses, depends_on, part_of, created_by, works_on, works_at, located_in, attended. Out-of-vocab predicates and claims below 0.7 confidence are quarantined as plain mention edges.
Indexing is idempotent (rows are stamped indexed_at) and incremental: re-running it only touches what’s new. One LLM call per row, so a large PDF makes an index run proportionally longer. GET /memory/schema reports the vocabularies with live usage counts. If an older, noisier pipeline already polluted your graph, rebuild it from scratch:
Keep in mind that indexing with LLMs is not deterministic: the same input can give different extractions every time. Belief edges (replaces, distinct) are never touched by a rebuild. Note that rebuilding needs the LLM: run it offline and the wipe still happens. The store stays consistent (everything is unindexed), but nothing is re-extracted until a key is configured. Indexed entities also feed retrieval: the entity arm of hybrid recall anchors a query to entities and pulls in whatever mentions them, including document sections whose wording shares nothing with your query. See Retrieval.

​
The schema

The vocabularies live in a registry (the viewer’s schema tab, GET /memory/schema, memloom schema in the CLI). There is no code to fork:
  • Add your own. A user-tier entity type or predicate (say, medication) joins the prompt on the next index run. The description you write is the extraction rule, so write it like one: “a named drug or supplement” rather than “medications”.
  • Proposals. When the extractor confidently wants a type or predicate the schema lacks, the name is held out of the graph and queued for review. Approve it and it becomes a user entry; dismiss it and the prompt blocklists the name from ever being proposed again. A name only surfaces after two independent extractions asked for it.
  • Disable anything, built-in or yours. Disabled entries leave the prompt; entities already extracted under them stay in the graph.
  • Delete a disabled user entry when you’re sure. Built-ins can only be disabled: the system tier is re-seeded by name, so a deleted built-in would come back active. Proposals can only be approved or dismissed; there is no delete for them.

​
Managing entities

Extraction is precision-first but not perfect, so the schema tab also lists every entity in the graph with its usage (mentions, documents, memories) and the tools to correct it:
  • Rename. The node re-embeds under the new name. Renaming onto an existing entity’s name is refused; that situation is a merge.
  • Retype to any active vocabulary type.
  • Merge into another entity: every mention and typed relationship moves to the survivor (duplicates and would-be self-loops are dropped), then the duplicate node is deleted. This is the fix for near-duplicates the name key could not fold.
  • Delete an entity and every edge touching it. The memories and documents that mentioned it are untouched.
The HTTP surface is GET /memory/entities, PATCH /memory/entities/{id}, POST /memory/entities/{id}/merge, and DELETE /memory/entities/{id}. memloom ui → the Graph tab:
  • Click a document diamond to expand it: its chunks appear as squares with their real chunk-level entity edges, and the rolled-up document edges swap out.
  • Click a chunk to read it in the side panel, with breadcrumb navigation (document › chunk) to move back up.
  • A document’s side panel lists all its chunks and has an Open file button that opens the source file with your OS default app.
The daemon only ever opens files it previously ingested (the lookup is by document id, never by a path from the request), so the open button can’t be pointed at arbitrary files.