Memory for AI: a map
Embedders and knowledge graphs — what they are, how they are measured, what memory systems choose.
Embedders and knowledge graphs — what they are, how they are measured, what memory systems choose.
Every system that gives memory to a language model rests on two technologies: embedders, which turn text into vectors so it can be retrieved by meaning, and knowledge graphs, which map its entities and relationships. This page is a sober map of both, with the names that matter and the criteria to judge them.
An embedder converts a piece of text into a vector of numbers, so that texts with similar meaning end up close together. It is the engine of semantic retrieval: you embed the query, find the nearest vectors in storage, and return the matching fragments. Embedders are trained with contrastive learning, pulling related pairs together and pushing unrelated pairs apart across billions of examples.
| Model | Notes |
|---|---|
| all-MiniLM-L6-v2 | Distilled, 384 dimensions, 2021. Small, local on CPU, free. Modest and mostly English. |
| BGE / bge-m3 | Open, self-hostable. bge-m3 is multilingual and long-context. A clear step up from MiniLM. |
| E5 / multilingual-e5 | Microsoft. Strong on retrieval, uses query/passage prefixes. |
| GTE | Alibaba. The large versions sit consistently near the top of the benchmarks. |
| Nomic Embed | Fully open, long context, adjustable dimensions (Matryoshka). |
| OpenAI 3 · Voyage · Cohere v3 | Practical state of the art, but via API: the data leaves the server. |
The shared yardstick is MTEB, the benchmark that ranks models across dozens of tasks. A lever often more effective than swapping the embedder is the reranker: a second stage that reorders the top candidates with a more accurate but slower model.
A knowledge graph is a network of entities (people, companies, projects) connected by typed relationships (works-for, founded, uses). It captures the structure that similarity retrieval cannot see. Building one takes three steps of increasing difficulty: recognising the entities, extracting the relationships, and resolving the duplicates.
Co-occurrence. Two entities appearing in the same fragment get linked by a generic edge. This is the baseline: simple, but with no type, no direction and no entity resolution, and noisy on real data.
Typed extraction. With more flexible NER (GLiNER, language models) and relation extraction (OpenIE, REBEL) you produce edges with a type and a direction: not "A is linked to B", but "A founded B".
GraphRAG. The state of the art (Microsoft): a language model builds the graph from the documents, then community detection (Leiden) groups it into themes with summaries, enabling even whole-corpus "global" questions.
Schema-first. The hand-curated graph, with an explicit schema of allowed types and relationships (the Wikidata model). Maximum precision, manual scale. A set of hand-linked notes with typed links is, in effect, a schema-first graph.
As a concrete example, the open-source memory system Memory Vault combines these building blocks as follows: storage on PostgreSQL with pgvector, hybrid vector plus full-text search fused with RRF, an all-MiniLM-L6-v2 embedder running locally, and a graph built with spaCy for entities and co-occurrence for relationships. It is exposed both as an MCP server and via REST, with a web dashboard. A clean, self-hosted combination of standard components.