Agent Memory Persistence: Vector, SQL, or Graph?

Most teams building agents reach for a vector store first. It's the default in almost every tutorial, the first integration listed in LangChain and LlamaIndex docs, and it feels right: you embed memories, retrieve the semantically closest ones, inject them into context. The agent "remembers." The demo looks impressive.

Then you hit production. Your agent needs to know whether a user has already completed an onboarding step. It needs to recall every decision made about a specific contract, in order. It needs to traverse a relationship: "who reported the issue that caused this incident?" None of these questions are similarity searches. They're lookups, aggregations, and graph traversals. A vector store will either fail them outright or return stale, approximate answers that look correct until they're silently wrong.

Agent memory persistence is a backend selection problem, not a retrieval problem. The store you pick should match the structure of your memory, not the marketing material of the tool. At Laxaar we've built agent memory layers across all three paradigms, and we have a clear point of view on when each one earns its place.

What you'll learn

• The three memory types agents actually need
• How vector memory works and where it wins
• When SQL is the right agent memory backend
• When graph databases earn their complexity
• Head-to-head comparison across key dimensions
• Combining backends in a layered memory architecture
• Common mistakes teams make with agent memory
• Frequently Asked Questions

The three memory types agents actually need

Before picking a backend, it's worth naming what agent memory actually contains. Three categories come up repeatedly across production systems:

Episodic memory. The record of past events: what the agent did, what the user said, what happened in prior sessions. "We discussed budget constraints in our last conversation." This is narrative, time-ordered, and often fuzzy to retrieve. You want the relevant past, not the complete past.

Semantic memory. Factual knowledge: product specs, user preferences, domain rules, reference data. "This customer is on the Enterprise plan." Lookups here are often exact. You're not asking "what is approximately true about this customer?" You're asking "what IS true."

Relational memory. The network of connections between entities: who owns what, which events caused which outcomes, how entities relate to each other. "Show me all decisions that affected this project, made by people in this team." Traversals over a graph.

The mistake is defaulting all three to the same backend. Vector stores handle fuzzy episodic retrieval reasonably well. They handle exact semantic lookups poorly and relational traversals not at all.

How vector memory works and where it wins

Vector memory is a storage pattern where memories are encoded as high-dimensional embeddings and retrieved by approximate nearest-neighbor search against a query embedding. The premise: semantically similar things will cluster in the embedding space, so "remind me of conversations about pricing" will surface sessions that discussed cost, budget, rates, and fees, even if none used the word "pricing."

import openai
from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams, PointStruct
import uuid

client = QdrantClient(":memory:")

# Create a collection for episodic agent memory
client.create_collection(
    collection_name="agent_episodes",
    vectors_config=VectorParams(size=1536, distance=Distance.COSINE),
)

def embed(text: str) -> list[float]:
    resp = openai.embeddings.create(model="text-embedding-3-small", input=text)
    return resp.data[0].embedding

def store_episode(session_id: str, content: str, metadata: dict):
    vector = embed(content)
    client.upsert(
        collection_name="agent_episodes",
        points=[PointStruct(
            id=str(uuid.uuid4()),
            vector=vector,
            payload={"session_id": session_id, "content": content, **metadata},
        )],
    )

def recall(query: str, top_k: int = 5) -> list[dict]:
    results = client.search(
        collection_name="agent_episodes",
        query_vector=embed(query),
        limit=top_k,
    )
    return [r.payload for r in results]

Vector memory earns its place in episodic retrieval with open-ended queries, long-term conversational context, and document-grounded knowledge bases. It's the right default when you can't enumerate what the agent will need to recall. The fuzzy match is the point.

The real trade-off: approximate retrieval means you'll occasionally surface the wrong memory and miss a relevant one. In a customer support agent, that's a minor quality degradation. In a financial compliance agent that needs to recall every documented decision, it's a reliability failure.

When SQL is the right agent memory backend

SQL memory is a pattern where agent memories are stored as structured rows in a relational database, retrieved with deterministic queries. No embeddings. No approximate search. Exact answers.

The obvious objection is that SQL "can't handle unstructured memory." That's true if your memory is genuinely unstructured. But most production agent memory is far more structured than teams admit at the start.

User state is structured. Session history has a schema. Preferences, settings, and decisions can all be modeled as rows. The resistance to SQL usually comes from conflating "memory has prose in it" with "memory has no structure." A user preference stored as { user_id, preference_key, value, updated_at } is still relational data, even if value is a text field.

import sqlite3
from datetime import datetime

conn = sqlite3.connect("agent_memory.db")
conn.execute("""
    CREATE TABLE IF NOT EXISTS memories (
        id          INTEGER PRIMARY KEY AUTOINCREMENT,
        agent_id    TEXT NOT NULL,
        user_id     TEXT NOT NULL,
        memory_type TEXT NOT NULL,  -- 'preference', 'fact', 'decision'
        key         TEXT NOT NULL,
        value       TEXT NOT NULL,
        confidence  REAL DEFAULT 1.0,
        created_at  TEXT NOT NULL,
        updated_at  TEXT NOT NULL
    )
""")
conn.execute("CREATE INDEX IF NOT EXISTS idx_user_key ON memories(user_id, key)")
conn.commit()

def remember(agent_id: str, user_id: str, memory_type: str, key: str, value: str):
    now = datetime.utcnow().isoformat()
    conn.execute("""
        INSERT INTO memories (agent_id, user_id, memory_type, key, value, created_at, updated_at)
        VALUES (?, ?, ?, ?, ?, ?, ?)
        ON CONFLICT(user_id, key) DO UPDATE SET value=excluded.value, updated_at=excluded.updated_at
    """, (agent_id, user_id, memory_type, key, value, now, now))
    conn.commit()

def recall_exact(user_id: str, key: str) -> str | None:
    row = conn.execute(
        "SELECT value FROM memories WHERE user_id = ? AND key = ? ORDER BY updated_at DESC LIMIT 1",
        (user_id, key)
    ).fetchone()
    return row[0] if row else None

SQL memory is the right choice when the agent needs exact answers, when memory has a defined schema, when you need aggregations ("how many times has this user asked about refunds?"), or when audit trails matter. It's also dramatically cheaper to operate than a vector store at moderate scale: no embedding inference cost, no ANN index, just a query.

The trade-off: SQL can't answer "what did we discuss that was related to the user's frustration last week?" That's a semantic question and SQL won't help you. SQL wins on determinism; it loses on fuzziness.

When graph databases earn their complexity

Graph memory stores entities as nodes and relationships as edges, queried with path traversal rather than similarity search or row lookup. It earns its complexity when memory has inherent network structure that flat tables can't represent cleanly.

Consider an AI support agent managing a complex software product. The agent needs to know: "Has a similar issue been reported before? If so, what was the resolution path? Were any dependent services affected? Who was involved?" This is a traversal question. A vector store will surface semantically similar issues but can't follow the resolution chain. A SQL join across five tables works but becomes increasingly brittle as the relationship depth grows.

from neo4j import GraphDatabase

driver = GraphDatabase.driver("bolt://localhost:7687", auth=("neo4j", "password"))

def store_incident(issue_id: str, description: str, reporter_id: str, affected_services: list[str]):
    with driver.session() as session:
        session.run("""
            MERGE (i:Issue {id: $issue_id})
            SET i.description = $description
            MERGE (u:User {id: $reporter_id})
            MERGE (u)-[:REPORTED]->(i)
        """, issue_id=issue_id, description=description, reporter_id=reporter_id)

        for svc in affected_services:
            session.run("""
                MERGE (s:Service {name: $svc})
                MERGE (i:Issue {id: $issue_id})
                MERGE (i)-[:AFFECTS]->(s)
            """, svc=svc, issue_id=issue_id)

def find_related_issues(service_name: str) -> list[dict]:
    with driver.session() as session:
        result = session.run("""
            MATCH (i:Issue)-[:AFFECTS]->(s:Service {name: $service_name})
            OPTIONAL MATCH (u:User)-[:REPORTED]->(i)
            RETURN i.id AS issue_id, i.description AS description, u.id AS reporter
            ORDER BY i.id DESC LIMIT 10
        """, service_name=service_name)
        return [dict(r) for r in result]

Graph memory's real strength is relationship-dense domains: knowledge management, organizational memory, dependency tracking, causal chains. If you find yourself writing multi-hop SQL joins to answer questions the agent needs to ask frequently, that's the signal to reach for a graph backend.

The trade-off: graph databases are operationally heavier than Postgres, the query language (Cypher for Neo4j, GQL broadly) adds a learning curve, and cloud-managed options have historically been pricier than managed Postgres. Don't introduce a graph store until you've confirmed the query patterns actually need it.

Head-to-head comparison across key dimensions

Dimension	Vector Store	SQL (Relational)	Graph Database
Query type	Semantic similarity	Exact lookup, aggregation	Path traversal, relationship depth
Memory structure fit	Unstructured / narrative	Structured / tabular	Networked entities and relationships
Retrieval determinism	Approximate	Exact	Exact (for defined paths)
Operational complexity	Medium	Low	High
Cost at scale	Higher (embedding inference + ANN)	Lower	Medium to high
Audit / compliance	Weak	Strong	Strong
Schema flexibility	High	Low to medium	Medium
Best memory type	Episodic, document-grounded	Semantic facts, user state	Relational, causal, organizational

The right read here isn't "pick the one with the most green cells." It's "which row matches your actual memory access patterns?"

Combining backends in a layered memory architecture

Production agents with non-trivial memory needs rarely survive on a single backend. The pattern that works is a tiered memory layer where each backend handles the queries it's actually good at.

A practical layered design:

• Working memory: in-context (the current conversation window, managed by the agent loop itself — no external store needed).
• Episodic memory: vector store for fuzzy recall of past sessions, long-tail user history, and document-grounded facts.
• Semantic / state memory: SQL for user preferences, profile facts, onboarding status, account state. Anything that needs an exact, up-to-date answer.
• Relational memory: graph database for entity relationships, dependency maps, and organizational knowledge. Only bring it in when the domain actually requires it.

The agent's retrieval layer then routes queries to the appropriate backend. This is less glamorous than a single "memory module," but it's how you avoid the failure mode where a vector store returns an approximate answer when the agent needed the exact one.

At Laxaar, when we build AI agents for production workloads, we start with SQL for state and add a vector store only after we've confirmed that the episodic recall problem is real and that semantic similarity is actually a better retrieval signal than a keyed lookup.

Common mistakes teams make with agent memory

Storing everything in a vector store and calling it done. Embeddings are a retrieval mechanism, not a storage format. You're trading query accuracy for query flexibility. That trade-off is only worth it for the queries that genuinely need semantic search.

No memory expiry or relevance decay. An agent that remembers every session equally has increasingly noisy context as history grows. Old memories crowd out recent ones. Production systems need a recency signal: timestamps used in ranking, a confidence score that decays over time, or explicit TTL-based expiry for low-value episodic memories.

Skipping a write schema for the vector store. Because vector stores accept arbitrary text, teams write raw conversation turns, summaries, tool outputs, and user messages all into the same collection with inconsistent structure. At retrieval time, the results are a random mix of memory types that the agent can't usefully differentiate. Standardize a memory record format (even a simple { type, content, metadata }) before you start writing.

Treating graph as a default when SQL would do. Graph databases are a compelling idea and genuinely powerful for relationship-heavy domains. But most "I need to track relationships" problems at the start of a project are two or three join tables in Postgres. Reach for a graph store after Postgres starts fighting you, not before.

For teams exploring AI automation services and custom agent architectures, the memory layer is usually where the production gap lives. Not in the model or the agent loop, but in the backend that stores and retrieves context.

Frequently Asked Questions

Can I start with a vector store and migrate to SQL or graph later?

Yes, but plan for it. If your agent writes memories in a structured format from day one (even if you embed and store them in a vector collection), you can re-ingest the structured data into SQL or a graph store later without re-building the memory writing logic. The migration cost is in re-indexing and updating the retrieval layer, not in the data itself. Don't delay starting because you're not sure which backend to pick; start with the most likely fit and make the write schema portable.

How many memories can a vector store realistically hold before retrieval quality degrades?

This depends on your query patterns and collection hygiene more than raw record count. A well-maintained collection with metadata filtering (scoping retrieval to a specific user or session) can stay accurate at millions of records. The quality degrades when the collection becomes a homogeneous blob: everything embedded, no metadata filtering, top-k returning memories from unrelated contexts. Collection design (namespacing, metadata schemas, filtering at query time) matters more than the total record count.

Does an agent actually need persistent memory, or is in-context enough?

For single-session tasks, in-context often suffices. The case for persistent memory is cross-session continuity: the agent should know the user's stated preferences from three weeks ago without being told again. It should know that a previous run failed at step four and why. If your agent doesn't need to remember anything across sessions, skip persistent memory entirely and save the complexity.

What's the operational cost difference between these options?

SQL (Postgres) is the cheapest to operate. You probably already have it, managed instances are inexpensive, and there's no additional inference cost per write. Vector stores (Pinecone, Qdrant, Weaviate) add embedding inference cost per write and ANN query cost per read; managed options run roughly $70-200/month at moderate scale depending on collection size. Graph databases (Neo4j Aura, Amazon Neptune) are the most expensive to run managed, typically starting at $200+/month for meaningful workloads. The cost differential justifies starting with SQL and vector, and only adding graph when the query patterns demand it.

Can I use a relational database for both exact memory and vector memory?

Yes. Postgres with the pgvector extension supports both row-level exact lookups and ANN vector search in the same database. For early-stage agents or teams that want to minimize infrastructure, pgvector is a pragmatic starting point: SQL for structured memory and vector similarity search in a single database, no separate vector store required. The trade-off is that Postgres with pgvector won't match a purpose-built vector database on throughput at large scale, but for most agent workloads it's more than sufficient.

---

Designing agent memory for a production system and not sure which backend fits your access patterns? The Laxaar team works on agent architectures across industries. Reach out and we'll help you map your memory requirements to the right backend before you've built yourself into a corner.