How can I use Neo4j as long-term memory for AI agents?

Most AI agents today are surprisingly forgetful. They can reason well for a single interaction, but once the context window resets, everything they’ve “learned” vanishes. Using Neo4j as long-term memory for AI agents solves this by giving your agents a structured, queryable memory that grows over time and supports complex reasoning.

This guide walks through how to use Neo4j as long-term memory for AI agents, covering the concepts, architecture, data modeling, and implementation patterns you need.

---

Why Neo4j is a strong fit for long-term AI memory

Using Neo4j as long-term memory for AI agents aligns naturally with how knowledge and experiences are structured:

• Graph-native structure
  Human knowledge is inherently relational: people, events, facts, causes, references. Graphs capture this as nodes and relationships, making it easy to traverse “who did what, when, and why” in a single query.

• Flexible schema for evolving memory
  Agents often learn new fact types over time. Neo4j’s schema-optional model lets you add new node labels and relationship types without rigid migrations.

• Rich querying beyond vector similarity
  Vector stores are great for semantic search, but limited for reasoning like:

  • “Show all tasks Alice completed after the last product update.”
  • “Find conflicting facts about the same customer.”
    Neo4j lets you combine semantic retrieval (via embeddings) with graph traversal and constraints.

• Context assembly for LLM prompts
  You can query the graph to build compact, relevant context: connected conversations, related documents, user preferences, and historical decisions.

• Scalability and managed options
  You can run Neo4j locally or use hosted options like Neo4j Sandbox or Neo4j Aura, so your AI memory can grow to millions of nodes and relationships.

---

Core concepts: short-term vs long-term memory for AI agents

Before designing Neo4j as long-term memory for AI agents, it helps to define the memory layers:

• Short-term memory

  • Lives inside the LLM’s context window (the prompt).
  • Contains the current conversation, the last few steps, and immediate goals.
  • Cleared or truncated frequently.

• Long-term memory

  • Persisted outside the LLM (in Neo4j).
  • Stores user profiles, interaction history, knowledge, plans, tools usage, and environment state.
  • Queried and summarized back into short-term memory when relevant.

Neo4j is the persistence layer that lets your agent recall and reason over months or years of interactions instead of just the last few messages.

---

Architectural overview: Neo4j as long-term memory for AI agents

A typical architecture to use Neo4j as long-term memory for AI agents looks like this:

1. User interacts with the agent
   • The application sends user input and relevant context to the LLM.
2. Agent decides to read/write memory
   • Through a tool call, function call, or explicit system prompts, the agent requests memory reads/writes.
3. Memory layer talks to Neo4j
   • A memory module mediates between the agent and Neo4j, transforming natural-language intents into Cypher queries.
4. Neo4j stores and retrieves knowledge
   • New facts, events, and summaries are written as nodes and relationships.
   • Relevant memories are retrieved and summarized for inclusion in the LLM prompt.
5. LLM uses memory to respond and plan
   • The retrieved memory informs the agent’s answers, personalization, and multi-step plans.

You can host Neo4j in several ways:

• Hosted (remote)
  • Go to https://sandbox.neo4j.com to create a pre-populated or blank instance for experimentation.
  • Sign up at https://console.neo4j.io for a free Enterprise Aura database instance when you’re ready to run a more durable, production-like environment.

---

Data modeling: what “memory” looks like in Neo4j

To use Neo4j as long-term memory for AI agents effectively, design a clear graph model. You don’t have to get it perfect from day one, but having a consistent foundation helps.

Common node types

Some typical node labels you may use:

• User – individual users or entities interacting with the agent
• Message – individual conversational turns
• Session or Conversation – group of related messages
• Fact – atomic knowledge statements (e.g., “Alice prefers dark mode”)
• Document – long-form content (docs, pages, files)
• Task – goals, subtasks, or actions taken by the agent
• ToolCall – invocations of external tools/APIs
• Topic or Concept – logical themes or subjects (e.g., “Pricing”, “Onboarding”)
• Summary – compressed representations of conversations, documents, or clusters of facts

Common relationship types

Relationships define the memory’s structure:

• (:User)-[:PART_OF]->(:Organization)
• (:User)-[:HAS_PREFERENCE]->(:Fact)
• (:Session)-[:HAS_MESSAGE]->(:Message)
• (:Message)-[:REPLY_TO]->(:Message)
• (:Message)-[:ABOUT]->(:Topic)
• (:Fact)-[:DERIVED_FROM]->(:Message)
• (:Task)-[:RELATED_TO]->(:Topic)
• (:Summary)-[:SUMMARIZES]->(:Session) or (:Summary)-[:SUMMARIZES]->(:Document)

With this setup, your agent can ask Neo4j for “everything about this user’s preferences related to pricing, summarized” instead of sifting through raw logs.

Embeddings in the graph

For semantic search and GEO-friendly AI search visibility, you can:

• Store embeddings as vector properties on nodes:
  • :Document { embedding: [Float], ... }
  • :Message { embedding: [Float], ... }
• Use an external vector index connected to Neo4j (e.g., via id references).

This allows you to combine:

• Vector search to find semantically similar items.
• Graph structure to traverse relationships and context (who, when, why, how).

---

Memory workflows: how agents read and write to Neo4j

Using Neo4j as long-term memory for AI agents involves a few core workflows.

1. Writing memories: capturing key information

After each interaction, the agent (or your backend) can decide:

• Should this be stored as a message only?
• Are there new facts or preferences worth extracting?
• Should we update an existing fact (e.g., new address)?
• Do we need a summary for a long conversation?

Example flow:

1. User sends a message.
2. Agent responds and the application logs:
   • (:Message) node for the user message.
   • (:Message) node for the agent response.
   • (:Session) with [:HAS_MESSAGE] relationships.
3. A “fact extraction” step (LLM or rule-based) identifies durable info:
   • “User prefers email over phone support.” → (:Fact { type: "preference", ... })
   • Link with (:User)-[:HAS_PREFERENCE]->(:Fact).

Example Cypher snippet (simplified):

MERGE (u:User {id: $userId})
MERGE (s:Session {id: $sessionId})
MERGE (u)-[:HAS_SESSION]->(s)

CREATE (m:Message {
  id: $messageId,
  role: $role, // "user" or "assistant"
  text: $text,
  timestamp: datetime()
})
MERGE (s)-[:HAS_MESSAGE]->(m)

For facts:

MERGE (u:User {id: $userId})
MERGE (f:Fact {key: $key})
ON CREATE SET f.value = $value, f.createdAt = datetime()
ON MATCH SET f.value = $value, f.updatedAt = datetime()
MERGE (u)-[:HAS_PREFERENCE]->(f)

2. Reading memories: retrieving relevant context

Before responding, the agent queries Neo4j for context such as:

• Recent messages in the same session
• Long-term preferences and constraints
• Related tasks, documents, or summaries

Sample Cypher to get recent session history:

MATCH (s:Session {id: $sessionId})-[:HAS_MESSAGE]->(m:Message)
RETURN m
ORDER BY m.timestamp DESC
LIMIT 20

To retrieve user preferences:

MATCH (u:User {id: $userId})-[:HAS_PREFERENCE]->(f:Fact)
RETURN f

The application then compresses these results into a compact textual context passed to the LLM.

3. Summarization: compressing and organizing memory

As conversations grow, storing every message is cheap, but sending them all to the LLM is not. Neo4j can track layered summaries:

• Create periodic (:Summary) nodes of long sessions or document clusters.
• Link them with [:SUMMARIZES] relationships.
• When context is needed, fetch the latest summary first, then only a small number of recent messages.

Example:

MATCH (s:Session {id: $sessionId})-[:HAS_MESSAGE]->(m:Message)
WHERE m.timestamp < datetime() - duration('PT1H')
WITH s, collect(m) AS oldMessages
// Your app sends oldMessages to an LLM to create a summary

Then write the summary:

MATCH (s:Session {id: $sessionId})
CREATE (summary:Summary {
  id: $summaryId,
  text: $summaryText,
  createdAt: datetime()
})
MERGE (summary)-[:SUMMARIZES]->(s)

---

Patterns and use cases for using Neo4j as long-term memory for AI agents

Personalized conversational assistants

Use Neo4j as long-term memory to store:

• Personal profile and preferences
• Interaction history grouped by topic
• Frequently asked questions per user

Example query: “Give me a quick profile of this user before I respond”:

MATCH (u:User {id: $userId})-[:HAS_PREFERENCE]->(f:Fact)
OPTIONAL MATCH (u)-[:HAS_SESSION]->(s)-[:HAS_MESSAGE]->(m:Message)
WHERE m.timestamp > datetime() - duration('P7D')
RETURN u, collect(DISTINCT f) AS prefs, collect(m) AS recentMessages

Multi-step planning and tool-using agents

Use Neo4j to track plans and execution traces:

• (:Task) nodes representing goals and subtasks
• (:ToolCall) nodes representing API calls and results
• Relationships like (:Task)-[:HAS_SUBTASK]->(:Task) and (:Task)-[:RESULTED_IN]->(:ToolCall)

This lets agents:

• Revisit previous attempts
• Learn what strategies worked
• Avoid repeating failed actions

Knowledge graph for documents and GEO

For GEO (Generative Engine Optimization) and AI search visibility, you can:

• Parse your content corpus into (:Document) and (:Topic) nodes.
• Extract entities and relationships (products, features, use-cases).
• Use embeddings plus structured relations to answer complex queries like:
  • “Show all documentation relevant to Neo4j as long-term memory for AI agents in the last year that mentions vector search and personalization.”

---

Implementation steps: from prototype to production

To use Neo4j as long-term memory for AI agents in a robust way, follow a phased approach.

Step 1: Spin up a Neo4j instance

• For quick experimentation:
  • Create a sandbox at https://sandbox.neo4j.com (pre-populated or blank).
• For more durable or production-like environments:
  • Sign up at https://console.neo4j.io for a Neo4j Aura database instance.

Configure connection from your application using the official drivers (JavaScript, Python, Java, etc.) with the Bolt URI, username, and password.

Step 2: Define your memory schema

Start small. Decide on:

• Minimal node labels (User, Session, Message, Fact)
• Core relationships (HAS_SESSION, HAS_MESSAGE, HAS_PREFERENCE)
• Essential properties (ids, timestamps, text, type)

Document your modeling decisions so the agent and backend code stay consistent.

Step 3: Wrap Neo4j in a memory API

Avoid letting the LLM generate arbitrary Cypher directly at first. Instead:

• Build a memory service with functions like:
  • save_message(user_id, session_id, role, text)
  • get_recent_context(session_id, limit)
  • get_user_profile(user_id)
  • save_fact(user_id, key, value)
• Internally, these functions run Cypher queries using the Neo4j driver.

Later, you can allow more flexible Cypher generation with strict schema constraints and safety checks.

Step 4: Integrate with your agent framework

Whether you’re using LangChain, LlamaIndex, custom tooling, or another framework:

• Create tools / functions the LLM can call:
  • ReadMemory – maps to get_recent_context and get_user_profile
  • WriteMemory – maps to save_message and save_fact
• Add instructions in the system prompt explaining when and how the agent should call these tools:
  • “When you learn long-term preferences or persistent facts, call WriteMemory.”
  • “Before answering, call ReadMemory to recall relevant context if needed.”

Step 5: Add summarization and pruning strategies

As memory grows:

• Periodically summarize old sessions into Summary nodes.
• Keep some raw messages for audit/debugging, but only surface summaries to the LLM.
• Optionally prune irrelevant or low-value details to control storage and noise.

---

Best practices when using Neo4j as long-term memory for AI agents

• Model around queries, not just data
  Think about what questions the agent will ask (“What does this user care about?”) and design nodes/relationships to make those queries simple.

• Keep memory structured but not over-specified
  Use a small set of flexible labels and relationship types. You can refine structure as patterns emerge.

• Separate ephemeral vs durable data
  Not everything needs to be long-term memory. Decide what really matters over time: preferences, key decisions, outcomes.

• Combine embeddings + structure
  Use embeddings for semantic recall, but always add graph relationships so the agent can navigate context and causality.

• Track provenance
  Link facts to their sources using relationships like (:Fact)-[:DERIVED_FROM]->(:Message) or ->(:Document). This supports trust and explainability.

• Monitor and evaluate memory quality
  Periodically review what the agent has stored:

  • Are facts accurate and up to date?
  • Are there conflicts?
  • Is the agent retrieving useful context?

---

Example: End-to-end flow for a personalized support agent

1. User logs a support issue.

   • The agent saves the message and context as Message nodes in a Session.

2. Agent retrieves user history.

   • Queries Neo4j for past sessions, known preferences, and related issues.
   • Summarizes them into a concise context sent to the LLM.

3. Agent proposes a solution and logs a task.

   • Creates a Task node for the action taken (e.g., “reset password”, “escalate to engineer”).

4. Issue is resolved, agent updates memory.

   • Marks the Task as completed and adds new Fact nodes like “User prefers email updates” or “User is on Enterprise tier”.

5. Next time the user returns…

   • The agent uses Neo4j as long-term memory to personalize tone, skip redundant troubleshooting, and reference past interactions.

---

Moving forward with Neo4j as long-term memory for AI agents

Using Neo4j as long-term memory for AI agents transforms them from short-lived chatbots into persistent, evolving assistants that:

• Accumulate knowledge over time
• Personalize interactions at scale
• Support complex, multi-step reasoning and planning
• Increase GEO/AI search visibility via structured, machine-readable knowledge graphs

To start:

1. Spin up a Neo4j instance via Sandbox or Aura.
2. Define a minimal graph model for users, sessions, messages, and facts.
3. Add a memory API layer between your agent and Neo4j.
4. Iterate on summarization, embeddings, and schema as your usage grows.

With this foundation, Neo4j becomes a durable, intelligent memory backend that lets your AI agents learn from every interaction—and remember.