Conversation Memory for AI Apps: State, Summarization, and Privacy

Memory Is Application Data, Not Model State

Most model APIs are stateless. Each request contains the instructions, messages, tool results, and retrieved information the model can use for that turn. The model may sound as if it remembers, but your application assembled that apparent memory and sent it again.

That distinction matters because three operations that feel similar in the interface have different engineering contracts:

• Conversation history preserves the current exchange. It keeps references such as "that second option" meaningful across nearby turns.
• Summaries compress older state. They trade detail for a smaller, more stable representation when a thread grows beyond its useful prompt budget.
• Durable memory carries selected facts across sessions. Preferences, project constraints, and user-approved details may outlive one conversation.

Retrieval is related but different. RAG searches an external corpus for evidence relevant to a question. Conversation memory reconstructs the state of an interaction and, when explicitly designed to do so, selected facts about a user or task. Both may use search, but they have different owners, retention rules, and failure modes.

Use Four Layers Instead of One Giant Transcript

A useful memory architecture separates information by lifetime and authority. This keeps temporary reasoning from silently becoming a permanent user fact.

1. Turn state: the current user message, current tool results, and temporary data needed to produce one response. Discard it when the request completes unless another policy requires retention.
2. Working context: the recent messages and active task state needed for conversational continuity. Keep it bounded by a token or message budget.
3. Thread summary: a compact record of completed decisions, unresolved questions, named entities, and current goals. It represents a specific conversation, not a global truth about the user.
4. Durable memory: a small set of explicit facts intended to survive across threads, such as an approved communication preference or a project constraint.

Keep canonical business state outside all four layers. An order status belongs in the order system. A user's permissions belong in the identity and authorization layer. A project deadline belongs in the project database. Memory can retain a reference or a past observation, but the application should re-read authoritative state before answering or acting.

system instructions
        +
authenticated user and tenant scope
        +
durable memories relevant to this task
        +
thread summary
        +
recent messages
        +
current tool results
        =
one model request

Store Structured Records With Provenance

A bare string such as "prefers Python" is easy to insert and hard to govern. You need to know whether the user said it, the model inferred it, an administrator configured it, or it was copied from an old summary. Those sources do not deserve equal trust.

A durable memory record should normally include:

• Owner and scope: user, tenant, project, assistant, and optionally thread.
• Typed value: a constrained category and validated payload rather than arbitrary prose.
• Source: the event, message, import, or user action that created it.
• Confidence and confirmation state: distinguish explicit user statements from model inferences.
• Lifecycle: creation, update, expiry, revocation, and deletion timestamps.
• Policy metadata: purpose, sensitivity, and whether the record may be used for personalization, automation, or only the current project.

type MemoryRecord = {
  id: string;
  tenantId: string;
  userId: string;
  scope: { kind: "user" | "project"; id: string };
  category: "preference" | "constraint" | "decision";
  value: unknown;               // validated by category
  sourceEventId: string;
  sourceKind: "user_explicit" | "admin" | "model_inferred";
  status: "proposed" | "confirmed" | "revoked";
  validFrom: string;
  expiresAt: string | null;
  supersedesId: string | null;
  purpose: string;
};

Treat model-extracted memories as proposals. If a user says, "The dashboard is too dense," the model might infer a permanent preference for minimal interfaces. That is a leap. Keep the original observation in the thread, or ask the user before promoting it to durable memory.

Updates should preserve lineage. A new preference can supersede an old record without erasing the evidence that explained earlier behavior. Retrieval should select the latest valid, non-revoked record and avoid sending contradictory versions to the model unless resolving the conflict is the task.

Compact Long Threads Without Inventing History

Sending the full transcript forever is simple, expensive, and increasingly noisy. Old jokes, failed attempts, obsolete tool results, and repeated instructions compete with the current task. A compaction policy decides what remains verbatim, what becomes structured state, and what can be dropped.

Start with a fixed budget rather than "summarize when the API rejects the request." Reserve room for system instructions, current input, expected output, and tool results. Allocate the remaining budget among durable memory, summary, and recent messages. When working context crosses its threshold:

1. Keep the most recent turns verbatim.
2. Extract deterministic state where possible: selected file, approved plan, unresolved error, chosen option, and completed tool actions.
3. Summarize older conversational content into a versioned thread summary.
4. Validate that the summary preserves required facts and does not promote guesses into decisions.
5. Store a pointer to the source message range so a user or support process can inspect what was compressed.

{
  "goal": "Prepare the July usage report",
  "confirmed_decisions": [
    { "text": "Use UTC for all daily totals", "sourceMessageId": "m_184" }
  ],
  "open_questions": [
    { "text": "Include test accounts?", "sourceMessageId": "m_191" }
  ],
  "completed_actions": [
    { "text": "Loaded billing export v3", "toolRunId": "tr_77" }
  ],
  "active_constraints": ["Do not email or publish the report"],
  "summaryThroughMessageId": "m_194"
}

Summaries are generated data, so they can omit, distort, or merge details. Use schemas, cite source IDs, and rebuild summaries when the format or summarization prompt changes. For high-stakes state, extract and verify individual facts instead of relying on one prose paragraph.

Make Durable Memory Visible and Reversible

Long-term memory changes the product relationship. Users may reasonably expect a thread to retain context while being surprised that an unrelated conversation reuses a personal detail months later. The interface should make persistence a visible feature, not an invisible side effect.

Good controls include:

• Explain what is remembered and why. Separate thread history from cross-session personalization in plain language.
• Ask before saving sensitive or surprising facts. A small "Remember this?" action creates clearer intent than silent extraction.
• Show a memory list. Let users inspect, correct, revoke, and delete individual records without deleting their whole account.
• Support temporary conversations. Give users a mode that neither reads nor creates durable memory.
• Confirm consequential reuse. A remembered preference can prefill a choice; it should not silently authorize purchases, messages, access changes, or other side effects.

Memory also needs a conflict policy. Prefer explicit, recent user confirmation over older inference. When two confirmed records disagree and the correct choice matters, ask. Do not hide ambiguity by concatenating both facts and hoping the model resolves them.

Enforce Privacy, Isolation, Retention, and Deletion

A memory store concentrates conversational data, inferred preferences, and project details. Treat it as sensitive application storage. Every write and read must be scoped from authenticated server state, not from a user ID or tenant ID generated by the model.

• Filter before retrieval. Apply tenant, user, project, status, purpose, and expiry constraints in the database query before ranking candidate memories.
• Minimize values. Store the smallest fact needed for the declared purpose. Avoid copying full messages into each extracted memory.
• Separate content from search metadata. Embeddings and summaries can still reveal sensitive meaning. Protect them with the same access and deletion rules as the source.
• Define retention by layer. Turn data, transcripts, summaries, durable facts, traces, backups, and derived indexes may need different schedules.
• Propagate deletion. Remove or tombstone the canonical record, search index, embedding, cache, summary reference, and downstream copy according to a testable process.
• Keep an audit trail without keeping the secret. Record that a memory was deleted, by whom, and when without preserving its sensitive value in logs.

const memories = await db.memory.findMany({
  where: {
    tenantId: session.tenantId,
    userId: session.userId,
    status: "confirmed",
    purpose: { in: allowedPurposesFor(request.route) },
    OR: [
      { expiresAt: null },
      { expiresAt: { gt: new Date() } }
    ]
  },
  take: 12
});

If deletion must reach a vector index or asynchronous cache, make it an observable job with retries, idempotency, and a completion state. Test the negative result: after deletion completes, the record must not appear in direct lookup, semantic retrieval, assembled prompts, support tools, or exports.

The data sanitization guide covers what should not enter a model context. The guardrails guide covers authorization and untrusted model output. Apply both principles to memory: stored text is untrusted data, and recalling it never grants authority.

Test Recall, Forgetting, Conflict, and Leakage

A demo proves that memory can recall one fact. Production tests must also prove that it forgets, updates, isolates, and declines to invent facts. Build a small evaluation set with explicit expected context for each request.

• Positive recall: the right confirmed memory appears when its scope and purpose match.
• Negative recall: irrelevant, expired, revoked, or unconfirmed memories do not enter the prompt.
• Tenant isolation: another user, project, or tenant cannot retrieve the record by ID, text, embedding similarity, export, or support interface.
• Correction: a superseding fact wins and the obsolete version is not presented as current.
• Compaction fidelity: confirmed decisions and open questions survive summary regeneration; abandoned guesses do not.
• Deletion: all serving paths stop returning the record after the documented deletion process completes.
• Injection resistance: instructions stored inside a past message or memory value are treated as data and cannot override current policy or call a tool.

it("does not recall another tenant's matching preference", async () => {
  await saveConfirmedMemory(tenantA, userA, "report_format", "CSV");

  const context = await buildContext({
    session: sessionFor(tenantB, userB),
    message: "Use my usual report format"
  });

  expect(context.memories).toEqual([]);
  expect(context.serialized).not.toContain("CSV");
});

Observe the pipeline, not private conversation text. Useful fields include the memory policy version, candidate count, selected memory IDs, scope, age, confirmation state, summary version, token allocation, compaction event, deletion job state, and outcome. Track correction rate, irrelevant-recall rate, missed-recall rate, summary-regression rate, and cross-scope access denials.

Alert when retrieval volume or prompt share changes suddenly, a deployment begins selecting expired records, deletion jobs stall, or a summarizer version increases lost-decision failures. The evals guide explains how to version these cases, while AI Observability covers traces and production signals.

Copy-Paste Prompt: Design or Audit an AI Memory Layer

Use this prompt before implementing persistent memory or when reviewing an existing chat system:

Review the AI feature in [files or service] and design its memory layer.

First map:
- turn state, recent conversation history, thread summaries, and durable memory
- canonical business data that must be re-read rather than remembered
- every store, cache, embedding index, log, backup, and export containing memory
- ownership and scope: tenant, user, project, thread, assistant, and purpose

For each memory type, define:
1. why it exists and who benefits
2. exact write trigger and whether user confirmation is required
3. typed schema, source reference, confidence, status, and expiry
4. authorization rules for create, read, update, export, and delete
5. conflict, correction, supersession, and revocation behavior
6. retention and deletion propagation across derived stores
7. prompt budget and selection rules
8. metrics and tests proving recall, non-recall, isolation, and forgetting

Design a compaction policy that:
- keeps recent turns verbatim within a fixed budget
- extracts confirmed decisions and unresolved questions with source IDs
- versions summaries and can rebuild them from source events
- never promotes an inference into a confirmed user fact

Then produce:
- a data-flow and trust-boundary diagram
- storage schemas and retrieval pseudocode
- the smallest implementation plan in reviewable stages
- migration and rollback steps for existing transcripts
- an eval set covering recall, conflict, correction, expiry, deletion,
  tenant isolation, and prompt injection inside stored content

Do not use memory as authorization or as the source of truth for mutable
business data. Do not store full transcripts by default when a smaller,
purpose-bound record is enough. Do not implement changes until I approve
the design.

Related Guides