MemTree API

Context Memory

With Context Memory, LLMs are no longer limited by their context window. Coding sessions, agents, and conversations can now continue indefinitely as context is compressed via a combination of hierarchical GraphRAG and summarization.

Coding

Models will now get better, rather than worse, as your coding sessions stretch into the millions of tokens.

Agents

Agents can now try many approaches, learning from their past mistakes and remembering important lessons, no matter how long ago they happened.

Chat

Keep your storytelling, role-play, therapy, medical, or other long-running threads going without worrying about the model forgetting important details or missing out on overarching themes.

Motivation

Humans can recall arbitrarily long events at arbitrary levels of detail. For example, people can provide a two minute or one hour version of stories that took place over a few hours to eons. Cognitive scientists refer to this ability as scale-invariant temporal memory, a key function of episodic memory. Episodic memory and semantic memory together form the two main pillars of long-term memory in people.

Many current LLM memories, like ChatGPT's memory and Cursor Memories, are examples of semantic memory. But without episodic memory, LLMs are forced to ingest the entirety of their experience at full resolution for every token they produce. In practice, this leads to context windows ballooning to contain too much detail and models producing slop. Then when threads are restarted or compacted, models must be reinformed with key details from previous threads. Many approaches use RAG to recall details, but they often recall poorly sized chunks and/or lose out on the benefits of summarization for understanding how those chunks fit into the larger context of the thread.

How It Works

Context Memory can be thought of as a hierarchical episodic memory that retains source excerpts alongside lossy summaries of long message histories. In our implementation, Context Memory is structured as a B-tree where the top of the tree contains a high level summary of the current message history and the bottom of the tree contains verbatim excerpts from that history that are relevant to recent messages. In between are summaries that get more detailed as you go towards the leaves. Rather than simply return matching chunks of text without context, as is typical with RAG, we retrieve relevant details contextualized within a tree of summaries. It's a specialized form of GraphRAG where relationships between nodes are encoded in the tree structure. This tree structure allows us to efficiently expand and collapse nodes based on relevance at query-time.

Example memory response when asking about a divide by zero error

  • Feature 1 summary
    • Task 1 summary
    • Task 2 summary
      • Step 1 summary
      • Step 2 summary
        • Divide by zero error details
    • Task 3 summary

Imagine that you were working to resolve a divide by zero error in a codebase, and that the only source verbatim relevant to the error is:

Divide by zero error details
By unfolding the tree and returning the path of summaries above retreived excerpts, we can provide a concise overview of what led to this divide by zero error.

When we compress: the whole-request budget

Compression is a cost, not a goal: it trades verbatim conversation for a lossy summary. So every request is measured against a whole-request budget — the system prompt, tools, memory, and recent messages all count — and we only compress when the request exceeds it.

The budget is chosen per request, in this order of precedence:

  1. compression_target_tokens if you send it. -1 disables compression entirely: every call passes through uncompressed, still sanitized and still indexed.
  2. Otherwise a policy for the model you send: 800k tokens for the 1M-context Anthropic models that hold up across the window (Claude Fable 5 / 5.1, Opus 5.5, Opus 5, Opus 4.8); 550k for Sonnet 5.5 and Sonnet 5 and 400k for Sonnet 4.6, where long-context retrieval degrades before the window; half of the measured effective context for other benchmarked models (e.g. Opus 4.5); 50k tokens for everything else, including models we do not recognize.
  3. Otherwise, with no model, the 50k static default.

The budget is always capped at model_context_limit. It is a soft target: the most recent message is never truncated, so a very large final message can push a compressed request past it. The comparison is done in characters at a conservative ~2.7 characters per token, so a request can fit your model's window in tokens and still be over a window-sized target — use compression_target_tokens: -1, not a huge target, when you want a guarantee of no compression.

You may set compression_threshold_tokens separately to delay compression until that size, while keeping a smaller target. An optional client_input_tokens measurement above the effective threshold also triggers a compression attempt when the character estimate fits. Neither field overrides compression_target_tokens: -1.

Soft vs. hard. compression_target_tokens is a soft target: we compress toward it, but the most recent message is never truncated and can push the result past it. model_context_limit is the hard cap: it is the only thing that produces a 413, and the target is always clamped to it. Send the model's real window as model_context_limit and the size you want conversations compressed to as compression_target_tokens; sending a small target as the hard cap turns every over-target conversation into a 413 wait instead of a compression.

What stays verbatim

Compression replaces indexed history with memory and retains recent messages where they fit. Under hard-window pressure, older un-indexed messages may also be truncated; the most recent message is preserved, or the request fails with 413:

Reading a MemTree

Every conversation's memory is a MemTree: a hierarchy of summaries whose leaves point at the exact transcript lines they summarize. You can read the tree a request was compressed with, either whole or through small query endpoints that return only what you ask for.

Finding the tree

Each /context_memory response, compressed or passthrough, carries an X-Polychat-Memtree-Url header: the request's MemTree page, e.g. https://app.polychat.co/m/2g36dGG1een5. For partner keys it is present only when a memory tree is ready to view for that response: show “View memory” whenever it is present. It is absent until a conversation's first tree is built, without an end_user_id, and it can be missing in conversations containing something shaped like a secret (stored copies are redacted). The last path segment is the page id: a 12-character link id, with -s appended when the page shows the tree the request was served from. Treat it as opaque. The header is optional: index_only responses and errors (including 413s) have none, so don't require it. /usage/memtree/<request id> is the same page, and every endpoint below accepts the link id, the full request id, or older 12-hex-digit ids under either prefix. Opened in a browser, the URL shows the tree; the endpoints below are the machine forms of the same URL. Ignore X-Polychat-Usage-Url if you see it: it links our internal usage dashboard.

A request's own tree is built in the background after its response. Until it is ready, these URLs answer with the tree the request was compressed against, marked "served_prefix": true (its node ids change once the request's own tree lands, so do not cache them across that switch). When there is no tree to show yet, they answer 202, or 410 once there never will be one (see below).

Building (202) vs. unavailable (410)

Every MemTree URL (the page, its .json, and the search, nodes, lines and messages reads) answers one of two ways when it has no tree to show:

GET /usage/memtree/sessions/{session_id}.json (owner only; share links cannot list) returns a bounded list of MemTree pages of one Claude Code session, newest first, each with its links.html and links.json. Its separate newest_tree field links the completed tree with the largest recorded message count in that session, including trees named by unbilled index-only requests. Both are scoped to the owner and, for pooled partners, the established end_user_id.

Auth

Partners calling on behalf of their own users: your partner key is a server-side credential. To show a page to one of your users, mint a one-time view code. Your authenticated server must supply or overwrite end_user_id from its own user identity, never trust an arbitrary client-supplied value, and check request ownership before forwarding reads. Tail and related-tree lookups use the end user recorded on the request. Session lookups require your server to pass the established end_user_id query parameter; without it, pooled accounts receive no session or related-tree links. An end-user id is a partner assertion, not a separate credential.

The whole tree: GET /m/{id}.json

One JSON document with the full tree and, where allowed, the transcript it indexes. Good for tooling that walks the tree locally; for a long session it runs to several MB, so prefer the query endpoints below when you need a few nodes.

Query endpoints

The same operations the MemTree tools in Claude Code Infinite give its agents, run on our side. Each takes the same auth as the .json, returns JSON, and starts with request_id and messages_hash (plus served_prefix when applicable) so you can tell which tree answered. Bad parameters are a 400 with a detail message; an unknown node or block is a 404. These reads are not billed.

Lexical search over every node's summary and every leaf's transcript lines. q is split on whitespace into lowercase terms with surrounding punctuation trimmed; common words (the, is, what, …) are dropped, and a query with no terms left is a 400. A term matches as a case-insensitive substring. Hits rank by how many distinct terms a node covers, then by how many terms a single line holds, then by summary matches and total occurrences; ties go to the later node. limit defaults to 10 and is clamped to 1–50. A leaf hit carries up to 3 snippets (whitespace collapsed, a window of about 240 characters around the first match); summaries are cut at 600 characters.

For the owner, the same terms are also matched against the messages after the tree that no tree covers yet (the .json tail, often how a session ended). Matching messages come back as tail_hits (at most 5, most terms first, then newest): each has the message position i (read it whole with /messages?start=i&end=i), its kind k, state, at when recorded, matched_terms and up to 3 snippets with the line number within the message. tail gives the tail's start, end and not_indexed_from. Both are absent when nothing in the tail matches, and for shares. 

curl -H "Authorization: Bearer $API_KEY" \
  "https://api.polychat.co/m/3f2a9c1b7e40/search?q=deploy+target&limit=5"
{
  "request_id": "3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17",
  "messages_hash": "9b1c…",
  "query": "deploy target",
  "terms": ["deploy", "target"],
  "limit": 5,
  "hits": [
    {
      "id": 6,
      "leaf": true,
      "depth": 2,
      "summary": "Deploy target and cache invalidation",
      "range": {"block": 1, "start": 2, "end": 4},
      "matched_terms": ["deploy", "target"],
      "line_hits": 2,
      "score": 250.5,
      "snippets": [
        {"line": 2, "text": "assistant: the Deploy target is cloud-run"},
        {"line": 3, "text": "assistant: Cache invalidation on DEPLOY; deploy deploy"}
      ]
    }
  ]
}

range is null for a branch; depth is the distance from the root. Pass a hit's range to /lines for the exact text, or its id to /nodes for its context.

GET /m/{id}/nodes/{node_id}

One node: its full summary, its leaf range if it is a leaf, the path from the root to it, and its children. Start at node 0 to browse top-down.

{
  "request_id": "3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17",
  "messages_hash": "9b1c…",
  "id": 4,
  "leaf": false,
  "depth": 1,
  "summary": "Cache work",
  "range": null,
  "path": [{"id": 0, "summary": "Session about deploys and caching"}],
  "children": [
    {"id": 5, "leaf": true, "summary": "Cache key choice", "range": {"block": 1, "start": 1, "end": 1}},
    {"id": 6, "leaf": true, "summary": "Deploy target and cache invalidation", "range": {"block": 1, "start": 2, "end": 4}}
  ]
}

GET /m/{id}/lines?block=&start=&end=

The exact transcript lines start–end (1-based, inclusive) of block block, all three required. end past the block's last line is clipped; start past it is a 400. One call returns at most 400 lines and about 40,000 characters, always whole lines except a single first line longer than that, which is cut and marked … [line truncated: N chars]. When a cap stops the read early, next_start is where to continue; end in the response is the last line returned. 

{
  "request_id": "3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17",
  "messages_hash": "9b1c…",
  "block": 1,
  "start": 2,
  "end": 4,
  "block_line_count": 4,
  "lines": [
    "assistant: the Deploy target is cloud-run",
    "assistant: Cache invalidation on DEPLOY; deploy deploy",
    "tail line without terms"
  ],
  "next_start": null,
  "truncated": false
}

GET /m/{id}/messages?start=&end=&full=

Recorded messages after this tree (the .json tail), including messages already covered by a newer tree, by position: start–end are inclusive values of i, both optional. Without start you get the newest messages, the end of the session being what an agent usually wants. With start the range fills forward; without it, the newest messages that fit are kept. Results are returned in chronological order. One call returns at most 50 messages and about 40,000 characters of text; oversized messages are cut and marked. A tool call or tool result longer than 2,000 characters or 40 lines is shortened to its first and last lines, with shortened: true on the message, a marker … [N lines / M chars omitted; full=true returns it whole] where text was left out, and a shortened_note in the response; user and assistant text is never shortened. Add full=true to get every message whole (the .json tail takes the same parameter). Each message has a state of in_newer_tree or not_indexed. The response carries tail_start and tail_end (the whole tail's range), start and end (what was returned) and messages, after the usual request_id and messages_hash. A wholly out-of-range request or a negative position is a 416; a partially overlapping range is clipped, and a non-integer position is a 400. Only the owner and shares made with the verbatim source can read it: a summaries-only share, or a tree with no recorded tail, gets a 404. The tail comes from the latest accepted job, not necessarily the complete live conversation: its newest <10k tokens may be absent until the next accepted job or final index. The .json inline tail keeps at most 1,000 messages and about 1,000,000 characters of text, preferring the newest; omitted_before identifies omitted older messages, which can be read by position. The page uses the same bounded snapshot. not_indexed_from marks the boundary between messages in a newer tree and messages in no tree yet (null when all are covered). 

curl -H "Authorization: Bearer $API_KEY" \
  "https://api.polychat.co/m/3f2a9c1b7e40/messages?start=2050&end=2051"
{
  "request_id": "3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17",
  "messages_hash": "9b1c…",
  "note": "Not yet indexed: these messages came after this tree and are in no tree yet. …",
  "tail_start": 2050,
  "tail_end": 2051,
  "start": 2050,
  "end": 2051,
  "messages": [
    {"i": 2050, "k": "user", "x": "open the PR", "at": "2026-10-06T01:00:00Z"},
    {"i": 2051, "k": "assistant", "x": "PR #42 opened"}
  ]
}

Finding sessions

Two endpoints look across all of your MemTree sessions at once: list them, and search the transcript text of every tree. A session is a Claude Code session (the id Claude Code resumes with). Requests from other clients are grouped by conversation (the same first message), or stand alone when no tree was built for them. Both endpoints return JSON, newest first or best first, and page with a cursor.

Partners: contact us before enabling a partner credential flow for these endpoints. An authenticated NanoGPT or OpenRouter account must supply end_user_id, the same opaque per-user id recorded in x-client-meta on its /context_memory calls; without it the finder returns 403. The id must contain 1–128 printable ASCII characters. Rows without that id are excluded. Ordinary owner accounts cannot supply this parameter (400). Shared partner credentials are not enabled merely by supplying an id.

GET /v1/memtree/sessions?since=&until=&project=&q=&cursor=&limit=

Your sessions, most recently active first. q keeps sessions whose title, first message or project contains it (case-insensitive substring). limit defaults to 20 and is clamped to 1–100. Free. The signed cursor retains the first page's request-time cutoff so newly arriving requests do not move an unseen session ahead of the cursor. Start without a cursor to include newer activity; tree completion and other live state can still change. 

curl -H "Authorization: Bearer $API_KEY" \
  "https://api.polychat.co/v1/memtree/sessions?project=polychat&since=2026-09-01&limit=2"
{
  "sessions": [
    {
      "id": "6025e1f7-074b-4abb-a8e7-dbf07ef1e81f",
      "kind": "claude_code_session",
      "session_id": "6025e1f7-074b-4abb-a8e7-dbf07ef1e81f",
      "title": "Usage dashboard overhaul and MemTree token tracking",
      "snippet": "Restyle the usage page to look like the MemTree page…",
      "first_at": "2026-09-28T17:02:11.412000+00:00",
      "last_at": "2026-09-29T01:40:57.018000+00:00",
      "project": {"dir": "polychat", "repo": "acme/polychat", "branch": "main", "commit": "c6fe251"},
      "models": ["claude-opus-5-5"],
      "request_count": 212,
      "tokens": {"prompt": 18400311, "completion": 0, "conversation": 164220},
      "latest_tree": {
        "request_id": "3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17",
        "ref": "3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17-v1-own",
        "links": {
          "url": "https://api.polychat.co/usage/memtree/3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17-v1-own",
          "html": "/usage/memtree/3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17-v1-own",
          "json": "/usage/memtree/3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17-v1-own.json"
        }
      },
      "ending": {
        "at": "2026-09-29T01:40:51.200Z",
        "message_count": 1840,
        "last_user": "ship it and open the PR",
        "actions": ["Bash", "Bash", "Edit", "Bash"],
        "last_assistant": "PR #212 is open: restyled usage page, MemTree token column…"
      }
    }
  ],
  "next_cursor": "<signed cursor>",
  "limit": 2
}

Searches the transcript lines (the leaves) of every tree in your sessions. Each passage shows once, from the newest tree of its conversation (same session or conversation and indexing style) that stores it: an older leaf is dropped when a newer tree's leaf has its stored lines, either all of them or all but a short tail the older tree alone had (at most a third of its lines, or 20 lines; Claude Code side requests leave such tails). In text mode that tail must hold none of the query's words, so nothing only the older copy shows is hidden. Divergent histories, leaves holding a redacted secret and other conversations remain separate. Deduplication runs after every ownership and provenance filter. q is required. limit defaults to 10 and is clamped to 1–50. Search pages read live data: newly indexed leaves may appear after the cursor, including new copies of previously seen transcript text. Unchanged embedding rows do not repeat across vector pages.

tree (a request id or a pinned reference such as <request id>-v1-own) limits the search to that one tree, in either mode. A leaf’s line range must be a node of the requested tree. Explicit provenance, when recorded, authorizes only its listed source namespaces and overrides legacy lineage checks; unknown explicit provenance denies the leaf. A legacy leaf without explicit provenance requires its source to belong to the requested tree’s own prefix-hash lineage. These are the same provenance rules used by unfold. Other leaves are removed before ranking and pagination. It searches your own trees only: a tree you do not own, including one shared with you by link or ?share= token, and any tree= search without a valid key, gets the same 404 as a page that does not exist (and counts toward the per-IP failed-lookup limit, 429, like the pages). To search a shared page, use its own /m/<id>/search, which applies the share's rules. It answers 409 while the tree is still building, and ignores project, since and until. Cursors are bound to it. Partner keys must also supply end_user_id matching the row’s client_meta.end_user_id, or receive 403. Scoped responses confirm the requested reference in tree. 

curl -H "Authorization: Bearer $API_KEY" \
  "https://api.polychat.co/v1/memtree/search?q=%22cache+invalidation%22+deploy&project=polychat&limit=1"
{
  "query": "\"cache invalidation\" deploy",
  "mode": "text",
  "limit": 1,
  "hits": [
    {
      "id": "6025e1f7-074b-4abb-a8e7-dbf07ef1e81f",
      "kind": "claude_code_session",
      "session_id": "6025e1f7-074b-4abb-a8e7-dbf07ef1e81f",
      "tree": {
        "request_id": "3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17",
        "ref": "3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17-v1-own",
        "created_at": "2026-09-29T01:38:02.211000+00:00",
        "links": {
          "url": "https://api.polychat.co/usage/memtree/3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17-v1-own",
          "html": "/usage/memtree/3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17-v1-own",
          "json": "/usage/memtree/3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17-v1-own.json",
          "lines": "/usage/memtree/3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17-v1-own/lines?block=1&start=2&end=4"
        }
      },
      "leaf": "leaf_node_1:2:4.txt",
      "range": {"block": 1, "start": 2, "end": 4},
      "snippet": "the Deploy target is cloud-run … Cache invalidation on DEPLOY",
      "score": 0.0913,
      "node": 7,
      "address": "3f2a9c1b-7e40-4d2a-9a51-0c8e2b6f4d17-v1-own#7",
      "path": [
        {"id": 0, "summary": "Deploys and caching for the API"},
        {"id": 4, "summary": "Cache work"}
      ]
    }
  ],
  "next_cursor": "<signed cursor>",
  "matches_capped": false
}

range is the leaf's [block, start, end] in that tree, as in the .json; tree.links.lines reads those exact lines. address names the matched node as <tree ref>#<node id>, and path lists its ancestors from the root with short summaries; /usage/memtree/<tree ref>/nodes/<node id> opens the node, its children and its path (whose ids lead to parents and siblings). A hit whose node cannot be located has no address and keeps its range. Per-tree search (/usage/memtree/<id>/search) hits carry the same address and path. Use tree.ref as the ccc tree argument to preserve that exact tree and its line ranges. A vector search answers with groups in place of hits: 

{
  "query": "how did we pick the deploy target",
  "mode": "vector",
  "limit": 10,
  "groups": [
    {
      "embedding_model": "voyage-3.5",
      "has_more": true,
      "hits": [
        {"id": "6025e1f7-…", "kind": "claude_code_session", "session_id": "6025e1f7-…",
         "tree": {"request_id": "3f2a9c1b-…", "links": {"lines": "/m/3f2a9c1b7e40/lines?block=1&start=2&end=4", "…": "…"}},
         "leaf": "leaf_node_1:2:4.txt", "range": {"block": 1, "start": 2, "end": 4},
         "snippet": "assistant: the Deploy target is cloud-run…", "score": 0.8826,
         "embedding_model": "voyage-3.5"}
      ]
    }
  ],
  "next_cursor": "<signed cursor>",
  "charged": true
}

score is cosine similarity within its model's group. charged is true for a first-page embedding receipt, including cache hits, and false for a continuation or a search with no embedding models.

Search SQL statements have an 8-second timeout and return 422 when that limit is reached, asking for a more specific query (typically a quoted phrase of common words; add a rarer word or narrow by time, project or tree). A vector operation still running after a 30-second wait returns 502; paid embedding work remains tracked until it finishes, including after a disconnected client. First-page usage is recorded even if ranking fails; continuation errors remain uncharged. A retry without a cursor starts another billed search, although it may reuse the provider result. Missing server cursor-signing configuration returns 503 before paid work.

⚡️ Speed

Retrieval uses embeddings and vector similarity search to find relevant excerpts and summaries, then unfolds their paths in the tree. Latency depends on request size, index state, provider latency, and database load; no fixed response-time guarantee is implied.

API

We build on the OpenAI compatible API format where the message history is passed in its entirety for every request. The main input is a list of OpenAI compatible messages. The output messages are a compressed version of the input messages.

The majority of older input messages get condensed into a single user message containing the context memory. The recent messages are then returned after the system message and memory messages.

Semantic Addressing

There is no memory ID as the messages themselves serve to identify the memory. This allows branching, reverting, and incrementally updating memories while critically avoiding the indexing of messages that are no longer in the message history. E.g. if a coding session goes down a wrong path and the messages are reverted to an earlier state or if a chat edits a message, branches into a multiple conversations, or clones a previous message history. This means that if previous messages are modified (except for system messages: see below), a new memory will be built at the point of the modification.

Indexing

Indexing happens in the background; duration varies with input size, structure, retries, and provider latency. We do not block API requests to wait for indexing unless we are unable to fit recent unindexed messages inside the whole-request budget. We may attempt to truncate unindexed messages using an exponential decay algorithm that prioritizes recent messages to avoid blocking. However, we will never truncate the most recent message as it's critical for the model to see it in its entirety. If the conversation does not fit in the model_context_limit and no memory exists yet, we start indexing and block on it (up to 5 minutes), returning the compressed conversation as soon as the index completes. If indexing fails, or is still running after 5 minutes, we return an HTTP 413 whose detail carries an estimated indexing time; retry after that.

Agent loops that make many tool calls per user turn can keep the index warm cheaply with index_only: it triggers indexing for the messages sent and returns immediately with no compression and no output messages. It bills only the un-indexed tail it hands to an indexing job (see pricing).

A new indexing job starts only once more than 10,000 un-indexed tokens have built up, so the last stretch of a conversation would otherwise stay out of every tree. When a conversation ends, send one last index_only call with final_index: true to index that tail even below the minimum.

Chunking

Inputs are chunked semantically at index time. This ensures chunks are semantically cohesive and that boundaries are clean.

Models

Gemini 3.8 Flash is used for indexing and Voyage 3.5 is used to generate embeddings.

Summaries: language and secrets

Usage

API Keys

Click here to generate and copy your PolyChat API key. Your API key can be further managed from within the polychat.co app under Settings > Account > API keys.

Claude Code: Claude Code Infinite

Claude Code Infinite (ccc) runs Claude Code with Context Memory: a small local proxy sends each conversation through /v1/context_memory, so sessions run past the context window without compacting, and its MemTree tools let the agent search and read earlier parts of the session. Your Anthropic login stays on your machine. Install it (Node 18 or newer) and run ccc: 

npm install -g https://github.com/crizCraig/claude-code-infinite/tarball/main
ccc

Setup Context Memory with Kilo Code, Roo Code, or Cline

  1. Set the Base URL to https://polychat.co/api
  2. Get your API key from within the polychat.co app under Settings > Account > API keys
  3. Add the Custom Header:
    • name: x-polychat-memory
    • value: on

/context_memory

This API takes in OpenAI compatible messages and outputs compressed OpenAI messages. It allows you to easily add memory to any model by replacing the original messages with the compressed messages this API returns.

Send a POST request with messages, your api key, and the context limit to compress messages. model and tools are optional but recommended: they size the budget and count the tools against it.

curl -X POST https://api.polychat.co/v1/context_memory \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "user", "content": "Hello"},
      {"role": "assistant", "content": "Hi there!"}
    ],
    "model_context_limit": 200000,
    "model": "claude-sonnet-4-5-20250929",
    "tools": [{"name": "search", "input_schema": {"type": "object"}}]
  }'

https://polychat.co/api/context_memory also accepts the same request, but it does not pass back the X-Polychat-Memtree-Url header or forward x-client headers; prefer https://api.polychat.co/v1/context_memory.

This outputs compressed messages, a verdict, usage, and the model's budget:

{
  "messages": [
    {
      "role": "user",
      "content": "[Memory message will be here*]"  // Present once an index exists and the request is over budget
    },
    {
      "role": "assistant",
      "content": "…recent messages, verbatim…"
    }
  ],
  "compressed": true,   // false => under budget (or not indexed yet); forward your original messages
  "usage": {
    "prompt_tokens": 260000,
    "prompt_tokens_details": {"cached_tokens": 248000},
    "completion_tokens": 47000,
    "total_tokens": 307000,
    "raw_prompt_tokens": 266000,
    "indexed_tokens": 248000
  },
  "model_budget_tokens": 50000,
  "index_pending": false,
  "memory_status": "ok"
}
The first message will only contain memory after a threshold of 10k tokens are indexed, and only when the request is over the budget.

Params

Response fields

Headers

Response headers:

GET /v1/context_memory/status

At https://api.polychat.co/v1/context_memory/status, with the same auth as the POST. Returns {"email": …, "paid": true|false, "payment_message": …} so a client can find out at startup whether compression calls would be rejected with 402, without sending a conversation.

Errors

/chat_completions

This is a fully OpenAI compatible API which uses models available on polychat.co to generate responses. It uses Context Memory under the hood to compress the messages for you. Send it uncompressed messages, and we will compress them and send them to the model.

Send a POST request with messages, your api key, and the x-polychat-memory header set to "on"

curl -X POST https://polychat.co/api/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-polychat-memory: on" \
  -d '{
    "model": "GPT-4o-mini",
    "messages": [
      {
        "role": "user",
        "content": "hi"
      }
    ]
  }'

System Prompt

We don't summarize the system prompt: it passes through and does not get reflected in the memory index. The one exception is size: a system prompt longer than 60,000 characters may be trimmed in the returned messages to its first 3,000 and last 57,000 characters, with a truncation marker between them. We do this as system prompts are typically important for every message turn (agent instructions, tools available, provider info, etc...). And since they can contain today's date/time, they can invalidate the memory.

Partners: your users' memory

Partners (currently NanoGPT) send many users' conversations under one partner key. The endpoints below let you show one of your users the memory behind a response, and delete it. They take your partner key (Authorization: Bearer <partner key>), are called from your backend only, and are enabled per partner: until yours is, they answer 404 like an unknown route. Base URL: https://api.polychat.co (app.polychat.co, the host in X-Polychat-Memtree-Url, serves the same endpoints).

Send the end user on every call

On each /v1/context_memory call, name the end user in x-client-meta:

curl -X POST https://api.polychat.co/v1/context_memory \
  -H "Authorization: Bearer $PARTNER_KEY" \
  -H 'x-client-meta: {"end_user_id": "u_6f1c2d3e"}' \
  -H "Content-Type: application/json" \
  -d '{"messages": [...], "model_context_limit": 200000, "expiration_days": 30}'

Retention

expiration_days is stored with each request, and each tree keeps the retention of the request that created it. A positive value is the retention in days, counted from the tree's last use by a Context Memory call (viewing does not extend it). 0 or omitted means the memory never expires; there is no default, so if your users have one, send it on every request. Once expired, memory can no longer be opened (view-code answers 410, open pages answer “This memory has expired”), listed or reused, and it is then deleted.

POST /partner/memtree/{id}/view-code

When your user clicks “view memory”, check on your side that the message is theirs, then mint a one-time code. {id} is the page id from X-Polychat-Memtree-Url, exactly as sent (-s included), or the full request id. end_user_id is required and must be the id the request was recorded with. 

curl -X POST https://api.polychat.co/partner/memtree/2g36dGG1een5-s/view-code \
  -H "Authorization: Bearer $PARTNER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"end_user_id": "u_6f1c2d3e"}'
{
  "request_id": "0f1c2d3e-4a5b-4c7d-8e9f-0a1b2c3d4e5f",
  "code": "Qk3…",
  "expires_in": 60,
  "expires_at": "2026-10-14T18:46:06+00:00",
  "url": "https://app.polychat.co/m/2g36dGG1een5-s?code=Qk3…",
  "redeem_url": "https://app.polychat.co/m/2g36dGG1een5-s/redeem"
}

Opening the page

Preferred: form-POST the code to redeem_url from the user's browser, so the code never appears in a URL:

<form id="f" method="post" action="{redeem_url}">
  <input type="hidden" name="code" value="{code}">
</form>
<script>document.getElementById("f").submit()</script>

Or open url with a GET (a new tab, a redirect, or an iframe src on your allowed origins). The code is spent on first use and the page removes it from the address bar. A used, expired or unknown code gets one generic “This link is invalid or has expired” page (404).

The page shows that one tree and the conversation behind it. It does not show your system prompts, developer messages or tool definitions, has no account, billing or usage details and no links to other trees, and masks secrets. A reload works for a while in the same browser; a copied link without a code shows nothing.

DELETE /partner/memtree

Call it when a user deletes a chat or their account. It only touches requests sent with your key.

curl -X DELETE "https://api.polychat.co/partner/memtree?end_user_id=u_6f1c2d3e" \
  -H "Authorization: Bearer $PARTNER_KEY"
curl -X DELETE https://api.polychat.co/partner/memtree/2g36dGG1een5-s \
  -H "Authorization: Bearer $PARTNER_KEY"
{
  "end_user_id": "u_6f1c2d3e",
  "conversations": 1,
  "trees_deleted": 7,
  "kept_shared": 1,
  "in_progress": 0,
  "embeddings_deleted": 412,
  "stored_objects_deleted": 950,
  "requests_unlinked": 9,
  "shares_revoked": 0,
  "cleanup_pending": 0
}

DELETE /partner/memtree/{id} returns request_id instead of end_user_id. Deleted trees and the requests' links to them are unreachable once the call returns; trees in kept_shared stay for the conversations that still use them. in_progress counts trees still being built, which are deleted once they finish (unless something else still uses them); cleanup_pending counts storage cleanup that will be retried (the data is already unreachable).

Pricing

Context Memory is billed per token at these rates, with no markup on top:

Token Type Cost per million tokens
Input $3.75
Cached input $1.00
Output $1.25

What each call bills (the same quantities usage reports):

The system message, Claude Code's <system-reminder> blocks, images and other media, and tool definitions are not billed. Calls that search the memory also carry a small memory-search charge for embedding the query, and a vector search across sessions bills its query embedding (see search). Reading a MemTree is free.