ArticleAI Agent Reliability

AI Agent Pagination: Know When Results Are Complete

A completed page is not a completed collection. Put continuation, scope and stop reasons inside the tool boundary so your agent can state what it actually retrieved.

Published: · Last reviewed: 2026-11-15

Three glass trays linked by a cyan line, with document cards in the first and last trays, an empty middle tray, and an amber light at the end.
An empty page can still lead to more results; continuation evidence determines when traversal ends.
TL;DR

A successful page is evidence about one response. Mark retrieval complete only after your adapter observes the upstream terminal condition under the same authorized query scope. Otherwise return partial results with a reason. Traversal completion does not establish a snapshot of a changing collection.

The Missing Issue Is a Coverage Failure

Suppose you ask an agent to list every open issue assigned to your team. Its read-only tool returns eight issues and a continuation token. The request succeeds, the JSON validates, and the agent produces a polished list of eight. If it drops the token, the answer silently changes “every matching issue” into “the first retrieved page.” Neither successful execution nor a valid response schema resolves that difference.

Put traversal in the trusted tool boundary, where you can enforce scope and budgets. Give the model a result envelope that distinguishes complete retrieval from a useful partial result. The model can explain the limitation; it should not decide that a short page looks sufficiently final. This is separate from the transport choice discussed in MCP versus function calling.

Translate the Provider Contract Explicitly

Three pagination contracts illustrate why a generic “continue while token is truthy” helper is insufficient. These are attributed examples, not interchangeable wire formats. The MCP statements below refer to the checked 2026-07-28 specification.

InterfaceContinuation evidenceAdapter responsibility
MCP discovery listsOptional nextCursor; an empty string can be valid.Preserve opaque strings. A missing nextCursor signals the end; do not infer completion from page length.
GitHub REST paginationA Link header can supply a URL labeled rel="next".Parse the next relation according to the endpoint contract. Do not invent a URL by incrementing a counter.
DynamoDB QueryLastEvaluatedKey continues via ExclusiveStartKey.Normalize an absent or empty key to terminal. An empty Items array alone is insufficient.

The MCP pagination specification covers resources/list, resources/templates/list, prompts/list and tools/list. Cursors are opaque, server page sizes can vary, and clients must accept an empty string cursor. The separate tools specification describes discovery and invocation. Discovery pagination does not define a universal pagination contract inside arbitrary tools/call content. Your issue-search tool needs its own documented output contract.

GitHub’s REST pagination guide documents next-page links and notes that available relations vary. Its Link header is omitted when pagination is unsupported or the results fit on one page. Establish endpoint behavior before treating an absent next relation as your normalized terminal signal.

DynamoDB’s Query reference explicitly permits an empty result with LastEvaluatedKey when filtering removes the items read. The pagination guide instructs you to continue with that key and the same KeyConditionExpression. A nonempty key does not promise another matching item. It means traversal has not established the end.

Make the Coverage Claim Machine Readable

Use the following application contract for a bounded read-only traversal. These fields are a proposed internal design, not an MCP standard. Normalize provider-specific responses before the collector sees them. A malformed or ambiguous provider response must become an error, rather than a terminal page.

FieldMeaning
items and returnedCountThe accepted items and their array length. This is neither a provider total nor necessarily a count of unique records.
coveragecomplete only after an explicit normalized terminal page; partial for every other stopping condition.
reasonterminal, page_budget, cursor_cycle, scope_changed, invalid_page or upstream_error in this fixture.
scopeA trusted identifier for tenant, authorization context, endpoint, filter, ordering and adapter version.
pages and nextReceived pages and the last accepted continuation. A rejected response still consumes a page slot; a thrown request does not increment pages.
consistencyunspecified here. Change it only when an upstream guarantee supports a stronger statement for the whole traversal.

Pin the scope before the first request. Do not let the model change the tenant, filter or sort order while reusing a continuation. An authenticated adapter must derive scope from the actual outgoing request and authorized identity. A model-supplied string claiming “tenant A” proves nothing. The short scope string in the example is a fixture convenience; production code should compare a canonical descriptor or a digest of that descriptor without embedding credentials.

Keep continuation state private to the backend unless the caller needs it. Treat it as untrusted input when resuming, bind it to scope, and recheck authorization. In this example, next is diagnostic: a cursor-cycle result is not safely resumable. For an upstream timeout, retry behavior and cursor validity require provider-specific handling. Do not blindly expose every partial result as a “resume” button.

Execute the Boundary Against Adversarial Pages

Save this program as pagination-test.cjs and run node pagination-test.cjs. It uses Node.js built-ins and synthetic pages only. The adapter shape deliberately reserves null for termination and accepts every string, including the empty string, as a continuation. An actual DynamoDB adapter would preserve the structured key internally or map it to a backend handle; it should not pretend the key is an MCP cursor.

const assert = require('node:assert/strict');
// Internal adapter shape: next is null only for a documented terminal page.
// Scope is computed by the trusted adapter, not copied from model arguments.
async function collect(fetchPage, scope, maxPages = 10) {
  const items = [], seen = new Set();
  let cursor = null, pages = 0;
  const finish = (coverage, reason) => ({
    items, coverage, reason, pages, returnedCount: items.length,
    scope, next: cursor, consistency: 'unspecified'
  });
  while (pages < maxPages) {
    let page;
    try { page = await fetchPage({ cursor, scope }); }
    catch { return finish('partial', 'upstream_error'); }
    pages++;
    if (!page || page.scope !== scope)
      return finish('partial', 'scope_changed');
    if (!Array.isArray(page.items) ||
        !(page.next === null || typeof page.next === 'string'))
      return finish('partial', 'invalid_page');
    items.push(...page.items);
    cursor = page.next;
    if (cursor === null) return finish('complete', 'terminal');
    if (seen.has(cursor)) return finish('partial', 'cursor_cycle');
    seen.add(cursor);
  }
  return finish('partial', 'page_budget');
}

const S = 'tenant-A|open|id-asc|v1';
const p = (items, next, scope = S) => ({ items, next, scope });
const cases = [
  ['single', [p(['a'], null)], 10, 'complete', 'terminal', 1, ['a']],
  ['empty terminal', [p([], null)], 10, 'complete', 'terminal', 1, []],
  ['empty then match', [p([], 'c1'), p(['b'], null)], 10,
    'complete', 'terminal', 2, ['b']],
  ['empty string cursor', [p(['a'], ''), p(['b'], null)], 10,
    'complete', 'terminal', 2, ['a', 'b']],
  ['short page', [p(['a'], 'c1'), p(['b'], null)], 10,
    'complete', 'terminal', 2, ['a', 'b']],
  ['repeated cursor', [p(['a'], 'x'), p(['b'], 'x')], 10,
    'partial', 'cursor_cycle', 2, ['a', 'b']],
  ['long cycle', [p(['a'], 'x'), p(['b'], 'y'), p(['c'], 'x')], 10,
    'partial', 'cursor_cycle', 3, ['a', 'b', 'c']],
  ['budget', [p(['a'], 'x'), p(['b'], null)], 1,
    'partial', 'page_budget', 1, ['a']],
  ['scope change', [p(['a'], 'x'), p(['other'], null, 'tenant-B')], 10,
    'partial', 'scope_changed', 2, ['a']],
  ['upstream error', [p(['a'], 'x'), new Error('timeout')], 10,
    'partial', 'upstream_error', 2, ['a']],
  ['invalid token', [p(['a'], 0)], 10,
    'partial', 'invalid_page', 1, []],
  ['terminal at budget', [p(['a'], null)], 1,
    'complete', 'terminal', 1, ['a']]
];
(async () => {
  for (const [name, pages, budget, coverage, reason, calls, items] of cases) {
    const requests = [];
    const result = await collect(async request => {
      requests.push(request);
      const response = pages[requests.length - 1];
      if (response instanceof Error) throw response;
      return response;
    }, S, budget);
    assert.deepEqual([result.coverage, result.reason, requests.length, result.items],
      [coverage, reason, calls, items], name);
    assert(requests.every(r => r.scope === S), name);
    if (name === 'empty string cursor') assert.equal(requests[1].cursor, '');
    assert.equal(result.consistency, 'unspecified');
    console.log(`${name}: ${coverage}/${reason}; calls=${calls}; items=${items.length}`);
  }
  console.log(`PASS ${cases.length}/${cases.length}; runtime=${process.version}`);
})();

Executed locally on 2026-11-15 with Node.js v24.13.0: all twelve cases passed. The assertions compare coverage, stop reason, request count and the exact accepted item sequence. They also check that every request keeps the initial scope, that an empty string reaches the second request unchanged, and that consistency stays unspecified. This measures collector behavior on declared fixtures, with no API calls, model calls or production reliability estimate.

Synthetic caseObserved resultRequests / returned items
Single terminal pagecomplete / terminal1 / 1
Empty terminal pagecomplete / terminal1 / 0
Empty page, then a matchcomplete / terminal2 / 1
Empty string continuationcomplete / terminal2 / 2
Short page with continuationcomplete / terminal2 / 2
Repeated cursorpartial / cursor_cycle2 / 2
Three-page cursor cyclepartial / cursor_cycle3 / 3
One-page budget, more availablepartial / page_budget1 / 1
Scope changed on second pagepartial / scope_changed2 / 1
Second request throwspartial / upstream_error2 / 1
Numeric continuation tokenpartial / invalid_page1 / 0
Terminal page at budget boundarycomplete / terminal1 / 1

The last row checks the order of decisions: reaching your page budget after observing termination is complete. The budget row stops before making the second request and stays partial. The scope row discards the second response’s items, while retaining the first page as qualified partial data. Cursor-cycle detection runs after accepting the current page, so that page’s items remain visible with the failure reason.

Bound Work Without Inventing Completeness

A page cap protects against an endless chain even if each token is new. Add deadlines, per-request timeouts and memory or byte limits in a production boundary. The sample only enforces page count: one oversized page can still exceed memory, and a hung fetch can still wait indefinitely. Validate a positive integer budget in the public wrapper; the fixtures use valid fixed budgets.

Define what happens when an item or byte cap lands in the middle of a page. If you return only half the page and save its next-page cursor, resuming would skip the undisclosed half. Preserve the remainder in backend state, return a stable handle to a stored result, or restart under a clearly documented policy. A smaller requested page size does not by itself prove that your response budget will fit.

Keep exact retrieval evidence separate from the summary shown to the model. If the backend traverses everything but truncates the items sent to the agent, label that delivery as partial or provide an accessible complete artifact with its own scope and count. A complete backend traversal cannot justify an exhaustive answer based on an undisclosed subset. The structured output comparison addresses schema shape; coverage needs this additional contract.

Complete Traversal Is a Narrow Claim

Our complete state means that a terminal signal was observed through the declared adapter without a detected scope or traversal failure. It does not prove the adapter implements the provider correctly. The synthetic cases do not test HTTP Link parsing, authorization changes, DynamoDB key serialization or real cursor expiration. Add provider adapter fixtures and integration checks before relying on those paths.

Stable query arguments also do not freeze a changing collection. Records may move, appear or disappear between requests. A terminal signal is not evidence that the combined pages represent one instant. Require an explicit upstream snapshot or equivalent traversal-wide guarantee before adding snapshot language. Deduplicating by stable record ID can remove repeated records; it cannot reconstruct records you never received, and it changes the meaning of returnedCount.

Return a user-facing statement consistent with the envelope. For the budget fixture: “Retrieved one issue before the page limit; additional results may remain.” For a terminal traversal without a snapshot guarantee: “Reached the end of this query’s pagination; the collection may have changed during retrieval.” Reserve “no matching issues” for a terminal empty traversal within the declared scope. A failed first request supplies no evidence of absence.

FAQ

Does a successful tool call mean all results were retrieved?

No. It can establish that one page succeeded. Collection completeness requires reaching the documented terminal condition under the same query scope.

Should an agent stop after an empty page?

Only if the provider contract identifies that response as terminal. DynamoDB Query can return an empty page with a continuation key.

Can an empty string be a valid MCP pagination cursor?

Yes. In the checked MCP specification, an empty string can be a valid cursor and must not be treated as the end of results.

Does complete pagination guarantee a snapshot?

No. This contract records traversal completion. A snapshot claim needs an explicit upstream guarantee covering the traversal.

Primary sources checked 2026-11-15