Skip to content
Durability

DurabilityTracker

DurabilityTracker API, behavior, and recovery contract.

Import DurabilityTracker from durability/experimental and pin an exact version. Use one query/wait interface over capabilities or your own TrackedSource implementations.

How it uses your Durable Object

DurabilityTracker has no context, tables, or alarm of its own. It reads what its sources already keep in the object:

  • ctx.storage.sql belongs to the sources. Durability reads durability_calls, DurabilityAlarms reads durability_alarms, and a custom source reads whatever table you give it. A record is never copied into the tracker.
  • The object’s one alarm is set by the sources’ scheduler, not by the tracker. It settles pending work after an eviction, and the tracker needs no delegated method; your class’s alarm() still delegates to the scheduler or capability.
  • The live instance holds every handle, wait() timer, and settlement listener. A deploy, eviction, or RPC disconnect discards them; reacquire a handle with get(source, id).
  • RPC between objects copies a handle’s data and turns get and wait into stubs that call back into the instance that created them. JSON transports get plain records() instead.

How DurabilityTracker uses your Durable Object

Shows Pipeline, DurabilityTracker, Durability. Focuses on DurabilityTracker.Pipeline Durable ObjectPipelineyour Durable ObjectDurabilityTrackerin memoryDurabilitysource: jobsget('jobs', id)get, watch

1/8 · The tracker is a field of the object

The Pipeline constructs the tracker with its Durability helper as the jobs source. The tracker has no context, no tables, and no alarm: everything it returns is read from its sources.

Open source ↗
/**
 * Producer side of the `job-dashboard` example. Runs durable operations, fans
 * their lifecycle out to the tenant's `Dashboard`, and hands out tracker
 * handles so callers can wait for a result over Workers RPC. Browsers never
 * connect here.
 */
import { DurableObject } from 'cloudflare:workers';
import {
  Durability,
  DurabilityFanout,
  DurabilityScheduler,
  type DurableCall,
} from 'durability';
import { DurabilityTracker } from 'durability/experimental';
import type { Env, JobEvent, OrderSummary } from './shared';

export class Pipeline extends DurableObject<Env> {
  private readonly scheduler = new DurabilityScheduler({ context: this.ctx });

  private readonly durability = new Durability({
    scheduler: this.scheduler,
    handlers: {
      importOrder: async ({
        id,
        payload,
        signal,
      }: DurableCall<{ orderId: string }>): Promise<OrderSummary> => {
        const response = await this.env.ORDERS.fetch(
          `https://orders/${payload.orderId}`,
          { headers: { 'Idempotency-Key': id }, signal }
        );
        if (!response.ok) {
          throw new Error(`Order lookup failed with ${response.status}`);
        }
        return response.json();
      },
    },
    // Completed lifecycle events carry the summary the dashboard publishes.
    includeResults: true,
  });

  private readonly tracker = new DurabilityTracker({
    sources: { jobs: this.durability },
  });

  // Push side. Delivery is durable: the dashboard gets the batch again after
  // an eviction or a failed call, until `ingest` returns without throwing.
  private readonly fanout = new DurabilityFanout<JobEvent>({
    scheduler: this.scheduler,
    targets: {
      dashboard: {
        deliver: (messages) =>
          this.env.DASHBOARD.getByName(this.tenant).ingest(
            messages.map((message) => message.body)
          ),
        ackOnReturn: true,
      },
    },
    batchDelayMs: 100,
  });

  constructor(ctx: DurableObjectState, env: Env) {
    super(ctx, env);
    // Lifecycle hooks are best-effort. If the object dies between a job
    // settling and this enqueue committing, that one event is lost until the
    // dashboard resyncs. Enqueue from inside the handler, keyed by `id`, when
    // that gap matters.
    this.durability.lifecycle((event) => {
      if (event.type === 'registered') {
        return this.publish(event, { status: 'pending' });
      }
      if (
        event.type === 'attempt_settled' &&
        event.outcome === 'completed' &&
        event.result !== undefined
      ) {
        return this.publish(event, {
          status: 'completed',
          result: event.result,
        });
      }
      if (
        (event.type === 'attempt_settled' && event.outcome === 'failed') ||
        event.type === 'terminal'
      ) {
        return this.publish(event, { status: 'failed', error: event.error });
      }
      return undefined;
    });
  }

  private get tenant(): string {
    return this.ctx.id.name ?? this.ctx.id.toString();
  }

  private async publish(
    event: { id: string; generation: string; timestamp: number },
    state: JobEvent['state']
  ): Promise<void> {
    await this.fanout.enqueue([
      {
        id: `${event.id}:${event.generation}:${state.status}`,
        body: {
          id: event.id,
          name: 'importOrder',
          generation: event.generation,
          createdAt: event.timestamp,
          state,
        },
      },
    ]);
  }

  /** Registers the import; resolves on acceptance, not completion. */
  importOrder(orderId: string) {
    return this.durability.importOrder({
      id: `import:${orderId}`,
      payload: { orderId },
    });
  }

  /**
   * Pull side. Over Workers RPC the handle's `get` and `wait` become stubs, so
   * `stub.track(id).wait({ timeoutMs })` waits in one pipelined round trip.
   */
  async track(orderId: string) {
    const handle = await this.tracker.get('jobs', `import:${orderId}`);
    if (!handle) {
      throw new Error(`Order "${orderId}" was never imported`);
    }
    return handle;
  }

  override alarm(info?: AlarmInvocationInfo) {
    return this.scheduler.alarm(info);
  }
}
Wait for a job over RPC
Details

Follow one wait from a Worker’s pipelined call to the settled state read back from the Pipeline’s storage.

Durable state
Nothing; the tracker writes no storage.
In-flight / in memory
The tracker and its source list, rebuilt with the object instance.

Click the diagram to open a full-screen explorer with the complete source files. Select a component to jump to its code, or select a linked code line to highlight the affected architecture. Use the ← / → keys or on-screen arrows to move through the flow; each step’s explanation sits under the diagram. Escape or the close button returns to the guide. Failure scenarios, the full step list, and durable/in-memory state notes are available under Details. These are explanatory traces of the real source, not executable simulations or guarantees about an external service.

Behind the scenes

Durable state: none of its own. The tracker delegates reads to its named sources. Each source owns its records, transactions, status semantics, and retention. Pagination visits sources in configured order; there is no globally ordered merged timeline.

A handle contains a record snapshot, its generation, and get()/wait() methods. Later reads compare generations so an old handle cannot accidentally observe a new record reusing the same ID.

Waiting is in memory: installing the watch before the initial read avoids a settlement-between-read-and-subscribe race, but watches and timers do not survive eviction, redeploy, or RPC disconnect. Reacquire a handle by ID and wait again. A custom source without watch is read at timeout; the tracker does not poll it continuously.

Transport: Workers RPC can carry callable handles; JSON sockets cannot. Use records() for snapshots over sockets. A dashboard projection and authoritative pipeline can each expose a tracker, but querying the former does not automatically reconcile it with the latter.

See live job dashboards for both transports and their failure boundaries.

Implementation: tracker.ts (collection and waiting) and tracked-source.ts.

Tracking work (experimental)

Everything under durability/experimental may change or be removed in a minor release and is excluded from the semver guarantee of the root package. Pin an exact version when depending on it. Nothing from this entry point is reachable from durability, and it adds nothing to the public types of Durability or DurabilityAlarms.

DurabilityTracker lists records from several sources. Sources are keyed by names you choose; the key becomes each record’s source.

import { DurabilityTracker } from 'durability/experimental';

export class ImageJobs extends DurableObject<Env> {
  // scheduler, durability, and alarms as above

  private readonly tracker = new DurabilityTracker({
    sources: { jobs: this.durability, alarms: this.alarms },
  });

  listWork(cursor?: string) {
    return this.tracker.list({ status: 'pending', limit: 50, cursor });
  }

  getWork(id: string) {
    return this.tracker.get('jobs', id);
  }
}

All three methods take the same query: { source, name, status, limit, cursor }. limit defaults to 50 and is capped at 100. Pass cursor back with the same query to continue; it is present only while more records may follow. Sources are read one after another, so there is no ordering across sources.

  • list(query) returns { items, cursor }. Each item has source, id, name, generation, createdAt, and state, plus get(), which reads the latest state, and wait({ timeoutMs }), which resolves once the record is completed or failed, or returns the latest state after the timeout without polling storage. Both return not_found once the record is purged or its ID is reused for new work.
  • get(source, id) returns one such handle, or undefined.
  • records(query) returns { records, cursor } with the same data and no functions.

Narrowing on source and name narrows state.result to that handler’s result type. A pending operation or alarm also lists state.memoized, the keys its next attempt will reuse.

Handles over RPC

Return handles from a Durable Object method. Over Workers RPC or Cap’n Web, the data fields are copied and get and wait become stubs, so a caller can wait without polling:

const { items } = await env.IMAGE_JOBS.getByName('jobs').listWork();
for (const item of items) {
  const result = await item.wait({ timeoutMs: 30_000 });
}

Workers RPC pipelining also lets a caller wait in the same round trip that returns the handle, for example stub.getWork(id).wait({ timeoutMs }) when the method always returns a handle. Waiting is in memory: if a deployment, eviction, or disconnect interrupts it, get a new handle by ID and wait again. Stubs follow the RPC system’s disposal rules; dispose them with using or let the request end.

Records over sockets

DurabilitySocket frames are JSON, so functions do not survive the trip. Expose records as an inbound snapshot and push changes as outbound events, for example from a lifecycle hook:

readonly socket = new DurabilitySocket({
  context: this.ctx,
  inbound: {
    work: {
      input: z.object({ cursor: z.string().optional() }),
      handle: (query: { cursor?: string }) => this.tracker.records(query),
    },
  },
  outbound: { settled: event<DurabilityOperationEvent<typeof handlers>>() },
});

A complete two-object version, with push through DurabilityFanout and request/response through a pipelined wait, lives in examples/job-dashboard.

Sources

Durability lists every operation. DurabilityAlarms lists pending and failed occurrences only, because success deletes an occurrence; a waiter still sees completed. Rescheduling an alarm replaces its occurrence, so waiters on the old one receive not_found.

Any object implementing TrackedSource can be a source, for example work kept in the application’s own table:

import type { TrackedSource } from 'durability/experimental';

type Approvals = { approval: { approvedBy: string } };

const approvals: TrackedSource<Approvals> = {
  get: async (id) => readApproval(id),
  list: async ({ limit, cursor, status, name }) =>
    listApprovals(limit, cursor, status, name),
  watch: (id, listener) => subscribeToApproval(id, listener),
};

new DurabilityTracker({ sources: { jobs: this.durability, approvals } });
  • get(id) and list(query) return records with id, name, generation, createdAt, and state. Keep generation stable unless the ID is reused for new work.
  • state.status is pending, completed, or failed. Express finer outcomes, such as a rejection, as failed with a specific error.name.
  • list may return fewer than limit records together with a cursor. Omit the cursor at the end.
  • watch(id, listener) is optional and returns an unsubscribe function. Call listener() after a record may have settled; pass the settled record when it can no longer be read. Without watch, wait reads again only at its timeout.

New members of TrackedSource are always optional, so a custom source keeps compiling across releases.

Edit this page
Durability

Type to search…

↑↓ navigate↵ selectEsc close