Skip to main content

Nexus operations

SUPPORT, STABILITY, and DEPENDENCY INFO

Temporal Nexus is now Generally Available for Temporal Cloud and self-hosted deployments.

Nexus Operations can be synchronous or asynchronous. Unlike a traditional RPC, an asynchronous Nexus Operation has an operation token that can be used to re-attach to a long-running Operation backed by a Workflow. An Operation's lifecycle spans scheduling, reliable delivery with retries, handler execution, and result or callback completion.

SDK support​

SDK GUIDES

Caller side: A caller Workflow executes a Nexus Operation through a Nexus Endpoint using the Temporal SDK.

Handler side: Nexus Services and their Operations are registered with a Worker that polls the Endpoint's target Task Queue. Operations are defined using SDK builder functions:

  • New-Workflow-Run-Operation - Start a Workflow as an asynchronous Operation.
  • New-Sync-Operation - Run a synchronous Operation: invoke a Query, Signal, or Update, or execute other reliable code using the Temporal SDK Client.

Nexus Operation lifecycle​

When a caller Workflow executes a Nexus Operation, the command is atomically handed off to the Nexus Machinery. The Machinery ensures at-least-once execution with automatic retries and reliable result delivery.

Nexus Overview

Nexus Overview

Synchronous Operation lifecycle​

Synchronous Operations must complete within the 10-second handler deadline, as measured from the caller's Nexus Machinery.

Nexus Sync Operation Lifecycle

Nexus Sync Operation Lifecycle

Lifecycle for a synchronous Operation (for example, to Signal, Query, or Update a Workflow, or run other reliable code):

  1. Caller Workflow executes a Nexus Operation.
  2. Caller Worker issues a ScheduleNexusOperation command.
  3. Caller Namespace records a NexusOperationScheduled event.
  4. Caller Nexus Machinery sends the start request.
  5. Handler Nexus Machinery sync-matches the request to a handler Worker.
  6. Handler Worker receives a Nexus Task by polling the Endpoint's target Task Queue.
  7. Handler processes the task using New-Sync-Operation.
  8. Handler responds with the Operation result.
  9. Caller Namespace records a Completed or Failed event.
  10. Caller Worker polls for a Workflow Task.
  11. Caller Workflow receives the result.
Nexus

Nexus

tip

Stay within the request deadline to avoid timeouts. Timed-out handlers are retried until the Operation's Schedule-to-Close timeout is exceeded.

Asynchronous Operation lifecycle​

Asynchronous Operations can run up to 60 days (the maximum Schedule-to-Close timeout in Temporal Cloud). Differences from the synchronous lifecycle are in bold.

Nexus Async Operation Lifecycle

Nexus Async Operation Lifecycle

  1. Caller Workflow executes a Nexus Operation.
  2. Caller Worker issues a ScheduleNexusOperation command.
  3. Caller Namespace records a NexusOperationScheduled event.
  4. Caller Nexus Machinery sends the start request.
  5. Handler Nexus Machinery sync-matches the request to a handler Worker.
  6. Handler Worker receives a Nexus Task by polling the Endpoint's target Task Queue.
  7. Handler processes the task using New-Workflow-Run-Operation.
  8. Handler responds with the start Operation response.
  9. Caller Namespace records a NexusOperationStarted event.
  10. Handler Workflow completes and a Nexus Completion Callback is delivered to the caller's Nexus Machinery.
  11. Caller Namespace records a Completed or Failed event.
  12. Caller Worker polls for a Workflow Task.
  13. Caller Workflow receives the result.
Nexus

Nexus

Executing code from a synchronous handler​

Synchronous handlers can execute code directly but must complete within the handler deadline. Use the Temporal SDK Client to invoke Signals, Queries, Updates, or other reliable code.

caution

Use async Operations for long-running work. Repeated sync handler failures can trip the circuit breaker, blocking all Operations from that caller to the Endpoint.

Nexus Operations with Arbitrary Code

Nexus Operations with Arbitrary Code

System interactions​

Nexus uses the same queue-based Worker architecture as the rest of Temporal. Workers interact with their Namespace gRPC endpoint. Nexus Machinery on both sides handles cross-Namespace communication.

Nexus Queue-based Worker Architecture

Nexus Queue-based Worker Architecture

At a high level, when a caller Workflow executes a Nexus Operation:

  1. The caller Worker schedules the Operation with a ScheduleNexusOperation command, atomically handing off execution to the caller's Nexus Machinery.
  2. The handler Worker receives a Nexus Task by polling the Endpoint's target Task Queue.
  3. The handler processes the task and returns the result (synchronous) or an Operation token (asynchronous).
  4. The caller's Nexus Machinery records a NexusOperation event (Started, Completed, Failed, Canceled, or TimedOut) in the caller's Workflow History.

Automatic retries​

Once an Operation is scheduled, the caller's Nexus Machinery retries on retryable errors or upstream timeouts until the Operation's Schedule-to-Close timeout is exceeded. Retries follow the default Retry Policy.

note

This differs from Activity and Workflow error handling. See errors in Activities and non-retryable errors.

To control retry behavior, return a non-retryable Nexus error. See errors in Nexus handlers.

Circuit breaking​

Nexus implements circuit breaking per caller-Namespace/Endpoint pair ("destination pair"). Each destination pair trips and resets independently. By default, the circuit breaker activates after 5 consecutive retryable errors.

After tripping, the circuit breaker enters the open state and stops sending requests. After 60 seconds, it transitions to half-open, allowing a single probe request. If the probe succeeds, the circuit breaker returns to closed (normal operation). If it fails, the circuit breaker returns to open for another 60 seconds.

Flow chart showing the states of the Temporal Nexus Circuit Breaker

Flow chart showing the states of the Temporal Nexus Circuit Breaker

Circuit breaker state surfaces in Pending Nexus Operations and Pending Callbacks. Check it in the UI, CLI, or DescribeWorkflowExecution API.

When open, pending Operations show a Blocked state with a BlockedReason:

Circuit Breaking

Circuit Breaking

Different Operations within the same destination pair contribute to the trip count. A given Operation may have fewer than 5 attempts when the circuit breaker opens.

From the CLI:

temporal workflow describe -w my-workflow-id
Pending Nexus Operations: 1

  Endpoint                 my-nexus-endpoint
  Service                  nexus-playground
  Operation                sync-op-ok
  State                    Blocked
  Attempt                  1
  LastAttemptFailure       {"message":"handler error (UPSTREAM_TIMEOUT): upstream timeout",...}
  BlockedReason            The circuit breaker is open.

Cancellation requests surface the same pattern with CancelationState: Blocked and CancelationBlockedReason.

Execution semantics​

At-least-once execution semantics and idempotency​

Nexus provides at-least-once execution up to the Schedule-to-Close timeout. The Machinery retries on handler timeouts or retryable errors, so a handler may be invoked multiple times for the same Operation.

Nexus Operation handlers should be idempotent, similar to Activities. Not strictly required in all cases, but highly recommended.

Exactly-once execution semantics​

To upgrade to exactly-once, back your Operation with a Workflow that uses a WorkflowIDReusePolicy of RejectDuplicates. This allows only one Workflow Execution per Workflow ID within a Namespace for the Retention Period.

Cancelation​

Cancelling a caller Workflow automatically propagates to all pending Nexus Operations and their underlying handler Workflows. A canceled handler Workflow reports a Canceled Failure to the caller.

Termination​

Terminating a caller Workflow abandons all pending Nexus Operations. Prefer cancellation when possible.

Versioning​

Task Routing is the simplest way to version Nexus service code. For backward-incompatible changes, use a different Service name and Task Queue (for example, prod.payments.v2). Callers migrate to the new version on their own deployment schedule.

Attaching multiple Nexus callers to a handler Workflow​

SUPPORT, STABILITY, and DEPENDENCY INFO

Using a Conflict-Policy of Use-Existing with New-Workflow-Run-Operation is in Public Preview.

Operations started with New-Workflow-Run-Operation automatically attach a completion Callback to the handler Workflow. Additional callers can attach to the same handler Workflow using a Conflict-Policy of Use-Existing.

Each handler Workflow has a Callback limit (32 in Cloud, configurable in self-hosted). Callers that exceed the limit receive an error.

When a handler Workflow uses Continue-As-New, existing completion Callbacks are copied to the new Execution. The previous Execution's Callbacks remain in Standby state indefinitely.