Skip to content
Last updated

Attempt Queueing

When a pool reaches its concurrency limit, new attempts assigned to that pool are either queued or rejected. Queueing behavior depends on how the attempt was created.

Queueing by attempt source

Attempt sourceQueueingNotes
Workflow scheduleEnabledScheduled attempts are queued when the pool is full.
require> operatorEnabledAttempts created by require> are queued when the pool is full.
BackfillEnabledBackfill attempts are queued when the pool is full.
PUT /api/attempts Treasure Workflow APIConditionalQueued only when the request includes allowQueueing=true. Otherwise the attempt creation request fails as soon as the pool is full. Data Workbench sends allowQueueing=true, so attempts created from Data Workbench are queued when the pool is full.
Workflow triggerEnabledTrigger-initiated attempts are queued when the pool is full.
Avoid relying on allowQueueing=false

Queueing is the default behavior for every other attempt source. A future release will queue API-initiated attempts unconditionally as well, at which point allowQueueing=false will no longer be accepted. Do not build workflows that depend on the current fail-fast behavior of allowQueueing=false.

Per-pool waiting-attempt limit

Each pool has a separate cap on the number of attempts that are waiting to run. The waiting-attempt limit covers every attempt in the pool that has not yet started running, regardless of whether it is blocked or queued. Once a pool reaches the waiting-attempt limit, any new attempt assigned to that pool fails to be created, even if queueing would otherwise be enabled for that source. The waiting-attempt limit is separate from the pool's concurrency limit, which caps the number of attempts that can run at the same time.

Per pool, not per account

The waiting-attempt limit applies to each pool individually. Reaching the limit on one pool does not prevent attempts from being created in another pool.

See Treasure Workflow Prerequisites and Limitations for the current waiting-attempt limit value.

What happens when an attempt cannot be queued

If the pool is full and either queueing is not enabled for the attempt source or the pool has reached its waiting-attempt limit, the attempt creation request fails. Different sources recover differently:

  • Schedule: the scheduler keeps retrying the attempt creation on its own until either the attempt is created or a skip_delayed_by condition on the schedule is met, at which point the attempt is skipped.
  • require> operator: the parent attempt keeps retrying the attempt creation on its own until it succeeds.
  • Workflow trigger: the trigger keeps retrying the attempt creation on its own until it succeeds.
  • PUT /api/attempts Treasure Workflow API: the API client sees the failure in the response and must retry the request itself.