Error Taxonomy
Source of truth:The QTG v3 error hierarchy classifies executor execution results into three categories: retryable / fatal / ambiguous. This classification determines node-state transitions and request-level judgment.src/qtg/domain/errors.pyError handling patterns:src/qtg/infrastructure/executors/cctp/burn.py,src/qtg/infrastructure/executors/observe/common.pyCEX-specific errors:src/qtg/infrastructure/executors/cex/adapters/errors.py
Table of Contents
- Error class hierarchy
- MovementError — Base
- TemporaryMovementError
- FatalMovementError
- AmbiguousMovementError
- Signed-artifact guard error codes
- Subclasses: specialized errors
- Error -> NodeState transition mapping
- Executor error-handling patterns
- Observe helper: fatal_result() / temporary_result()
- CEX adapter error system
- Cross-References
Error class hierarchy
All errors carrymessage (str) and detail (object | None). MovementError.__init__ also sets self.code = message, so every error exposes a code attribute (defaulting to the message string) that callers can read without unpacking detail.
MovementError
self.code defaults to the message string — distinct from the structured detail["error_code"] that executors attach (see the signed-artifact guard error codes section).
detail field convention
detail is usually passed as a dict, and executors use it to carry error codes and additional information:
detail as a dict and extract error_code:
TemporaryMovementError
When to use
- transient RPC node failure (timeout, connection reset)
- exchange API rate limit
- transient network instability
- 503/429 responses from external services
Runtime behavior
WhenTemporaryMovementError occurs, it is handled differently depending on the executor stage:
Key point:
TemporaryMovementError during submit() transitions to UNKNOWN. A signed tx may already have been sent, so simple retry is not possible.
Additional fields in detail
Fields thattemporary_result() extracts from detail:
FatalMovementError
When to use
- configuration error (RPC endpoint missing, probe unregistered)
- insufficient balance (on-chain)
- insufficient allowance
- address validation failure
- protocol-level deterministic error (tx reverted, nonce conflict)
Runtime behavior
FatalMovementError always transitions to FAILED. It is not retried.
error_code convention in detail
AmbiguousMovementError
When to use
- connection drops after transaction submission but before receipt confirmation
- timeout before receiving the response after calling an exchange withdrawal API
- impossible to tell whether the external system is processing or has failed
Runtime behavior
AmbiguousMovementError is not directly caught in the current code, but in the domain model it expresses the meaning of UNKNOWN NodeState. Executor implementations usually use TemporaryMovementError and handle ambiguous situations by returning UNKNOWN as next_state.
Design intent: a type that explicitly distinguishes cases where it is impossible to decide whether a side effect occurred. It can be used later when executors perform more refined error classification.
Signed-artifact guard error codes
Source of truth:On-chain send executors enforce the signed-recipient invariant: the recipient/source/target re-derived from the actual signed artifact must equal the allowlist-validated authority. The check runs at prepare-time (intent-only resolvers) and again at submit/broadcast-time — the signed legacy tx is RLP-decoded and the Gateway burn-intent EIP-712 digest is re-hashed. Every guard failure raises a non-retryablesrc/qtg/infrastructure/executors/recipient_guard.py,src/qtg/infrastructure/executors/signed_evm_tx.py
FatalMovementError with detail["error_code"] set to one of the codes below. Because these are FatalMovementError, they always fail closed to FAILED (terminal, no retry) when raised in any executor stage.
Action-type scope (
ONCHAIN_ACTION_TYPES in application/services/address_allowlist.py):
- Destination-only (destination address validated; source is signer-bound):
stargate_send,usdt0_send,lighter_secure_withdraw. - Strict source + destination:
cctp_burn,cctp_mint,gateway_approve,gateway_deposit,gateway_intent,gateway_mint,ccip_send,evm_erc20_transfer.
Submit-time guards close the “validated address != signed address” fund-outflow bypass. CCIP participates in the same signed-artifact guard set: the submit path decodes the signed
ccipSend transaction and pins router, destination chain selector, receiver, token, amount, native fee, signer identity, and source chain id before sidecar broadcast. A mismatch fails closed as ccip_signed_artifact_mismatch.Subclasses: specialized errors
MovementNotFoundError
- Subclass of
FatalMovementError - Raised when a resource such as a request, template, or node cannot be found
- Default message:
"not found" - Mapped to HTTP 404 at the API layer
MovementConflictError
- Raised on state conflicts (for example, trying to approve an already-approved request again)
- Mapped to HTTP 409 at the API layer
MovementValidationError
- Raised when input validation fails (for example, required field missing, invalid format)
- Mapped to HTTP 422 at the API layer
MakerCheckerError
- Raised by the approval path when
MG_REQUIRE_MAKER_CHECKERis on and the authenticated approver key equals the key that created the movement detailcarrieserror_code = maker_checker_violationpluscreator_key_id- Mapped to HTTP 403 at the API layer — the only domain error that maps to 403
ConfigRefResolutionError
- Raised when the runtime
build_execution_context()stage cannot resolve a top-level$ref:node_key.field - Representative detail fields:
config_keyref_patherror_code = CONFIG_REF_UNRESOLVED
- The dispatcher catches it and transitions the current node to
FAILED
BalanceReservationError
- Subclass of
MovementConflictError(and thusFatalMovementError) - Raised at approval/reservation time when a pool has no balance snapshot or insufficient withdrawable amount
- Carries an extra
reasonattribute on top ofmessage/detail - See the CEX balance scope debug skill for the typical
no snapshot or null withdrawablefailure chain
IllegalStateTransitionError
- Subclass of
MovementValidationError - Raised when
set_node_state/set_request_stateis given a transition not inNODE_TRANSITIONS/REQUEST_TRANSITIONSwithoutallow_transition_override=True detailcarrieserror_code = ILLEGAL_STATE_TRANSITIONplusentity,old,new
BindingResolutionError / AmbiguousBindingError
- Both subclass
MovementValidationError BindingResolutionError— no active registry row resolves the executor/signer/venue selectorAmbiguousBindingError— more than one active registry row matches and the selector cannot pick deterministically
Summary: subclasses -> HTTP mapping
Error -> NodeState transition mapping
Overall picture of how errors map to NodeState transitions:Detailed stage-by-stage mapping
preflight()TemporaryMovementError is treated as FAILED. This is safe because there is no side effect at that stage.
submit()
TemporaryMovementError is treated as UNKNOWN. The transaction may already have been sent.
observe()
TemporaryMovementError is treated as OBSERVING (waiting for retry). FatalMovementError is treated as FAILED.
Executor error-handling patterns
Pattern 1: CCTP Action Executor (burn/mint)
The CCTP executor catches errors directly in each of the preflight/submit/observe/recover stages and converts them intoExecutionResult.
preflight() pattern:
- In preflight, exceptions are all mapped to FAILED (no side effect)
- If
detailis a dict, extracterror_code; otherwise use the default code return Nonemeans preflight passed (proceed to the next stage)
- In submit,
TemporaryMovementError->UNKNOWN(unclear whether the tx was sent) - On success, store
txidinprovider_refswithCOMPLETED FatalMovementErroris not caught — fatal conditions should not happen in submit (they should be filtered in preflight)
- If
provider_contextcontains a txid ->COMPLETED(transaction confirmed) - If there is no txid ->
FAILED(cannot recover)
Pattern 2: CctpMintExecutor.recover() — advanced recovery
- txid exists -> inspect receipt -> 3-way branch: success/failure/unknown
- no txid -> check mintability with eth_call -> 3-way branch: already processed/not processed/error
Observe helper
Shared helper for observe executors defined insrc/qtg/infrastructure/executors/observe/common.py.
fatal_result()
- Always returns
next_state="FAILED" - Supports
KeyErrorhandling (occurs when the probe registry is missing an entry) - Uses
detail.error_codefromFatalMovementErrorif present, otherwise falls back todefault_code
temporary_result()
- Always returns
next_state="OBSERVING"(waiting for retry) - Preserves
provider_refs(keeps existing context) - Default
retry_after_secondsis 15 seconds - Default
provider_stateis"rpc_retry"
Comparing fatal_result vs temporary_result
CEX adapter error system
The CEX executor (CexWithdrawalActionExecutor) uses a separate TransferError hierarchy instead of the domain error hierarchy. These errors are caught inside the executor and converted into ExecutionResult.
TransferError hierarchy
CEX error -> ExecutionResult mapping
Key point: In CEX submit,
httpx.TimeoutException / httpx.ConnectError are treated as UNKNOWN. The exchange may already have received the withdrawal request.
Summary of error-handling design principles
- In preflight, every error becomes FAILED: safe to fail because there is no side effect
- In submit, uncertainty becomes UNKNOWN: simple retry is not possible because external state may already have changed
- In observe, transient failures become OBSERVING: retry observation while preserving
provider_refs - In recover, inability to confirm stays UNKNOWN: reserve another recovery attempt
- side-effect node completed + another node failed = MANUAL_INTERVENTION: automatic judgment is impossible while funds are in transit
- detail.error_code convention: executors provide structured error codes to support operator tracking
Cross-References
- states-and-transitions.md — NodeState.UNKNOWN recovery and MANUAL_INTERVENTION judgment
- types-and-enums.md — NodeKind action vs observe distinction
- ../executors/cctp-lane.md — details of CCTP burn/mint executors
- ../executors/cex-lane.md — details of CEX withdrawal/observe executors
- ../executors/observe-probes.md — observe executor and probe architecture
- ../workers/runtime-workers.md — error handling in dispatch/observe/recover workers