Explore more
...
Scenarios
Incomplete executions

Automatic retry of incomplete executions

4min

 checks the origin of every incomplete execution when it's created. automatically retries incomplete executions that have been created because of:

  • RateLimitError
  • ConnectionError
  • ModuleTimeoutError

and incomplete executions that were created with the Break error handler with the automatic run completion enabled.

 takes the following steps when retries incomplete executions:

  1.  schedules the incomplete execution retries.
  2.  retries the incomplete execution.
  3. Based on the incomplete execution retry result schedules another attempt or marks the incomplete execution resolved.

Automatic retry scheduling

 schedules automatic retries with the incomplete executions backoff schedule. The backoff schedule prevents the situation when you would get the same error multiple times in a row.

For example, when you get the ConnectionError because the app is unavailable, it might take some time until it's back. spaces out the retry attempts to get a successful retry even at a later time after the original error.

Error type

Retry schedule

  • RateLimitError
  • ConnectionError
  • ModuleTimeoutError
  1. 1 minute (1 minute after the original run).
  2. 10 minutes (11 minutes after the original run).
  3. 10 minutes (21 minutes after the original run).
  4. 30 minutes (51 minutes after the original run).
  5. 30 minutes (1 hour 21 minutes after the original run).
  6. 30 minutes (1 hour 51 minutes after the original run).
  7. 3 hours (4 hours 51 minutes after the original run).
  8. 3 hours (7 hours 51 minutes after the original run).

Error handled by the break error handler

If you enable automatic run completion in the error handler settings. The default is:

  • maximum number of retry attempts: 3
  • retry delay: 15 minutes

You can customize the defaults in the error handler settings.

Other error types usually require changes in the incomplete execution and manual resolving. doesn't retry these error types automatically by default.

Automatic retry processing

After schedules the retries, runs the again, starting with the module that caused the error.

For each , there is a limit of 3 incomplete execution retries running in parallel. If there are more incomplete executions scheduled from the same , retries them in batches of 3 after the previous batch finishes.

In addition, the retry doesn't start when the original is running already.

The 3 parallel retries limit applies to retries from the same . When retries incomplete executions from multiple , then each of them has their own limit.

This limitation is to prevent your from getting follow-up rate limit errors if you are retrying a lot of incomplete executions at the same time.

For example:

You have a that runs for 10 minutes every hour. There was a disruption of a third party service for 5 hours.

  • The now has 5 incomplete executions scheduled for automatic retry.
  •  first waits until the original finishes if it's running already. This takes 10 minutes if the started just now.
  • After the finishes, retries the first 3 incomplete executions. This takes an additional 10 minutes (20 in total).
  •  retries the remaining 2 incomplete executions after the previous batch finishes. This takes another 10 minutes (30 in total).
  • After another 30 minutes (1 hour in total), starts the again according to the schedule.

Automatic retry result

If a retry attempt succeeds, marks the incomplete execution as Resolved and stops retrying.

If all of the retry attempts fail, marks the incomplete execution as Unresolved. You can then retry the incomplete execution when the app that caused the error is available again or you can resolve the incomplete execution manually.