Fixing rate limit errors

in {{product name}} , you'll see the ratelimiterror when requests to an app have exceeded its api rate limit —the maximum number of requests allowed within a certain time period (e g , 100 requests per minute) when its rate limit is exceeded, the app blocks further requests until the specified timeframe passes for an example, see the api rate limits for google sheets the rate limit error is common in {{scenario plural lowercase}} with instant triggers (e g , webhooks ), apps that process many records in a short time (e g , google sheets or airtable ), and apps that you use multiple times across {{scenario plural lowercase}} this article outlines strategies for preventing rate limit errors in instant {{scenario plural lowercase}} , scheduled {{scenario plural lowercase}} , and both types the rate limit error in {{product name}} co rresponds to the http 429 too many requests error code while {{product name}} follows the standard error codes and their definitions, third party apps may not always comply with this standard strategies by scheduling type when a module returns the rate limit error and no error handler is present, {{product name}} responds based on the following factors the {{scenario singular lowercase}} scheduling type whether incomplete executions are enabled the chart below outlines all possible responses to encountering a rate limit error scheduling type incomplete executions disabled incomplete executions enabled scheduled {{product name}} pauses the next {{scenario singular lowercase}} run for 20 minutes {{product name}} doesn't rerun the scenario {{product name}} pauses the next {{scenario singular lowercase}} run for 20 minutes {{product name}} reruns the incomplete execution with exponential backoff instant {{product name}} reruns the incomplete execution from its start with exponential backoff {{product name}} reruns the incomplete execution with exponential backoff the strategies to avoid rate limit errors depend on whether a {{scenario singular lowercase}} receives data instantly, with instant triggers, or on a scheduled basis instant scenarios in instantly triggered {{scenario plural lowercase}} , s et a scenario rate limit to control how often a {{scenario singular lowercase}} runs per minute this strategy spreads sudden spikes in requests over longer periods, enabling queued requests to be processed at a manageable rate you can configure a {{scenario singular lowercase}} rate limit in the maximum runs to start per minute field find this setting by clicking the lightning icon on the instant trigger module or schedule setting in the scenario toolbar you can also enable sequential processing or use sleep modules however, these strategies have limitations and are useful in select cases see fixing rate limit errors docid\ snkihipl3ukvwtrf1kqlo to learn more scheduled scenarios in scheduled scenarios (e g , with polling triggers ), you can try these strategies, depending on your use case set longer scheduling intervals when you expect many data records in a single run, prolong the time between {{scenario singular lowercase}} runs to spread incoming records over a longer period for example, schedule your {{scenario singular lowercase}} to run every 5 minutes and process 100 rows per run you can configure scheduling intervals in schedule settings in the scenario toolbar set a low number of records to process per run if you require frequent {{scenario singular lowercase}} runs, such as one run per minute, lower the number of records processed per run a limit of up to 20 records per run is recommended to avoid exceeding rate limits for example, schedule your {{scenario singular lowercase}} to run every minute and process 20 rows per run you can configure a records per run limit in a module's settings in the limit field both types regardless of the {{scenario singular lowercase}} scheduling type, you can do the following to reduce requests and prevent rate limit errors use bulk action modules, such as google sheets > bulk add rows and bulk update rows , to process multiple bundles in a single request for example, requesting once to google sheets to add 20 rows to a spreadsheet, rather than 20 times—once per row use aggregator modules to merge several bundles into a single bundle add a short delay by adding a fixing rate limit errors docid\ snkihipl3ukvwtrf1kqlo before an error causing module if rate limit errors continue for more strategies, continue to the next section advanced strategies this section includes more rate limit error strategies to try when the ones mentioned are less effective or relevant to your use case sequential processing when you enable sequential processing , the trigger module processes incoming data one by one, in the order they arrive, rather than in parallel {{product name}} waits until a {{scenario singular lowercase}} run is complete before starting the next one you can enable sequential processing in scenario settings in the scenario builder limitations sequential processing stops the {{scenario singular lowercase}} in the event of errors, even with incomplete executions disabled if a {{scenario singular lowercase}} has incomplete executions and sequential processing enabled, the {{scenario singular lowercase}} pauses until incomplete executions are processed to maintain the order of incoming data sleep module the sleep module delays requests to an app for up to 300 seconds, or 5 minutes set the delay to align with the number of requests allowed in the rate limit for example, if the app is limited to 10 requests per minute, set the delay to 6 seconds you can add a sleep module before the module receiving the delayed requests use sleep modules to address rate limit errors in these cases in instant {{scenario plural lowercase}} with multiple modules of the same app in {{scenario plural lowercase}} with modules that run multiple times to process many bundles (e g , 10) limitations apart from slowing down runtime, sleep modules often delay the issues causing rate limit errors rather than solve them for example, when requests that occur in rapid succession reach the sleep module, the module only delays these problematic requests for a couple of seconds incomplete executions when you enable incomplete executions , you can address errors later and keep {{scenario plural lowercase}} running if a rate limit error occurs with incomplete executions enabled, {{product name}} automatically reruns the incomplete execution in gradually increasing intervals in most cases, this automatic rerun solves the issue, and the {{scenario singular lowercase}} run continues as normal you'll only need to manually resolve errors in case of logical errors (e g , incorrect mapping, filter conditions, etc ) to enable incomplete executions, select yes in the allow storing of incomplete executions field in scenario settings an app counts the number of requests towards its rate limit in all {{scenario plural lowercase}} using the app if your {{scenario singular lowercase}} includes a module that consumes many operations, or you frequently use the same module in your {{scenario plural lowercase}} , consider checking the app's api rate limit or using the error handling strategies in this article to preemptively avoid frequent {{scenario singular lowercase}} runs