Scenario execution, cycles, and phases

{{product name}} is a transactional system, similar to relational databases , where all operations within a scenario are executed as part of the controlled process each {{scenario singular lowercase}} execution goes through the following phases i nitialization one or more cycles finalization initialization phase during the initialization phase , all necessary connections are created and verified {{product name}} also checks whether each module can perform its intended operation(s) cycles the initialization phase is followed by at least one cycle, which consists of o peration phase (reading or writing) c ommit/rollback phases (finalizing or discarding changes) each cycle represents an indivisible unit of work composed of a series of operations it is possible to set the maximum number of cycles in the scenario settings the default number is 1 operation phase during the operation phase, {{product name}} performs reading and writing operations r eading operations consist of obtaining data from a service that will then be processed by other modules according to a predefined {{scenario singular lowercase}} for example, the dropbox > watch files module returns new bundles (files) created since the last {{scenario singular lowercase}} execution w riting operations consist of sending data to a given service for further processing for example, the dropbox > upload a file module uploads a file to a dropbox folder commit and rollback phases after the operation phase, {{product name}} either commits or rolls back the operations if operations are successful for all modules, the commit phase begins during this phase, all operations performed by the modules are committed {{product name}} sends information to all the services involved in the operation phase about its success if an error occurs during the operation or commit phase for any module, the phase is aborted and the rollback phase is started, making all operations during the given cycle void not all modules allow for rollback modules that support rollback are marked with the acid tag modules without this tag do not support rollback and cannot be reverted back to their initial state in case of an error in other modules in the context of rollbacks, setting the maximum number of cycles can be particularly useful when processing multiple bundles from a trigger in a single run for example, consider a scenario where you use a polling trigger to check for new data you don’t need the scenario to run every minute, so you schedule it to run once per hour there are multiple bundles to process in one execution in this case, you have two options increase the trigger limit in the module settings raise the maximum number of results the trigger retrieves per cycle (e g , from 2 to 10) the scenario will still use a single cycle, but process more data in one go increase the maximum number of cycles in the scenario settings keep the trigger limit low (e g , 1), but configure the scenario to run in multiple cycles (e g , from 1 to 10) if an error occurs while processing the 10th bundle, for example with the first option (one cycle, higher trigger limits), the entire cycle is rolled back none of the processed bundles are saved, and everything has to be reprocessed in the next run with the second option (multiple cycles, lower trigger limits), only that last cycle (the failed bundle) is rolled back since previous cycles are already committed, only that failed bundle has to be reprocessed in the next run this approach can save time and prevent unnecessary reprocessing of successfully completed work f inalization phase during the finalization phase , open connections (e g , ftp connections, database connections, etc ) are closed and the {{scenario singular lowercase}} execution is completed example transfer of bundles between databases the following example shows how to connect three acid modules the aim of the {{scenario singular lowercase}} is to get new rows from a mysql database, insert (transfer) them into an mssql database, and then insert the ids of the rows from the mssql database into a postgresql database when the {{scenario singular lowercase}} starts, the initialization phase is performed first {{product name}} verifies connections to the mysql, mssql and postgresql databases one at a time if everything goes well and the connections are successful, {{product name}} moves on to the operation phase if an error occurs, the finalization phase starts instead of the operation phase and the {{scenario singular lowercase}} is terminated if there isn't an error, the operation phase begins a preset procedure selects (reads) the table rows (bundles) from mysql those rows are then passed to the next module that writes them to a selected table in the mssql database if everything is in order, the last postgresql procedure is called to insert the row ids returned by the preceding module into the table if the operation phase is completed successfully, the commit phase begins {{product name}} calls the sql commit command for each database, and the write operations are committed however, if the operation or commit phase fails due to an error (e g , connection failure), {{product name}} calls for a rollback during the rollback phase , {{product name}} goes through all modules and executes the sql rollback command for each module to revert each database back to its initial state finally, during the finalization phase , each module closes its connection to the database