Asynchronous Processing

Overview

When building data processes and integrations, you have a few options. Depending on the goal of your application, you might need these to work differently. For example, perhaps you want all your data processed at once. Or, you might need your data processed in a certain order. If that's the case, you'll use synchronous processing. Synchronous configurations work like dominoes, each process triggering after the prior process ends.

But this strict order isn't always necessary when processing data or submissions. For example, you might start processing one group of data, then start processing another before the first finishes. Here, you don't require one process to finish before starting another. This is asynchronous processing.

What You'll Learn

In this article, you'll learn:

What is Asynchronous Processing?

Asynchronous processing doesn't require one process to finish before starting the next. This is useful if your application has large amounts of data to process quickly. It's also helpful if your application receives submissions on a continuous basis. In this case, you often can't wait for all data to be received before starting a process. Or the results of one process might not impact the results of another. Because of that, you don't need to run them in any particular order. In these situations, asynchronous processing can boost speed and efficiency in an application.

How Does Asynchronous Processing Work?

This is an overview of how asynchronous processing works. Think of this like a looping Data Workflow. Once you have the looping architecture, you can add the details your process requires.

The easiest way to explain how asynchronous processing works is to look at an example. So, say your application manages food delivery orders, processing hundreds every hour. Each order must be assigned to a delivery driver, but you can't perform that task once an hour for a few reasons. First, food delivery is far too time-sensitive to process orders hourly. Second, you'd never get ahead on the orders you need to fulfill. And finally, the number of orders you receive exceeds the number you can process at once anyway. (Keep in mind, API (application programming interface) calls and integrations limit the amount of data they can process and the amount of time they can spend processing that data.) Instead, you'll process orders more often: once every minute.

In this example, your orders are all going through the same process and the orders aren't related to each other. So, you don't need one set of order assignments to finish before the next set begins. That makes an asynchronous configuration the best solution.

With asynchronous processes, you must tell your application which submissions are in-progress. This ensures no data is processed twice. In the above example, you could assign the same order to 2 delivery drivers if the same data is processed twice. So, you must mark each order as either Needs Assignment or Doesn't Need Assignment. You'll build this directly into your process. That way, if a new process starts before the first finishes, orders from the first aren't in the second.

1. Retrieve orders that need processing.
2. Mark the retrieved orders as in-progress.
3. Process the retrieved orders.
4. Mark orders as resolved once assigned to a driver.