Gadget framework
Glossary
FAQ

Background actions 

Gadget allows you to offload time-consuming actions by running them asynchronously in the background. These actions are called background actions, and they ensures that critical operations, such as API calls, data processing, or batch updates, do not block the main execution flow. With built-in scheduling and execution management, Gadget handles these tasks efficiently, helping you maintain a responsive and optimized application. Both model-scoped and globally-scoped actions can be run as background actions.

a simple example of enqueueing a background action
JavaScript
await api.enqueue(api.someModelOrGlobalAction, {}, { queue: "my-queue" });
await api.enqueue(api.someModelOrGlobalAction, {}, { queue: "my-queue" });

When to use? 

Background actions are useful for long-running tasks that don't need to happen immediately. These actions are useful when:

1. Processing time-consuming operations

For tasks that take a considerable amount of time to complete, such as generating reports, processing images, or performing complex calculations, background actions allow these tasks to be offloaded from the main application flow.

2. Finicky actions

Retries are desirable for finicky actions which can throw errors at random times, such as an action involving an unreliable API. However, retries are only available on background jobs, so background actions are ideal for these calls.

3. Handling large quantities of external API calls

When interacting with external services or APIs, network latency and rate limiting can introduce delays. Background actions are ideal for managing these calls, especially when the outcome does not need to be immediately presented to the user. They also allow for efficient retry mechanisms in case of failures.

4. Batch processing

Performing operations on large datasets, like batch updates, exports, or imports, can be resource-intensive. By using background actions, these operations can be processed in smaller, manageable chunks without blocking any main functionality within your application.

Adding background actions 

To add a background action to your Gadget application, you need to define a model-scoped or globally-scoped action, and then enqueue it to run in the background. Background actions must have an API trigger.

Once you've enqueued your action, Gadget will run it as soon as it can in your serverless hosting environment. You can view the status and outcome of this background action in the Queues dashboard within Gadget.

Enqueuing a background action can be made from the backend or the frontend and is done with the api.enqueue function or with the useEnqueue React hook.

Enqueues take in three inputs:

  1. The action to enqueue.
  2. The input for the action, based on your action's needs.
  3. Options for the enqueue method.

Let's take a look at a couple of examples to use a background action both within the backend and frontend.

Any model-scoped or globally-scoped action can be enqueued in the backend using api.enqueue.

Enqueue a globally-scoped actions 

Let's say we have a globally-scoped action defined in api/actions/sendAnEmail.ts:

api/actions/sendAnEmail.js
JavaScript
export const run: ActionRun = async ({ params, emails }) => { await emails.sendMail({ to: params.to, subject: "Hello from Gadget!", // Email body html: "an example email", }); }; export const params = { to: { type: "string", }, };
export const run: ActionRun = async ({ params, emails }) => { await emails.sendMail({ to: params.to, subject: "Hello from Gadget!", // Email body html: "an example email", }); }; export const params = { to: { type: "string", }, };

You can run this action in the background by making an API call and enqueuing it:

run the action in the background
JavaScript
await api.enqueue(api.sendAnEmail, { to: "[email protected]" });
await api.enqueue(api.sendAnEmail, { to: "[email protected]" });

Enqueue a model-scoped action 

If instead, you had a model-scoped action, such as api/models/students/actions/sendData.ts:

api/models/students/actions/sendData.js
JavaScript
export const run: ActionRun = async ({ params, record, logger, api, connections, }) => { const res = await fetch("https://sample-api.com/api/v1/test", { method: "POST", headers: { "content-type": "application/json", }, body: JSON.stringify(record), }); if (res.ok) { const responseMessage = await response.text(); logger.info({ responseMessage }, "response from endpoint"); } else { throw new Error("Error occurred"); } };
export const run: ActionRun = async ({ params, record, logger, api, connections, }) => { const res = await fetch("https://sample-api.com/api/v1/test", { method: "POST", headers: { "content-type": "application/json", }, body: JSON.stringify(record), }); if (res.ok) { const responseMessage = await response.text(); logger.info({ responseMessage }, "response from endpoint"); } else { throw new Error("Error occurred"); } };

You can enqueue this action in the background in either a model-scoped or globally-scoped action.

api/models/school/actions/create.js
JavaScript
export const onSuccess: ActionOnSuccess = async ({ params, record, logger, api, connections, }) => { await api.enqueue(api.student.sendData, { id: params.studentId }); };
export const onSuccess: ActionOnSuccess = async ({ params, record, logger, api, connections, }) => { await api.enqueue(api.student.sendData, { id: params.studentId }); };

Enqueue in an HTTP routes 

While HTTP routes themselves cannot be enqueued, we can use them to enqueue model-scoped and globally-scoped actions:

api/routes/POST-send-student-data.js
JavaScript
import { RouteHandler } from "gadget-server"; const route: RouteHandler<{ Body: { studentId: string } }> = async ({ request, reply, api, logger, connections, }) => { const { studentId } = request.body; await api.enqueue(api.student.sendData, { id: studentId }); await reply.code(200).send(); }; export default route;
import { RouteHandler } from "gadget-server"; const route: RouteHandler<{ Body: { studentId: string } }> = async ({ request, reply, api, logger, connections, }) => { const { studentId } = request.body; await api.enqueue(api.student.sendData, { id: studentId }); await reply.code(200).send(); }; export default route;

Enqueue an action in the frontend 

Actions can be enqueued directly from your frontend code with the useEnqueue React hook.

React
export function UpdateUserProfileButton(props: { userId: string; newProfileData: { age: number; location: string } }) { // useEnqueue hook is used to enqueue an update action const [{ error, fetching, data, handle }, enqueue] = useEnqueue(api.user.update); const onClick = () => { return enqueue( // pass the params for the action as the first argument { id: props.userId, changes: props.newProfileData, }, // optionally pass options configuring the background action as the second argument { retries: 0, } ); }; // Render the button with feedback based on the enqueue state return ( <> {error && <>Failed to enqueue user update: {error.toString()}</>} {fetching && <>Enqueuing update action...</>} {data && <>Enqueued update action with background action id={handle.id}</>} <button onClick={onClick}>Update Profile</button> </> ); }
export function UpdateUserProfileButton(props: { userId: string; newProfileData: { age: number; location: string } }) { // useEnqueue hook is used to enqueue an update action const [{ error, fetching, data, handle }, enqueue] = useEnqueue(api.user.update); const onClick = () => { return enqueue( // pass the params for the action as the first argument { id: props.userId, changes: props.newProfileData, }, // optionally pass options configuring the background action as the second argument { retries: 0, } ); }; // Render the button with feedback based on the enqueue state return ( <> {error && <>Failed to enqueue user update: {error.toString()}</>} {fetching && <>Enqueuing update action...</>} {data && <>Enqueued update action with background action id={handle.id}</>} <button onClick={onClick}>Update Profile</button> </> ); }

Background action status 

Queues dashboard 

The Queues dashboard in Gadget provides real-time insights into the status and execution timeline of background actions. Developers can use this dashboard to track job progress, troubleshoot failures, and optimize task execution.

The status of your running action can be grouped into one of the below categories:

Scheduled: The job has been enqueued with a specific time to run.

Waiting: The job is ready for execution and will be carried out as soon as possible.

Running: The job is currently in progress.

Retrying: The job has failed at least once but still has remaining retry attempts.

Failed: The job has exhausted all retries and has failed permanently.

Complete: The job has successfully completed execution with no retries needed

Gadget offers a user-friendly interface (UI) that allows users to monitor and track the status of ongoing jobs. Through this interface, users can view real-time updates, check progress, and access detailed information about each job's current state.

A view of Gadget's queues dashboard where you can observe background action statusA view of a background actions timeline

Bulk enqueuing 

Gadget supports running many instances of the same action on a model with bulk actions.

Bulk actions are automatically created for model-scoped actions by Gadget. Unlike your CRUD actions, these are not visible in the model's actions folder. They can be run in the foreground, or enqueued to run in the background.

When to use bulk enqueueing 

Bulk enqueuing is most useful as a performance optimization to submit a large chunk of actions to your app all at once, in one HTTP call. It can be slow to enqueue jobs in a loop, so if you're experiencing slow speeds, switch to enqueuing in bulk.

JavaScript
const widgets = [{ name: "foo" }, { name: "bar" }, { name: "baz" }]; // slow version: this works, but runs in serial and can get slow due to the overhead of each API call to enqueue the action for (const widget of widgets) { await api.enqueue(api.widget.create, widget); } // fast version: this submits all your actions at once and will enqueue them all in parallel await api.enqueue(api.widget.bulkCreate, widgets);
const widgets = [{ name: "foo" }, { name: "bar" }, { name: "baz" }]; // slow version: this works, but runs in serial and can get slow due to the overhead of each API call to enqueue the action for (const widget of widgets) { await api.enqueue(api.widget.create, widget); } // fast version: this submits all your actions at once and will enqueue them all in parallel await api.enqueue(api.widget.bulkCreate, widgets);

Types of bulk actions 

Bulk actions can categorized under one of four types:

  • bulkCreate: use to create many records
  • bulkUpdate: use to update many records
  • bulkDelete: use to delete many records
  • bulkCustomAction: use for any custom model-scoped actions you define

For example, if you had a User model-scoped, you could use api.user.bulkCreate, api.user.bulkDelete, and api.user.bulkUpdate upon creation of the model. If you added a model-scoped action to User, such as incrementLoginCount, you could also call api.user.bulkIncrementLogInCount without having to create this action yourself.

Invoking a bulk action 

You can invoke an action in the foreground in bulk by calling the <model>.bulk<Action> function:

Create 3 widget records using bulkCreate
JavaScript
// create 3 widgets in bulk in the foreground const widgets = await api.widget.bulkCreate([ { name: "foo" }, { name: "bar" }, { name: "baz" }, ]);
// create 3 widgets in bulk in the foreground const widgets = await api.widget.bulkCreate([ { name: "foo" }, { name: "bar" }, { name: "baz" }, ]);

And you can enqueue an action in bulk by calling api.enqueue with the foreground bulk action function:

JavaScript
// create 3 widgets in bulk in the background const handles = await api.enqueue(api.widget.bulkCreate, [ { name: "foo" }, { name: "bar" }, { name: "baz" }, ]); // wait for the result of the first create action const widget = await handles[0].result();
// create 3 widgets in bulk in the background const handles = await api.enqueue(api.widget.bulkCreate, [ { name: "foo" }, { name: "bar" }, { name: "baz" }, ]); // wait for the result of the first create action const widget = await handles[0].result();

Each element of your bulk action will be enqueued as one individual background action with its own id, status, and retry schedule. In the above example, this means 3 different widget.create actions will be enqueued and executed. This means that some of the enqueued actions can succeed right away and others can fail and be retried.

Enqueuing actions in bulk returns an array of handle objects for working with the created background actions. If you want to wait for all the actions in your bulk enqueue to complete, you must await the result of each individual returned handle.

JavaScript
const handles = await api.enqueue(api.widget.bulkCreate, widgets); // wait for all enqueued actions to complete const widgets = await Promise.all( handles.map(async (handle) => await handle.result()) );
const handles = await api.enqueue(api.widget.bulkCreate, widgets); // wait for all enqueued actions to complete const widgets = await Promise.all( handles.map(async (handle) => await handle.result()) );

Similarly to foreground actions, each model's actions are all available in bulk in the foreground and background:

JavaScript
// update 2 widgets in bulk in the background const handles = await api.enqueue(api.widget.bulkUpdate, [ { id: 1, name: "bar" }, { id: 2, name: "baz" }, ]); // delete 3 widgets in bulk in the background const handles = await api.enqueue(api.widget.bulkDelete, ["1", "2", "3"]);
// update 2 widgets in bulk in the background const handles = await api.enqueue(api.widget.bulkUpdate, [ { id: 1, name: "bar" }, { id: 2, name: "baz" }, ]); // delete 3 widgets in bulk in the background const handles = await api.enqueue(api.widget.bulkDelete, ["1", "2", "3"]);

Bulk background action options 

Like individual background actions, enqueuing bulk actions supports the full list of background action options: queue, id, startAt, and retries. Each of these options will apply to each individually enqueued background action, and each action's options won't affect the others.

For retries, each enqueued background action from the bulk set will get its own set of retries according to the options you specify.

JavaScript
// bulk create some widgets, and retry each widget create action up to 3 times const handles = await api.enqueue(api.widget.bulkCreate, widgets, { retries: 3 });
// bulk create some widgets, and retry each widget create action up to 3 times const handles = await api.enqueue(api.widget.bulkCreate, widgets, { retries: 3 });

For queue, each enqueued background action from the bulk set will be added to the same concurrency queue, and obey the maximum concurrency you specify. The concurrency limit applies to each individually enqueued background action, not the bulk set as a whole.

JavaScript
// bulk create some widgets in the `user-10` concurrency queue, and create at most 1 widget at a time const handles = await api.enqueue(api.widget.bulkCreate, widgets, { queue: { name: "user-10", maxConcurrency: 1 }, });
// bulk create some widgets in the `user-10` concurrency queue, and create at most 1 widget at a time const handles = await api.enqueue(api.widget.bulkCreate, widgets, { queue: { name: "user-10", maxConcurrency: 1 }, });

For id, the passed id option is suffixed with the index of each enqueued background action to create unique identifiers for each. For example, if you enqueue 3 create widget actions, you'll get back 3 handles, each with a unique ID:

JavaScript
// bulk create some widgets in the `user-10` concurrency queue, and create at most 1 widget at a time const handles = await api.enqueue( api.widget.bulkCreate, [{ name: "foo" }, { name: "bar" }, { name: "baz" }], { id: "test-action", } ); // => test-action-0 handles[0].id; // => test-action-1 handles[1].id; // => test-action-2 handles[2].id;
// bulk create some widgets in the `user-10` concurrency queue, and create at most 1 widget at a time const handles = await api.enqueue( api.widget.bulkCreate, [{ name: "foo" }, { name: "bar" }, { name: "baz" }], { id: "test-action", } ); // => test-action-0 handles[0].id; // => test-action-1 handles[1].id; // => test-action-2 handles[2].id;

Gadget always appends the index of the item submitted with a bulk action to ensure action id uniqueness.

Customizing background actions 

Background actions accept different parameters than traditional actions, which allows for scheduling and retrying. The results of a background action may also be awaited to ensure the results are received.

Scheduling a background action 

To schedule a background action, you can pass the startAt option when enqueueing an action. This allows you to specify when an action is to be executed in the background. The startAt option must pass the scheduled time formatted as an ISO string.

Here's a basic example, we can directly input the ISO string like this:

Using a valid ISO 8601 string to schedule the `change` globally-scoped action
JavaScript
export const run: ActionRun = async ({ record }) => { await api.enqueue( api.change, {}, // Scheduled to start at noon on April 3, 2024, UTC { startAt: "2024-04-03T12:00:00.000Z" } ); };
export const run: ActionRun = async ({ record }) => { await api.enqueue( api.change, {}, // Scheduled to start at noon on April 3, 2024, UTC { startAt: "2024-04-03T12:00:00.000Z" } ); };

Creating a Date with newDate():

Using a JS Date to schedule the `change` globally-scoped action
JavaScript
export const run: ActionRun = async ({ record }) => { await api.enqueue( api.change, {}, // remember to format the created date as an ISO String like below { // Starts in 10 minutes startAt: new Date(Date.now() + 1000 * 60 * 10).toISOString(), } ); };
export const run: ActionRun = async ({ record }) => { await api.enqueue( api.change, {}, // remember to format the created date as an ISO String like below { // Starts in 10 minutes startAt: new Date(Date.now() + 1000 * 60 * 10).toISOString(), } ); };

Retrying on Failure 

Retries are crucial for handling failures in background actions, especially for unreliable operations like network requests.

By default, background actions are retried up to 6 times using an exponential back-off strategy. This strategy increases the delay between retries, allowing intermittent issues to be resolved. The retry behavior can be customized by specifying the number of retries and the delay strategy when enqueueing an action.

For more details on configuring retries, refer to the Retry Reference Documentation.

You can configure retries in two ways:

  • As an integer: Specifies the number of retry attempts using the default strategy.
  • As an object: Allows full control over retry settings, such as delay intervals and backoff factors.

Here are some example of using retries:

If you do not want the background action to retry upon failure, set retries to 0.

No retries allowed
JavaScript
await api.enqueue(api.publish, { type: "article", pages: 10 }, { retries: 0 });
await api.enqueue(api.publish, { type: "article", pages: 10 }, { retries: 0 });

To allow a single retry attempt in case of failure, set retries to 1.

1 retry allowed
JavaScript
await api.enqueue(api.publish, { type: "article", pages: 10 }, { retries: 1 });
await api.enqueue(api.publish, { type: "article", pages: 10 }, { retries: 1 });

If you want the action to retry once but introduce a delay of 30 minutes before retrying, specify retryCount as 1 and initialInterval as 30 * 60 * 1000 milliseconds.

1 retry allowed, with a delay of 30 minutes
JavaScript
await api.enqueue( api.publish, { type: "article", pages: 10 }, { retries: { retryCount: 1, initialInterval: 30 * 60 * 1000 } } );
await api.enqueue( api.publish, { type: "article", pages: 10 }, { retries: { retryCount: 1, initialInterval: 30 * 60 * 1000 } } );

For more granular control, you can specify additional retry settings:

5 retries allowed with different retry settings specified
JavaScript
/** @type { ActionRun } */ export const run = async ({ api }) => { await api.enqueue( api.someModelOrGlobalAction, { foo: "foo", bar: 10 }, { retries: { retryCount: 5, // Retry up to 5 times maxInterval: 60000, // Max delay between retries: 60s backoffFactor: 2, // Doubles delay for each retry initialInterval: 1000, // Start with a 1s delay randomizeInterval: true, // Randomizes retry delay }, // OR simply retries: 5 } ); };
/** @type { ActionRun } */ export const run = async ({ api }) => { await api.enqueue( api.someModelOrGlobalAction, { foo: "foo", bar: 10 }, { retries: { retryCount: 5, // Retry up to 5 times maxInterval: 60000, // Max delay between retries: 60s backoffFactor: 2, // Doubles delay for each retry initialInterval: 1000, // Start with a 1s delay randomizeInterval: true, // Randomizes retry delay }, // OR simply retries: 5 } ); };

Retry Configuration Options

  • retryCount: Maximum number of retries before failure. (Default: 6)
  • maxInterval: Maximum delay for retries when using exponential backoff. (Default: none, meaning it can back off indefinitely)
  • backoffFactor: Multiplier for exponentially increasing delay between retries. (Default: 2)
  • initialInterval: Time (in ms) before the first retry attempt. (Default: 1000 ms)
  • randomizeInterval: Randomizes retry delays by applying a multiplier between 1 and 2. (Default: false)

For more details, visit the Retry Reference Documentation.

Awaiting the result 

You can retrieve the final result or error of a background action on both the client and server sides using the BackgroundActionHandle object. This handle is returned by api.enqueue calls and can also be generated later if you have the background action's ID.

For example, you can enqueue a publish action and then await the result:

JavaScript
// enqueue an action and get a handle back const handle = await api.enqueue(api.publish, {}); // await the result of the action const result = await handle.result();
// enqueue an action and get a handle back const handle = await api.enqueue(api.publish, {}); // await the result of the action const result = await handle.result();

If the background action succeeds, .result() returns the result. If it fails all retries, it throws the final error. If some attempts fail but the action eventually succeeds, .result() returns the successful attempt's result.

Considerations for await handle.result() 

  • Since await handle.result() waits for completion, it can take time depending on retry schedules.

  • If an action has slow retries, it may take hours or even days to return a result.

  • If the retry schedule is fast or retries are limited, the result will return quickly.

Subscribing to background action results is supported client-side using the api object.

Selecting properties of result 

If you've enqueued a model-scoped action in the background, you can select the fields you desire, including related fields, when accessing the action's results.

You can pass the select action to handle.result() to retrieve specific fields of the record returned from a background action:

select specific fields of a background action result
JavaScript
const handle = await api.enqueue(api.widget.update, { id: 1, name: "foo" }); const record = await handle.result({ select: { id: true, name: true, gizmos: { edges: { node: { id: true } } }, }, }); // record.id will have the resulting record id // record.gizmos will have the record's gizmos relationship loaded
const handle = await api.enqueue(api.widget.update, { id: 1, name: "foo" }); const record = await handle.result({ select: { id: true, name: true, gizmos: { edges: { node: { id: true } } }, }, }); // record.id will have the resulting record id // record.gizmos will have the record's gizmos relationship loaded

Fetching by ID 

You can create a BackgroundActionHandle object anywhere you have an api object if you know the action that was invoked and the id of the enqueued background action.

For example, you can get a handle for a background action that was enqueued in the past like this:

JavaScript
// get a handle for a background action that was enqueued in the past const handle = api.handle(api.publish, "app-job-12345"); // await the action succeeding and then retrieve result of the action const result = await handle.result();
// get a handle for a background action that was enqueued in the past const handle = api.handle(api.publish, "app-job-12345"); // await the action succeeding and then retrieve result of the action const result = await handle.result();

If you know you're likely to need an action's result in the future, a common pattern is to store the enqueued action id on a record of a model. For example, we can store the id of the background action on a widget record like this:

api/models/widget/actions/create.js
JavaScript
export const onSuccess: ActionOnSuccess = async ({ params, record, logger, api, connections, }) => { const handle = await api.enqueue(api.publish, { id: params.id }); // record the enqueued action id on the widget record.backgroundActionId = handle.id; await save(record); };
export const onSuccess: ActionOnSuccess = async ({ params, record, logger, api, connections, }) => { const handle = await api.enqueue(api.publish, { id: params.id }); // record the enqueued action id on the widget record.backgroundActionId = handle.id; await save(record); };

Then, later, you can use this stored id to boot up a handle, say on the frontend:

web/some/component.js
JavaScript
const widget = await api.widget.findFirst(); // get a handle for the background action const handle = api.handle(api.publish, widget.backgroundActionId); // await the result of the action const result = await handle.result();
const widget = await api.widget.findFirst(); // get a handle for the background action const handle = api.handle(api.publish, widget.backgroundActionId); // await the result of the action const result = await handle.result();

Performance and billing 

Accessing the result of a background handle uses an efficient websocket based subscription protocol to await results from the backend. Subscribing to a handle result doesn't incur charges for simply subscribing to the result, but if you subscribe within a server-side request or action, that server-side request will be charged the normal usage rates. For example, if you await handle.result() within the frontend of your application, you won't be charged any extra request time for opening the websocket and listening until the action has completed. But, if you await handle.result() in a global action, you'll be billed for the time that global action ran.

Permissions 

To access the result of a background action, the requesting api client must have the same identity as the client that ran the action itself. This means that for example, unauthenticated users cannot access the results of actions enqueued by other unauthenticated users or by authenticated users.

If you want to make the result of a background action accessible by more users, you can store the result on a data model, and use the normal access control system to grant access to that data model.

Queuing and concurrency control 

To manage the execution of this background action with specific concurrency limits, it can be placed in a dedicated queue. By default, actions are added to a globally-scoped queue with no concurrency restrictions.

For targeted control:

Single concurrency: To place the action in a named queue that allows only one action to run at a time, simply provide the name of the queue as a string. This implicitly sets the maximum concurrency to 1.

JavaScript
await api.enqueue( api.publish, {}, // setting a queue with default max concurrency set to 1 { queue: { name: "dedicated-queue" } } );
await api.enqueue( api.publish, {}, // setting a queue with default max concurrency set to 1 { queue: { name: "dedicated-queue" } } );

Custom concurrency: To define a custom concurrency level, pass an object with two properties: the queue's name (string) and the desired maxConcurrency (number). This enables precise control over how many instances of the action can be executed simultaneously within the specified queue.

JavaScript
await api.enqueue( api.publish, {}, // setting a queue with custom max concurrency { queue: { name: "dedicated-queue", maxConcurrency: 4 } } );
await api.enqueue( api.publish, {}, // setting a queue with custom max concurrency { queue: { name: "dedicated-queue", maxConcurrency: 4 } } );

When to apply concurrency control 

Consider using targeted concurrency control when your application requires some of the following criteria:

  1. Sequential processing: Consider a scenario where a user accidentally triggers the same action twice, such as upgrading a plan. Without proper concurrency management, both actions could proceed simultaneously, potentially resulting in double charges or duplicate operations. Sequential processing ensures that such operations are executed one after the other, eliminating the risk of duplication.

  2. Resource-specific concurrency: In many cases, operations need to be serialized per resource rather than application-wide. For example, while it's necessary to prevent simultaneous upgrades for a single customer's account, different customers should be able to upgrade their plans concurrently without waiting for each other. This approach ensures efficiency and isolation, preventing one customer's actions from impacting another's.

  3. Interacting with rate-limited systems: When your application interacts with external systems that have rate limits (e.g., APIs), managing concurrency becomes crucial to avoid exceeding these limits. Properly configured concurrency settings ensure that your application respects these limits, maintaining a good standing with the service providers and ensuring reliable integration.

Handling API rate limits 

You can effectively manage rate limits by configuring the maxConcurrency when enqueueing an action. Adjusting this will control the number of background actions that can run simultaneously, indirectly influencing the request rate.

To align with a third-party API's rate limit, you must calculate the appropriate maxConcurrency based on the execution time and the rate limit itself.

Formula to calculate needed concurrency

maxConcurrency = Rate limit (actions/requests per second) × Average action execution time (in seconds)

Best practice

  1. Determine the average execution time of your action (e.g., 200ms).

  2. Calculate the maximum allowable actions per second based on the third-party's rate limit (e.g., 5 requests per second).

  3. Adjust maxConcurrency to ensure that executing jobs concurrently will not exceed the calculated rate.

For example, let's say an action takes about 1 second to execute, with an API rate limit of 5 requests per second, you would calculate the maxConcurrency to be 5. This setting is optimal because it aligns precisely with the rate limit, ensuring you fully utilize the available capacity without the risk of exceeding the limit.

But in another case, imagine a scenario where each action your system performs completes in approximately 500 milliseconds or 0.5 seconds. If the API you're integrating with allows 5 requests per second, you'd calculate your maxConcurrency as 2.5. To avoid surpassing the rate limit, you round down to 2. This adjustment ensures you're using the API efficiently while staying within allowed usage boundaries.

Background action limits 

There are some concurrency limitations to background actions that developers may need to be aware of:

  • Each queue has an upper limit maxConcurrency value of 100 . This limit can be raised by contacting Gadget support.
  • Each action can have a maximum payload size of 100MB when serialized using msgpack. This limit cannot be raised. If you need to store large data blobs for processing with background actions, you can use the file field to store unlimited size data in cloud storage, and then download it within your action.

Was this page helpful?