Working with BigCommerce data 

You will often want to store BigCommerce data in your Gadget database after subscribing to BigCommerce webhooks.

This guide will show you how to read and write data to BigCommerce using Gadget, and how you can store BigCommerce data in your Gadget database.

Calling the BigCommerce API 

Before you can store data, you need to fetch it from BigCommerce! BigCommerce webhooks often only provide a resource ID, so you will need to use the BigCommerce API to fetch the full resource data.

The best way to call the BigCommerce API is to use the included BigCommerce API client.

Here's a simple example of using the API client to read products from BigCommerce in a global action.

You can also use the API client to write data back to a BigCommerce store:

Store context for API client 

The connections.bigcommerce object provides access to a BigCommerce API client. You can use this client to read and write data to BigCommerce.

Depending on how an action was triggered, the current store context may be available on the connections object. For example, if you trigger a global action from a BigCommerce webhook, or call an action from a single-click frontend, the current store context will be available with connections.bigcommerce.current:

If the current store context is not available, such as calling an action from another action in Gadget, you can use the forStoreHash method to get the API client for a specific store hash:

This is common when calling model actions from webhook-triggered global actions.

Read more about connections.bigcommerce in the gadget-server reference.

Storing data in Gadget 

You can store data read from BigCommerce in Gadget data models. This allows you to access all data required to power your application in a single, highly available and performant, non-rate limited database.

When you set up your BigCommerce connection, a store data model was created at api/models/bigcommerce/store. You can also add custom models to the bigcommerce namespace.

To add a custom BigCommerce data model:

1. Right-click on the api/models/bigcommerce directory in the Gadget editor and select Add model
2. Give your model a name, for example product or order
3. Add a field to store the BigCommerce id
4. Add a relationship field between the new model and the bigcommerce/store model so that bigcommerce/store has many bigcommerce/newModel
5. Add fields to store the rest of the BigCommerce data you need to power your app

You can use the auto-generated CRUD (create, read, update, delete) API to interact with your new model.

For example, you can call a bigcommerce/product model's create action to store a new record in Gadget:

Storing BigCommerce IDs 

It is important to store BigCommerce resource IDs in Gadget! This is the unique identifier for the record in BigCommerce, and is required to handle update or delete webhooks, and associate data stored in Gadget with BigCommerce records.

It's also a good idea to add both the Required and Uniqueness validations to the bigcommerceId field on your models! For multi-tenant applications, you should also scope this uniqueness validation to the store relationship so that each store can only have one record with a given bigcommerceId.

Upserting data and handling /updated webhooks 

When you are storing data from BigCommerce in Gadget, you may want to update existing records if they already exist, or create a new record if it doesn't. This is known as upserting data. This is particularly useful when handling /updated webhooks, where a record may or may not already exist in your Gadget database.

Your Gadget API provides an upsert meta action that manages the upsert logic for you. For BigCommerce data, you often want to upsert based on the bigcommerceId field. For multi-tenant applications, you should also scope the upsert to the store relationship. This ensures that you only update or create records for the current store.

Here is an example of how you could upsert product data in Gadget when a store/product/updated webhook is received:

This will check to see if a product with the given bigcommerceId and store relationship already exists in Gadget. If it does, it will update the existing record. If it doesn't, it will create a new record.

Storing data from child models 

You can store data from child models in Gadget by adding relationships between models. When you create a new model, you can add a relationship field to associate the new model with the parent model.

An example of this relationship is products and product variants. You can create a bigcommerce/productVariant model and associate it with the bigcommerce/product model using a relationship field.

Whether you are syncing or creating data from a webhook, you can include the required child data when reading from BigCommerce so it is returned in the same payload.

You also have the option to make a separate API call to retrieve the child data and store it in Gadget. Refer to the BigCommerce API reference for more information on retrieving child data.

Once you have the child data, you can store it in the child model using the CRUD API.

For example, here is how you could store product variant data in Gadget when a store/product/created webhook is received:

Syncing all existing data 

If you need to sync all existing data from a BigCommerce store, you can use the BigCommerce API client to fetch all data and store it in Gadget. There are a couple of things you need to be aware of and handle properly, including the BigCommerce API rate limits and max page size for fetching data.

Gadget's built-in background actions queue can help you manage this process. Background actions allow developers to set custom concurrency controls to adhere to a rate limit, and have built-in retry logic for failed actions, ensuring that all data is synced.

To maximize the speed of the sync, you can use bulk actions to enqueue multiple actions at once. For a data sync, you will often want to use Gadget's upsert meta API so that you can update existing records and create new records in a single action.

Here is an example of how you could sync all product data from a BigCommerce store:

This will sync all products from a BigCommerce store to Gadget. It may take some time before all records are synced!

If you need to sync on a regular schedule, you can use a scheduled action to run the sync at a set interval.

Handle BigCommerce rate limits 

Rate limits in BigCommerce are per store and shared across all apps installed on a store. This means that 429 response codes must be handled properly.

The included BigCommerce API client will automatically handle rate limits for you and will retry requests that return 429 or 5XX response codes using an exponential backoff.

Gadget's background actions can also be used to manage rate limits. You can set the concurrency of a background action to match the rate limit of the BigCommerce API and use the built-in retry logic to handle rate limit errors.

To do this, use the maxConcurrency option in the enqueue function to limit how many actions are run concurrently.

For example, to create new products in BigCommerce, you would run the following action in a background action:

And then enqueue the action with a maxConcurrency option:

Working with BigCommerce metafields 

Similar to any other resource in BigCommerce, you can read and write metafields using the BigCommerce API client. You might wish to store metafield data in line with the resource data in Gadget.

There are two ways to store metafield data next to your model data:

1. Subscribe to metafield webhooks in a global action and update the model data in Gadget when a metafield is created, updated, or deleted. Resources that can store metafield data have their own metafield webhook topics, so you can subscribe to these topics and update your model data accordingly.

2. Fetch metafield data when you fetch the resource data and store it in the model. This would be required when syncing data into your Gadget database and the code would look similar to the sync example.

Subscribe to metafield webhooks 

Metafield payloads always have a metafield_id and a resource_id field, which you can use to update the correct record in your model:

json
Copy
{
  "metafield_id": 2,
  "namespace": "Sales Department",
  "resource_id": "133",
  "storeHash": "bbcolvaxts"
}

Both the resource_id and metafield_id should be stored in your model to be able to update and delete metafields.

Here is an example of a store/product/metafield/created webhook subscription that updates the bigcommerce/product model when a metafield is created for a product. The product model stores resource_id as bigcommerceId and metafield_id as metafieldId.

You can also use the namespace and key to identify the metafield, which is useful when you have multiple metafields for a resource.

Data security and multi-tenancy 

If you are building a public app, you will need to ensure that data is stored securely and that one store's data is not accessible by another store. Gadget provides tools that help manage multi-tenancy and row-level security (RLS) for models and actions.

Securing models and model actions 

Gadget provides a set of data access tools that can be used to secure models and their actions. These tools include:

• relationship fields on models so that you can associate data with a store record
• a Uniqueness validation to make sure that each store has only one record with a given ID
• built-in functions for preventing cross-shop data access in model actions
• filtered model permissions that restrict access based on the related store

Follow these steps to set up these tools and enforce multi-tenancy on a model:

1. If you haven't already done so, add a relationship field to any custom model that associates the model with a store record

For example, add a belongs to relationship field on the bigcommerce/product model. The field should be named store and should reference the bigcommerce/store model, so that product belongsTo store. Then define the inverse of the relationship so that store hasMany products.

2. Add a Uniqueness validation to the BigCommerce resource ID field on your model and scope it to the store relationship. This is important to ensure that stored IDs are unique per store. See storing BigCommerce IDs for more information.

3. In your custom model, import preventCrossStoreDataAccess from gadget-server/bigcommerce and use it in your model actions

For example, in the bigcommerce/product model, you can call preventCrossStoreDataAccess in the run function of the create action to ensure that a product can only be created for the current store:

Each time preventCrossStoreDataAccess runs, it will make sure the given record has the correct storeId for the store processing this action. For existing records, that means it will verify that the storeId of the saved record the matches the storeId on the incoming record.

4. Add a tenancy filter to the read action for the custom model in accessControl/permissions.

Tenancy filters are expressed with Gelly, Gadget's data access language. These filters check the store ID of the current session against the ID of the store that the data being read belongs to.

Most tenancy filters for BigCommerce look similar to this:

accessControl/filters/bigcommerce/product.gelly
gelly
Copy
filter ($session: Session) on BigCommerceStore [
  where storeId == $session.bigcommerceStoreId
]

Securing global actions and HTTP routes 

When using global actions or HTTP routes to read or write data, you cannot use Gelly tenancy filters or functions like preventCrossStoreDataAccess because there is no backing model or relationship to a store record.

For global actions or HTTP routes, you often need to filter data based on the bigcommerceStoreId, which is included as a properly on the connections parameter. Filtering is built into your app's API and available on most field types. Docs for filtering are available in the API reference.

For example, if I am fetching bigcommerce/order data in a global action before doing some processing and returning the results, I would do the following:

This will only work when there is an authenticated session!

Otherwise, you will need to manually fetch the store ID from bigcommerce/store using the store hash:

The same pattern can be used for HTTP routes, where you would fetch the store ID from the bigcommerce/store model and then filter the results based on the store ID.