Gadget framework
Changelog
Glossary
FAQ

API access to data 

Gadget automatically generates and documents API endpoints that allow you to interact with your data. The API is how you fetch information for arbitrary queries (e.g. a query that is powered by the user's input on the frontend) or simple, predefined queries. For handling data in aggregates, or complex predefined queries, we recommend using computed fields.

Each time you add a new model to your project, Gadget instantly generates a public and private API to interact with the records stored in that table:

  • The public API is how your production application runs its business logic. The public API is described in actions that run both the database work we want our backend to do, along with any additional logic you specify. For example, a blog might have a createBlog action that persists blog data to the Gadget database before running a function to send a success email to the author.

  • The internal API is how you quickly interact with data (create, read, update, delete) at the database level, without running any additional business logic.

Your production application's frontend will typically make API calls to your Gadget backend via your public API, whereas your project's internal API is primarily used to fetch information to run the server side functions that power your actions.

Learn more about when to use the public API vs the internal API.

Data aggregation using the API 

Gadget supports several styles of data aggregation, including:

  • Computed fields, which store one pre-aggregated value per record of a model
  • Computed views beta, which compute aggregate values across many records and/or models
  • Client side pagination, which allows you to paginate through a large number of records to compute an aggregate in JavaScript

Computed views are the recommended approach to handling predefined aggregate queries (e.g. counts and sums) in Gadget. They can take in dynamic variables for controlling filters, sorts, aggregations, and pagination, and are executed server-side within the database for the best performance.

If you want to run more complicated business logic in JavaScript, or process streams of data, you can also use client-side aggregation by reading pages of data from the API and producing your own aggregate values. Client side aggregation is much slower than computed views or computed fields, so use it as a last resort.

Client-side aggregation with pagination 

To produce read-time aggregates with dynamic filtering, you can fetch the desired records and paginate through them to get the desired aggregate data. This allows you to apply any filters you want to the GraphQL query and ensures you get an accurate count.

The page size is controlled using the first or last GraphQL field arguments.

Limitations 

Paginating through all records client side performs poorly. The maximum page size for a paginated read in Gadget is 250 records, so if you need to aggregate 100000 records, walking through them will require fetching 400 pages of data.

When to paginate 

  • You are confident that you won't have a large number of total records in your collection
  • You have dynamic filtering options that don't allow for counts to be pre-calculated

When to avoid pagination 

  • You may need to count a large number of records
  • You don't need dynamic options for filtering your counts

For instance, if you want to get the count of all products in a Shopify store:

JavaScript
export const run: ActionRun = async ({ api, record, params, logger }) => { const shopId = "my shop id"; const productCount = 0; // api is an instance of my Gadget API client let products = await api.shopifyProduct.findMany({ // use the maximum page size first: 250, // select the minimum amount of data from the model as needed select: { id: true, }, // filter by the shop id // adding more filter conditions is usually a good idea, let the database handle filtering for you! filter: { shopId: { equals: shopId, }, }, }); productCount += products.length; // paginate through all our products, 250 at a time while (products.hasNextPage) { products = await products.nextPage(); productCount += products.length; } logger.info( { productCount }, "contains the total number of products for a given store" ); };
export const run: ActionRun = async ({ api, record, params, logger }) => { const shopId = "my shop id"; const productCount = 0; // api is an instance of my Gadget API client let products = await api.shopifyProduct.findMany({ // use the maximum page size first: 250, // select the minimum amount of data from the model as needed select: { id: true, }, // filter by the shop id // adding more filter conditions is usually a good idea, let the database handle filtering for you! filter: { shopId: { equals: shopId, }, }, }); productCount += products.length; // paginate through all our products, 250 at a time while (products.hasNextPage) { products = await products.nextPage(); productCount += products.length; } logger.info( { productCount }, "contains the total number of products for a given store" ); };

Paginating nested relationships 

When you need to paginate through nested data relationships, efficiently utilize cursors to navigate from one model of related records to the next, ensuring seamless and performance-optimized data retrieval.

For instance, let's take a look at the example below: when fetching a class record with paginated student data, then sequentially retrieving further student records, enabling efficient navigation through large sets of nested data.

JavaScript
const classRecord = await api.class.findFirst({ select: { id: true, students: { pageInfo: { hasNextPage: true, endCursor: true, }, edges: { node: { id: true, }, }, }, }, }); // Using the cursor to define the end of retrieval of records within the class model so we can go and fetch from the student model const nextPageOfStudents = await api.student.findMany({ first: 50, after: classRecord.students.pageInfo.endCursor, select: { id: true }, }); // We now define that after the end of the retrieval we go right back to the next 50 results within the class model const nextNextPageOfStudents = await api.student.findMany({ first: 50, after: nextPageOfStudents.endCursor, select: { id: true }, });
const classRecord = await api.class.findFirst({ select: { id: true, students: { pageInfo: { hasNextPage: true, endCursor: true, }, edges: { node: { id: true, }, }, }, }, }); // Using the cursor to define the end of retrieval of records within the class model so we can go and fetch from the student model const nextPageOfStudents = await api.student.findMany({ first: 50, after: classRecord.students.pageInfo.endCursor, select: { id: true }, }); // We now define that after the end of the retrieval we go right back to the next 50 results within the class model const nextNextPageOfStudents = await api.student.findMany({ first: 50, after: nextPageOfStudents.endCursor, select: { id: true }, });

Pre-aggregating data 

When possible, Gadget recommends using computed views or computed fields to aggregate data. However, there are a few cases where you can consider pre-aggregating data at write time:

  1. Maximum performance is necessary: Computed views and computed fields are executed server-side within the database, so they are faster than client-side aggregation, but they re-compute the aggregate value each time they are read. If you need to compute an aggregate value that is read very frequently, you may need to pre-compute the value at write time. Serving pre-aggregated data is faster than re-computing the aggregate on each request.

  2. Huge datasets: If the Pre-aggregation is particularly beneficial when dealing with large datasets where performing aggregations or complex calculations on the fly can time out from being too computationally expensive.

To pre-aggregate data, add a new field to your model to hold your aggregate, and then update it in your action code when a record is created or updated.

Was this page helpful?