Extending with Code

Gadget can be extended in a variety of ways using JavaScript files that interplay with the existing bits of the platform. You can add Validations, Preconditions, and Effects written in JS to make your application do exactly what it needs to.

For Computed fields, refer to the Computed Fields guide.

When to write code

Code is the right tool to solve many problems for developers, and there's a whole lot of it out there already! Gadget is built for this reality. While Gadget handles a good amount of the boilerplate, there's lots of business logic unique to your problem that is best captured in code.

We recommend that when there is a platform tool that solves your problem, you try to use it before dropping down to coding your own. The Gadget team is constantly adding features and fixing bugs so that you don't have to do it yourself. Where possible, staying on the upgrade train is generally preferable so your application stays correct, performant, and secure.

If you need to write code though, you should not hesitate! Gadget strives to be an excellent neighbor and provides all the same runtime tooling for custom code as it does for the built-in primitives. Your code can use the Gadget logging system, the scalable runtime, and do all the same things the builtins do like writing data to the database. If code is the right tool for the job, you should go right ahead and use it.

Deploying new code

Gadget is a deployless system -- all code written in the Gadget editor is running live in the cloud as soon as it is saved. This allows you to make changes and then preview your application immediately, keeping your feedback loop short and helping you get stuff done. Your code runs with the full power of Gadget's optimized, elastic cloud systems; so if you need to chew through a lot of data during a big migration, or access sensitive third-party APIs, you're able to do so in development just as easily as you might in production.

Validations

Gadget has a variety of built in validations that cover the common cases for data validation. If you have a complicated or unique validation rule that governs data in your application, Gadget supports adding your own validations written in JavaScript. The Run Code validation runs a JS function each time a record is created or updated to check if that change should succeed. If the function reports no errors, then the save will complete, and if the data is somehow invalid, the validation can add structured error messages to the different fields of the record being changed.

Each Run Code validation file must export a function as the default export function from the module. The function can be marked async or return a Promise The function doesn't need to return anything, and instead, should add errors to any fields determined to be invalid using the passed-in errors object. The errors object is an Errors that holds a structured set of errors keyed by field.

By convention, Run Code validations are placed in a root-level folder named after the model, and then in the validations folder. For a model named Widget for example, the Gadget editor will suggest you add the first validation at widget/validations/some-useful-code.js. You can move or rename these files however you wish.

Validation examples

Here's a validation that tests if a record's full name is not null, and its last name is also not null:

module.exports = async ({ record, errors }) => {
  if (record.firstName && !record.lastName) {
    errors.add("lastName", "must be set if the first name is set");
  }
};

Here's a validation that tests if a record's phone number is valid:

1const PhoneNumber = require("awesome-phonenumber");
2
3module.exports = async ({ record, errors }) => {
4  const number = new PhoneNumber(record["customerPhoneNumber"]);
5  if (!number.isValid()) {
6    errors.add("customerPhoneNumber", "is not a valid phone number");
7  }
8};

If a validation function throws an error while executing, the action validating the current record will fail and its effects rolled back.

Preconditions

Preconditions capture logic about when an action can be taken. If the preconditions fail, no action effects run. Preconditions are useful specifically because they can be run independently of the action itself to figure out if the action can currently be taken. This is great for disabling buttons that run actions in a UI or hiding bits of functionality a user is unable to access.

Preconditions are all expressed as files of JavaScript code. Each Precondition file is expected to be a valid JavaScript module that exports a function that returns true or false as its default export. The function can be asynchronous and will be awaited by the Gadget runtime before running the rest of the preconditions or action. If the precondition returns true, the action execution will proceed, and if it returns false, the action's execution will be halted and any database effects rolled back.

When you create a precondition in the Gadget behavior system, Gadget will automatically create you a .js file to hold your preconditions' source code. It will be placed in a root-level folder named after the model, and then in a folder named after the action the precondition is on, and then in the preconditions folder. For an action named Update on a model named Widget for example, you'd find the first precondition you add to Update at widget/update/preconditions/check-A.js. You can rename this file to make the name more descriptive if you like.

If a precondition function throws an error while executing, the action acting on the current record will fail, and its effects will be rolled back.

Precondition examples

Here's a precondition that only allows a record to be updated if the record's owner field matches the current logged-in user:

module.exports = ({ api, record, session }) => {
  const currentUser = session.get("user");
  return record.owner._link == currentUser._link;
};

Here's a precondition that only allows blog posts to be published on weekdays:

module.exports = () => {
  const currentDay = new Date().getDay();
  return currentDay > 1 && currentDay < 6;
};

Here's a precondition that only allows tasks in a todo application to be started if their due date is present. This would allow tasks to be created in an unstarted state without due dates, but prevent them from being started until the due date is set.

module.exports = ({ record }) => {
  return record.dueDate != null;
};

Effects

Effects do useful stuff for users of Gadget applications. Gadget has some built-in effects for basic manipulation of the database, but often, specific bits of business logic should be written in code to solve a problem well. Run Code effects are where you should put this code. Each Run Code effect is created by adding one with the plus button to one of the Action effect lists:

• the Run Effects
• or the Success Effects

Run Effects generally run in a database transaction, and Success Effects run after the transaction has been committed. For more information about the differences and Action transaction boundaries, see the Actions Guide. The interface for effect code is the same regardless of which effect list an effect runs in.

Effect functions

Like Validations and Preconditions, each Run Code effect file must export a function as the default export from the module. If asynchronous, this function will be awaited by the Gadget runtime during action execution. The function should run whatever logic it needs to, which most often is using the api object to make changes to the Gadget database, or inspecting the record object to send its data elsewhere.

Each Run Code effect lives in one .js file. When you create a Run Code effect, Gadget will automatically create a .js file to hold the effect's source code. Once you name the file, it will be placed in a root-level folder named after the model, and then in a folder named after the Action the effect runs within. For an Action named Update on a model named Widget, for example, you'd find the first Run Code effect you add to Update at widget/update/yourFileName.js. Existing files in the widget/update directory will be shown, however, if you'd like to use an existing file that is not located in this directory you may start typing to search for the file.

For example, we could create a Run Code effect on an update action of a model named Widget like so:

module.exports = ({ record, logger }) => {
  logger.info({ record: record.id }, "Hello from an Effect!");
};

Effect context

Run Code functions get passed one argument, which is a big EffectContext context object describing the current action execution. Effects don't get passed individual arguments, but instead one object with all the necessary keys. This is because there are many different elements you may or may not need. Most Gadget users use JavaScript destructuring to pull out just the context keys that they require.

The EffectContext object passed into each effect function has the following keys:

• api: A connected, authorized instance of the generated API client for the current Gadget application. See the API Reference for more details on this object's interface.
• params: The incoming data from the API call invoking this action.
• record: The root record this action is operating on.
• session: A record representing the current user's session, if there is one.
• config: An object of all the environment variables created in Gadget's Environment Variables editor.
• connections: An object containing client objects for all connections. Read the connections guide to see what each connection provides.
• logger: A logger object suitable for emitting log entries viewable in Gadget's Log Viewer.
• model: An object describing the metadata for the model currently being operated on, like the fields and validations this model applies.

record use in Effects

For Model Actions, Gadget automatically loads the record that the action is being taken on from the database at the start of the action. The record is stored in the record key on the action context, so it is there if you'd like to access it. Effects that change the record should apply those changes to the record object within the context. Note that the record is not automatically saved and that you must use a Create Record or Update Record effect or API calls to persist record changes.

api use in Effects

The api object that gets passed into an Effect function is an instance of the generated JavaScript client for your Gadget application. It works the same way in an effect as it does elsewhere, and has all the same functions documented in the API Reference. You can use the api object to fetch other data in an effect:

1// example: send a notification to the user who owns the record if they've opted in
2const twilio = require("../../twilio-client");
3module.exports = async ({ api, record }) => {
4  const creator = await api.users.findOne(record.creator._link);
5  if (creator.wantsNotifications) {
6    await twilio.sendSMS({
7      to: creator.phoneNumber,
8      from: "+11112223333",
9      body: `Notification: ${record.title} was updated`,
10    });
11  }
12};

Or you can use the api object to change other data in your application as a result of the current action:

1// example: save an audit log record when this record changes
2module.exports = async ({ api, record, model }) => {
3  const changes = record.changes();
4
5  // run a nested `create` action on the `auditLog` model
6  await api.auditLogs.create({
7    action: "Update",
8    model: model.apiIdentifier,
9    record: { _link: record.id },
10    changes: changes.toJSON(),
11  });
12};

Sub-action execution in Gadget is still governed by the same rules as top-level action execution: actions need to pass the preconditions and records need to be in the right state for the action being executed.

Using the Public API vs the Internal API

Depending on what you're trying to accomplish with an effect, it is sometimes appropriate to use the Public API of your application, via api.someModel.create, and sometimes it's appropriate to use the Internal API, via api.internal.someModel.create.

Since effects are used to run the important business logic of your application, you will generally want to use the Public API. The Public API for a Gadget application invokes Model Actions and Global Actions with all their preconditions and effects. This means that if you call api.someModel.create() within an effect, you are invoking another action from within the currently invoking action. This is supported by Gadget and is often the right thing to do if your business logic needs to trigger other high-level logic.

Sometimes though, you may want to skip executing the effects that make up an action. The Internal API does just this. Internal API endpoints make low-level changes to the Gadget database, and don't run effects or preconditions of any action. The Internal API can be used safely within effects by running api.internal.someModel.<action name>. You can think of the Internal API like raw SQL statements in other frameworks where you might run explicit INSERT or UPDATE statements that skip over the other bits of the framework. The Internal API is significantly faster, and is often used for building high-volume scripts that need to import data or touch many records quickly. Because the Internal API skips important logic for actions though, it is generally not recommended for use unless you have an explicit reason.

Action Infinite Loops

Invoking actions within actions can cause infinite loops, so care must be taken to avoid this. If you invoke the currently underway Action from within itself, your effects will run again, potentially invoking the action again in a chain. Or, if you invoke an Action in model B from model A, but model B invokes an action in model A, you can create a chain of action invocations that never ends. Action chains like this can show up in the logs as timeouts or errors that happen at an arbitrary point in an Action.

To avoid infinite loops, you must break the loop somehow. There are a couple of options for this:

• use record.changed('someField') to only run a nested action some of the time when data you care about has changed
• use the Internal API instead of the Public API (api.internal.someModel.update instead of api.someModel.update) to make changes, skipping the effect stack that retriggers the loop

Connection Action Infinite Loops

Calling third-party APIs from your effects is common, but it's important to be aware that third-party APIs may trigger subsequent actions in your Gadget app in response to changes.

For example, if you have a Gadget app connected to a Shopify store with a connected Shopify Product model, every time Shopify sends the app a product/updated webhook Gadget will run the Update action. If the business logic inside the Update action makes API calls back to Shopify to update the product again (say to update the tags of a product in response to a change in the other fields), the API call back to Shopify will trigger a second webhook sent back to Gadget. Without care, this can cause an infinite loop of actions triggering webhooks that retrigger the actions.

See Action infinite loops for strategies to break this infinite loop.

logger use in Effects

Each action effect is passed a logger object conforming to the logger API for logging to Gadget's built-in Log Viewer.

1module.exports = ({ api, record, logger }) => {
2  if (!record.someField) {
3    logger.error({ record }, "record is missing required field for processing");
4  } else {
5    logger.info({ recordID: record.id }, "processing record");
6    doImportantThingWithRecord(record);
7  }
8};

Read more about the logger object in the Logging guide.

connections use in Effects

The connections object in an effect's context contains prebuilt client objects with the appropriate set of credentials, which you can use to make remote API calls. This is useful when the remote calls you need aren't provided as effects, or you need them to happen conditionally.

For example, suppose you have a Shopify connection to your store best-sneakers.myshopify.com and you want to update a product. You could write an effect that creates a Shopify client for this shop, and makes an API call to Shopify like so:

1module.exports = async ({ api, record, connections }) => {
2  const shopifyClient = await connections.shopify.forShopDomain(
3    "best-sneakers.myshopify.com"
4  );
5  await shopifyClient.product.update(productId, productParams);
6};

It's important to keep in mind that changing your remote data could result in an update webhook being sent back to Gadget. Care will need to be taken to ensure you don't get into an infinite feedback loop with the remote system. One good way to ensure this doesn't happen is to make remote updates in success effects. This guarantees the record has been committed to the database, so you can check the incoming webhook to see if a relevant field has changed before making the remote call.

Specifying custom params

Sometimes actions need to accept parameters that aren't directly stored in the model, like a sendNotifications param that might enable or disable some business logic for sending emails. Custom params are added to an action by exporting a params object from a custom code effect on the action. Gadget expects the exported params object to be a subset of a JSON schema specification. For example, if you want to expose a boolean flag and name object in your action or global action:

1module.exports = async ({ params }) => {
2  if (params.sendNotifications) {
3    // ...
4  }
5};
6
7module.exports.params = {
8  sendNotifications: { type: "boolean" },
9  fullName: {
10    type: "object",
11    properties: {
12      first: { type: "string" },
13      last: { type: "string" },
14    },
15  },
16};

With that in place in your code effect, you can now make the following GraphQL call:

1# action
2mutation {
3  myActionModelA(
4    sendNotifications: true
5    fullName: { first: "Jordan", last: "Stag" }
6  ) {
7    success
8    # …other selections…
9  }
10}
11
12# global action
13
14mutation {
15  myAction(sendNotifications: true, fullName: { first: "Jordan", last: "Stag" }) {
16    success # …other selections…
17  }
18}

The params specification must come after the default export, module.exports = ..., so that there's something to attach the params specification to. Note from the above example that the schema is written as a JavaScript object.

Gadget currently supports the following types from JSON schema:

• object
• string
• integer
• number
• boolean
• array

Global Action Run Code Effects

Global Actions can also use Run Code effects for whatever they need to accomplish. Run Code effects within Global Actions won't be passed a record object or a model object, but they will still be passed the params, the config, the logger, and all the associated goodies you would find in a model Run Code effect.

Effect examples

Here's an example effect that updates a user's fullName field to be a combination of the firstName field and lastName field:

module.exports = async ({ api, record, session }) => {
  await api.internal.user.update(record.id, {
    user: { fullName: record.firstName + " " + record.lastName },
  });
};

Note that this effect uses api.internal.user.update, so it's using the Internal API to make a fast, simple update to the User model in the Gadget database, and is not running the Update action of the User model.

Here's an example effect that checks if an incoming purchase is above a certain amount, and if so, creates a Fraud Review record to power a business process to take a look at the especially high-value order:

1const HIGH_VALUE_THRESHOLD = 20000; // cents
2
3module.exports = async ({ api, record, session }) => {
4  if (record.totalPrice.amount > HIGH_VALUE_THRESHOLD) {
5    await api.fraudReviews.start({ order: { _link: record.id } });
6  }
7};

Note that this effect uses the api.fraudReviews.start function, which triggers the execution of another action, the Start action on the Fraud Review model. This is desirable because perhaps that action itself has some effects that are important, so this code can run those effects without needing to know exactly what they are.

Here's an example effect that uses the Akismet spam detection API to ensure a comment isn't spammy during the Create action:

1const { AkismetClient } = require("akismet-api");
2
3// instantiate the client for the remote service
4const client = new AkismetClient({
5  key: "my-api-key",
6  blog: "https://myblog.com",
7});
8
9module.exports = async ({ api, record }) => {
10  const isSpam = await client.checkSpam({
11    content: record.body,
12    email: record.email,
13  });
14  if (isSpam) {
15    // throwing this error will roll back the transaction, aborting any in progress creates or updates
16    throw new Error("Can't save the comment because it seems spammy");
17  }
18};

Sharing code between files

All the code for your Gadget application is stored in a directory structure representing a plain old node.js module, so you can do all the things you normally can with a plain old node.js module like requiring files! You can create shared utilities and put them in whatever folder structure you like.

For example, if you have a Gadget application with these files:

widget/
  create/
    run-A.js
  update/
    run-B.js

and you'd like to share a utility function between those two run effects, you could create a file at widget/utils.js with the function in it:

module.exports.leftPad = (string, padding) => string.padStart(padding);

and then require it in widget/create/run-A.js and widget/update/run-A.js:

const { leftPad } = require("../utils");

module.exports = async ({ api, scope }) => {
  // ... effect code that uses `leftPad`
};

You can also create other folders or put source code files at the root of your project. We recommend sticking to the one-folder-per-model convention for structuring your validation, precondition, and effect code so that when Gadget generates you the boilerplate it will be close to the other code that touches the same model.

Sharing code effects between models

If you want to share a code file between actions on different models, and need to customize the behavior of the code file based on the model you're working with, you can do so by discriminating on the __typename property of your record.

In JavaScript code effects, you can use jsdocs to @typedef both context types that use this code effect, and define a union type for the context param.

1/**
2 * Effect code for Widgets and Gizmos
3 * @typedef { import("gadget-server").CreateWidgetActionContext } CreateWidgetActionContext
4 * @typedef { import("gadget-server").CreateGizmoActionContext } CreateGizmoActionContext
5 * @param {CreateWidgetActionContext | CreateGizmoActionContext} context
6 */
7module.exports = async ({ record }) => {
8  if (record.__typename == "Widget") {
9    // do something specific to widgets
10  } else if (record.__typename == "Gizmo") {
11    // do something specific to gizmos
12  }
13};

The editor will typecheck your record and the fields available in each block of the if statement will be narrowed to fields on that model. The __typename of each record is the camelized apiIdentifier of its model.

Packages from npm

Gadget supports installing node.js packages from the npm package registry like you might in other node.js systems. Each Gadget application has a standard, spec-compliant package.json file at the root which lists the packages the application needs to run. To add a package to your application, add a new key to the dependencies object inside package.json, and Gadget will install the most recent version of that package matching your version constraint. If Gadget fails to install your dependencies, a small red status icon will appear beside the package.json entry in the file tree with an error message explaining the issue.

Once installed, packages can be required by requiring them as you might in any other node.js project using require.

For example, if we add lodash to a package.json so it looks like this:

1{
2  "name": "an-example-gadget-app",
3  "version": "0.1.0",
4  "private": true,
5  "description": "Internal package for Gadget app an-example-gadget-app (Production environment)",
6  "dependencies": {
7    "@gadget-client/an-example-gadget-app": "link:.gadget/client",
8    "lodash": "^4.17.0"
9  }
10}

then we can require it in a validation, precondition, or effect file with require:

const _ = require("lodash");

module.exports = async ({ api, scope }) => {
  // ... effect code that uses `_.defaults` or `_.groupBy` or any other Lodash helper function.
};

Gadget also requires that your app depend on the generated Gadget API client. All Gadget apps are generated with an existing dependency on the Gadget API client, which the platform places into node_modules/.gadget automatically. This dependency shouldn't be removed.

Environment variables

Environment variables are variables whose values are set outside of the code that's leveraging them. Environment variables are often used to store sensitive information that the developer does not want to expose in code, say API credentials.

Gadget offers built-in support for storing and accessing environment variables.

Managing environment variables

To store environment variables in Gadget, click on Environment Variables found under CONFIG in the navigation menu. To add a new environment variable, click on "+ Add Variable" and complete the form as displayed below.

If the variable in question is sensitive and you wish to hide its value from being displayed in the Gadget editor, you can simply click on the lock icon next to the value input. You can also edit environment variables by overriding the value in the input field. Finally, environment variables can be deleted, simply by clicking on "Remove".

Using environment variables in code

Gadget allows you to set environment variables for use within code effects and all JavaScript files. These values are accessible using two different methods:

First, your app's environment variables are available within the config property of an action's effect context. Here's an example use case within a Global Action:

1/**
2 * Effect code for global action Example
3 * @typedef { import("gadget-server").ExampleGlobalActionContext } ExampleGlobalActionContext
4 * @param {ExampleGlobalActionContext} context - Everything for running this effect, like the api client, current record, params, etc
5 */
6module.exports = async ({ logger, config }) => {
7  logger.info({ value: config.FOO }, "logging an environment value");
8};

Second, to allow access outside of action contexts, such as in shared library code, all environment variables are also available within the process' environment, which you can access with process.env:

/**
 * Shared Example client
 */
module.exports = new ExampleClient(process.env["EXAMPLE_AUTH_TOKEN"]);

NODE_ENV environment variable

Gadget sets up each application with the NODE_ENV environment variable set to development for the Development environment and production for the Production environment. This is a common convention for node.js applications that instructs node modules to run a potentially slower but more explanatory version of code in development, and a higher performance but maybe less debuggable version in production.

If you'd like to opt-out of this behavior and run with a different value of NODE_ENV across your environments, you can adjust the value of the NODE_ENV environment variable on your Environment Variables page.

Runtime environment

Gadget runs JavaScript code, with support for TypeScript and other languages on the way. Gadget runs JavaScript using node.js version 16, and runs code inside containers on Linux. This means Gadget has support for:

• JS packages from the npm package registry, see the section on Packages from npm
• JS packages that require native extensions, like sharp or esbuild
• including other file types in your app and reading them, like metadata files in JSON or YAML
• including images and other assets you might need

Gadget's backend node.js processes are orchestrated using a fast, custom serverless runtime layer running in Google Cloud Platform. Each Gadget application is automatically deployed when anything changes, and each app is scaled up and down as needed without any manual knob-turning required.

Module system

The node.js runtime powering your Gadget application supports code written in the CommonJS style (with require statements) as well as the ECMAScript Modules style (with import statements) out of the box. Gadget compiles your code down to CommonJS to execute on node, so if you use import syntax or other newer ESM features, they will be transpiled to code that node supports out of the box, as opposed to run as they are.

For example, if you want to use ESM to write effect functions for a Gadget model, you can use import and export:

1import twilio from "../../twilio-client";
2
3export default async function ({ api, record }) {
4  const creator = await api.users.findOne(record.creator._link);
5  if (creator.wantsNotifications) {
6    await twilio.sendSMS({
7      to: creator.phoneNumber,
8      from: "+11112223333",
9      body: `Notification: ${record.title} was updated`,
10    });
11  }
12}

Gadget uses swc for code transpilation for minimal runtime overhead when executing code that needs to be transpiled.