Building models

Every Gadget project comes with a built-in database powered by PostgreSQL for storing data. The Gadget database is relational, schema-ish, and has whatever shape and rules you design. Gadget has a built-in set of field types, relationships, and validations that can be used to build data models for your application.

Data modeling

Gadget's database is relational, which really just means that it stores data in rows within tables. Gadget calls rows records and tables models. To store data, you create a model which defines the shape of the data and then create records of that model using the API. You can think of the Gadget database as a big, slightly more strict spreadsheet: many rows coming and going that all share the same set of columns, with different spreadsheet tabs for each kind of data you want to store.

This is very similar to traditional SQL-based relational databases like PostgreSQL or MySQL, and different from document stores like MongoDB or Firebase. We believe you should be able to teach your applications what valid data looks like, and so Gadget's modeling system has tools to only allow data of a certain type into a column, or to validate that data in complicated, problem-specific ways to accept or deny it.

If you don't want to validate data in order to go fast, or you can't because the data is constantly changing shape, Gadget has a json field type that allows you to store data without enforcing any schema at all. This is called schema-ish, in that you can choose to use schema-full, enforced rules when it suits you, and relax those rules when it doesn't.

Schema management

You can extend or change models stored in Gadget's database at any time. The Gadget platform takes care of creating and migrating the underlying data storage system as you change the shape of a model. This means you don't need to manually run something like CREATE TABLE or ALTER TABLE SQL statements as you change the shape of your model -- Gadget just keeps on working as you change your models.

This also means that all the schema change operations in Gadget are non-destructive and copy-on-write. For more information, see Changing a field's type.

Field types

Gadget offers many field types to choose from. Fields represent all sorts of different data: scalar values like string and number, lists of values like enum, schemaless values like json, and relationships between models like belongs to or has many.

Here are all the field types Gadget currently supports:

Field typeDescription
stringStores an arbitrary length string with UTF-8 encoding.
numberStores a numerical value, optionally with a certain precision. Used to store both integers, floats, and arbitrary precision values.
idStores a unique identifier for each record in this model. IDs are unique within the model, but not across models.
enumStores one string or a list of strings picked from a global list of allowed values. Can also be configured to accept any string as a valid input. Similar to an enum in traditional SQL databases.
rich textStores an enhanced-Markdown formatted string. Can produce formatted HTML for the markdown.
booleanStores a true or false value.
date / timeStores a date or date and time value with a timezone. Accepts and serves data as an ISO-8601 string. Fields configured to not include time will ignore the time / timezone portion of input strings.
emailStores an email as a string.
urlStores a URL as a string.
fileStores a file, like an image or a PDF, in cloud storage. See Storing files for more information.
computedProduces a value according to a Gelly code snippet using other field values.
colorStores a hexadecimal representation of a color.
jsonStores any JSON value. Any valid JSON is accepted.
passwordStores a bcrypt hashed password. One way encrypts the value, so the unencrypted text can't later be retrieved. Suitable for storing passwords and other secrets you might need to compare against but not reveal.
encrypted stringStores a string that is encrypted at rest and decrypted when accessed for extra security. Suitable for storing secrets whose contents you need later.
role listStores a list of Roles

Each field type in Gadget has specific configuration values that govern its behavior. You can change how a field behaves by changing this configuration on the Storage page for the model. For example, some field types (such as string ), allow you to input a default value that gets passed to the API unless it is overridden by the user.

Changing a field's type

You can always change the type of a field in a model. If the old field type can be converted losslessly into the new field type (like say a number into a string), Gadget will record the change in the shape of the model and not change any stored data.

If the old field type can't be converted to the new field type, Gadget will start writing new data with the new type to a new spot on the disk, without destroying the old data. Currently, the old data is not accessible, so if you want to migrate the old data to a new place, we recommend creating a new column and then writing a script to migrate the data from the old column to the new column via API calls.

password and encrypted string

Gadget has two field types for storing sensitive values: password and encrypted string. password are stored in your app's database using bcrypt, an industry standard one-way password hashing algorithm. Once stored, password values can't be retrieved again. Instead, incoming passwords for login attempts can be compared against the stored value to test if the user has the original password. password fields are a wise default for storing user authentication values like passwords or tokens whose cleartext you never want to see again.

encrypted string fields store the sensitive value in the database using two-way encryption with tweetnacl. Field values can be stored and read again if the user has access to the record. encrypted string are useful for storing access tokens or API keys which you need to be able to retrieve untouched, but whose value you want to protect with extra security.

Validations

Validations allow you to prevent misshaped or incoherent data from entering your system. When changing records, Gadget ensures that any new data passes the validations for each field. An action trying to write invalid data fails, the invalid data isn't persisted, and instead structured error messages about what fields are invalid are returned to the client.

Gadget offers a selection of common validations for each field type. These vary from one field type to another:

Field typeApplicable Validations
stringRequired, Uniqueness, String Length Range, Run Code Snippet, RegExp Pattern
numberRequired, Uniqueness, Number Min/Max Range, Run Code Snippet
idNo applicable validations
enumRequired, Uniqueness, Run Code Snippet
rich textRequired, Uniqueness, String Length Range, Run Code Snippet, RegExp Pattern
booleanRequired, Uniqueness, Run Code Snippet
date / timeRequired, Uniqueness, Run Code Snippet, RegExp Pattern,
emailRequired, Uniqueness, String Length Range, Run Code Snippet, RegExp Pattern
urlRequired, Uniqueness, String Length Range, Run Code Snippet, RegExp Pattern
fileRequired, File Size Range, Image File, Run Code Snippet
colorRequired, Run Code Snippet, Color
computedNo applicable validations
jsonRequired, Uniqueness, Run Code Snippet
passwordNo applicable validations
encrypted stringNo applicable validations
role listNo applicable validations

The Run Code Snippet and RegExp Pattern validations allow you to write custom validation logic in the form of a JavaScript function or regular expression. If you cannot find a field type that matches your expectations, you can build your own by adding a Run Code Snippet or RegExp Pattern validator to the field type that is closest to the definition you are looking for.

Gadget uses the RegExp engine built into node.js to run RegExp Pattern validations. For information on how to write JavaScript regular expressions, please refer to MDN's reference

For more information on the format of validation errors when using the Gadget API, see the `GGT_INVALID_RECORD` error docs.

Uniqueness Validations

The Uniqueness validation ensures that each value for the field occurs only once in the database. Uniqueness validations are appropriate for things like the email field on a User model to ensure that each email can only be registered once in your system, or username fields.

Uniqueness validations are enforced using underlying constraints in the database, so they can be trusted to enforce uniqueness in transactional contexts and under load.

If a Uniqueness validation is added to a model which already has data in the database, the validation can fail to set up if the data for the field already contains duplicates. Failed uniquness validations do not enforce uniqueness -- you must correct the duplicate data by deleting records or changing the offending fields such that the data really is unique per row, and then re-add the validation and ensure it succeeds.

Uniqueness validations can be made case sensitive or case insensitive by toggling the Case Sensitive option. If Case Sensitive is off, then values will be considered duplicates and rejected regardless of the casing of the letters within them. This option is appropriate for things like URL slugs or emails, where users may make casing mistakes when entering the values despite the casing not mattering for uniqueness.

Scoped Uniqueness Validations

The Uniqueness validation can optionally be scoped by another field. This enforces uniqueness of the field the validation is on for each group of data sharing the same value for the scoping field. This allows duplicate field values from a global perspective but prevents them for each scoped value.

For example, you might want to validate that each user of a game picks a unique nickname for each game round. The same nickname can be used by different people in different rounds, but not by different people in the same round. To enforce this, you would add a Uniqueness validation to a Nickname field, and then turn on the Scope by another field option, and select the Game Round field as the scoping field.

Image Validations

The Image File validation ensures that uploaded files meet three criteria:

  1. They have an image based MIME type or subtype — see MDN's reference.
  2. They are in one of the following image file formats: gif, jpg, png, svg, or webp (common web formats); apng, bmp, bpg, cr2, cur, dcm, flif, heic, ico, jp2, jpm, jpx, jxr, psd, or tif (additional formats).
  3. They do not contain an animation if the Allow animated images option is disabled.

Relationships and relationship fields

Relationship fields allow you to model the relationships between objects that have distinct life cycles from one another. For example, in a blog, posts get created at different times and by different people than comments, so modeling posts and comments as two separate models that are related to one another works great. This way, each model can have its own permissions, and its own effects, and records of each model can be created or destroyed independently. Relationships link models together such that they are easy to fetch together in the API as well. Gadget relationships are powered by storing the identifier for one record in the other record's data, like a post_id column in a comments table in a SQL database.

You are not required to set up any foreign key columns in Gadget — you just specify the relationships, and Gadget stores the necessary identifiers. The identifiers remain accessible via the GraphQL API. You can change the value of relationship fields using the API in the same way you would change other fields. Gadget has extra resources in the API generated for relationship types to do the extra things you might want to do with relationships, like move a record between parent records or remove a relationship between records without deleting either.

Adding a relationship

You can model a relationship in Gadget by adding a relationship field that joins two models. As an example, let's create a relationship between a Post model and a Comment model through a has many relationship.

On the Post model, we start by adding a new field, called Comments, and assigning it the has many Children field type. Note that we call this field Comments, as this field represents zero-to-many Comment records associated with a given Post record.

Selecting the relationship field type in Gadget

Then, we can select the model that is the child of Post, Comment:

Selecting the child model in Gadget

Once we've selected the Comment model, we can instantly create a belongs to field to reflect this relationship on the Comment model:

Completing the relationship connection between Comment and Post in Gadget

And with that, our relationship between Post and Comment is created on both models.

A completed relationship field connection in Gadget

You'll notice if you look at the data shape of our Comment model in the API reference, that there is a postID field present, even though you didn't manually create this field. When a relationship is created between two models, Gadget automatically creates an ID field to store the foreign keys needed to link instances of the related models to one another.

GraphQL
1type Comment {
2 """
3 The time at which this record was first created. Set once upon record creation and never changed. Managed by Gadget.
4 """
5 createdAt: DateTime!
6
7 """
8 The globally unique, unchanging identifier for this record. Assigned and managed by Gadget.
9 """
10 id: GadgetID!
11
12 """
13 The time at which this record was last changed. Set each time the record is successfully acted upon by an action. Managed by Gadget.
14 """
15 updatedAt: DateTime!
16
17 post: Post
18 postId: GadgetID
19
20 """
21 Get all the fields for this record. Useful for not having to list out all the fields you want to retrieve, but slower.
22 """
23 _all: JSONObject!
24}

Relationship types

There are four relationship types available:

  • belongs to relationships are used to express that a record references zero or one other parent record. Examples: books belong to one author, blog comments belong to one blog post. belongs to denotes the inverse of a has many or has one relationship.
  • has many relationships are for many-to-one relationships where one parent record is referenced by many children records. Examples: blog posts have many comments, authors have many books
  • has one relationships are for one-to-one relationships between two models.
  • has many relationships are used when describing a many-to-many association between two models. Examples: students can have many courses, and courses have many students; or in e-commerce, a single product can be categorized into multiple categories, while each category holds many products. A has many requires an in-between join model that has one row per unique combination of the two related models.

belongs to relationships

Belongs To relationships set up a connection between one record of a model to one record of another model. belongs to relationships should be used anytime one record is directly owned or categorized by a record of another model. Adding a belongs to to a Gadget model is like adding a something_id column and a FOREIGN KEY constraint to a table in an SQL database.

For example, if you have a Blog Post model representing each post to a blog, you might also have a Comment model that allows readers of the blog to discuss each post. In this case, each comment pertains directly to one post, and so we say the comment belongs to the post, and create a belongs to relationship from the Comment model to the Post model.

Each Comment record will then have an associated Post object. Different comments can have different posts, but each comment has only one post. belongs to relationships create a one-way relationship from the model they are on to the related model. If you want to access the relationship from the other way around, to say access all the comments for a given post, you would need to create a relationship field on the other model. belongs to powers one model's side of a one-to-many or one-to-one relationship, so you can use a has one or has many on the other model to govern how many related records the other model can have.

belongs to relationships actually store an id value in the Gadget database under the hood. This powers the belongs to relationship itself and allows Gadget to go fetch the related record when asked, and it also powers the has many or has one relationships that might be on the other side.

In the example above, a belongs to on the Comment model will create a post field on each Comment object in the GraphQL API that returns that comment's parent Post record. Comment records will also have a postId field if you need to access just the ID of a comment's parent post.

Here's what the setup looks like in Gadget:

A completed belongs_to relationship in Gadget

Other examples of belongs to relationships include:

  • a Blog Post record belonging to an Author record in a blogging app
  • a Product Image record belonging to an Product record in an e-commerce app
  • a Person record belonging to an Company record in a CRM app
  • a User record belonging to a Team record in a multi-tenant SaaS app

has many relationships

Has Many relationships set up a connection between one record of a model to many records of another model. has many relationships should be used anytime one record owns or categorizes several records of another model. Adding a has many doesn't have a direct equivalent in SQL because it doesn't store anything -- it just sets up useful API fields for accessing and mutating the list of related records.

In the example of a blog app, we might have a Blog Post model and a Comment model where each comment belongs to a post, and so the Comment model would have a belongs to relationship pointing to the Post model. In this case, we'd want each post record to have many comments so that several different users could come and write comments. We say that the post has many comments, and create a has many relationship on the Post model linking the Comment model, so that in the API each post's list of comments can be fetched easily.

has many relationship fields create a one-way relationship from the model they are on to a related model, and they require a belongs to relationship field on the related model to work. This is because they are powered by the data stored in the other model's relationship. For example, the list of comments for a particular blog post is determined by looking at all the comments, and seeing which ones have a stored reference to the post in question in their belongs to relationship field. has many and belongs to together power a one-to-many relationship.

In the example above, a has many on the Blog Post model will create a comments field on each BlogPost object in the GraphQL API that returns that post's list of Comment records.

Here's what the setup looks like in Gadget:

A completed has_many relationship in Gadget

Other examples of has many relationships include:

  • an Author record having many Blog Post records in a blogging app
  • a Product record having many Product Image records in an e-commerce app
  • a Company record having many Person records in a CRM app
  • a Team model having many User records in a multi-tenant SaaS app

has one relationships

Has One relationships set up a connection between one record of a model to one record of another model. has one relationships should be used any time two models are always siblings, and always only have zero or one records on the other side of the relationship.

has one relationship fields represent one side of a one-to-one relationship while belongs to represents the other. You must use a belongs to on one model -- you can't use two has one fields, or two belongs to fields. This is because Gadget needs to know which side of the relationship to store the ID reference data in.

For example, we might have a User model in an application that may or may not have set up a billing relationship with the company running the app. In this instance, each user can only have one Billing Account, and each billing account corresponds to exactly one user. We have a one-to-one relationship, and so we add a belongs to to the Billing Account model and a has one to the User model:

has one relationships tend to be somewhat rare. If the two models that have a one-to-one relationship have the same lifecycle such that they are created and deleted at the same time, it can be annoying to manage them as two separate models, and so it's often easier to just combine them into the same model and not have any relationship at all. If the two models in a one-to-one relationship have different lifecycles, has one can be useful to allow one model to come and go independently of the other.

It can also be hard to determine which side of the relationship should get the has one and which side should get the belongs to. Usually, it is best to select the model that tends to exist for longer as the owner of the has one. That way, when the model that gets deleted more often gets deleted, you don't have to go update the other model that is still around, because of the belongs to field's value has just been deleted. If there isn't really a pattern between when the two models get created and deleted, then it tends to not matter which side gets which relationship type.

In the example above, a has one on the User model will create a billingAccount field on each User object in the GraphQL API that returns that user's singular BillingAccount record.

Here's what the setup looks like in Gadget:

A completed has_one relationship in Gadget

has many relationships

Has Many Through relationships set up a connection between two models where records on each side of the relationship can have many related records on the other side. has many relationships should be used when both sides of a relationship have many records on the other side, or there's no clear owner in a relationship between two models.

Because they represent many-to-many relationships, there is no correct place to put an id reference on either of the related models, as that would only let us model a relationship to one record instead of to many. For this reason, Gadget uses a third, intermediate model, where each record of this third model represents one instance of the relationship between the first two models. Because of this, has many relationships require a bit more configuration than usual to set up.

For example, let's consider a school registration system, where we have a Student model representing each person enrolled at the school, and a Course model representing each subject being taught at the school. Students need to enroll in many courses because they take more than one course at once, and courses are taught by teachers who can teach more than one student at once. Each side of this relationship has more than one related record on the other side, so we call it a many-to-many relationship and use a has many to represent it. We create a third model that sits between Student and Course called Registration that will have one record for each course that each student has enrolled in.

If one student enrolls in three courses, that will generate three Registration records representing those three enrollments. If a second student then enrolls in the same three courses, that will generate three more Registration records. These Registration records have independent lifecycles, so to model a student dropping a course, we'd delete the relevant Registration record, and leave the Student and Course records as they were.

In the example above, a has many on the Student model will create a courses field on each Student object in the GraphQL API that returns that students' list of registered Course records using the Registration model to build the list underneath.

Here's what the setup looks like in Gadget:

A completed has_many_through relationship in Gadget

Other examples of has many relationships include:

  • a Category record having many Product records through Categorization records in an e-commerce app. Since products can be in many categories, and categories can have many products, there must be an intermediate through model to model the many-to-many relationship.
  • a Recipe record having many Ingredient records through a Recipe Amount record in a cooking app. Since ingredients can be used for more than one recipe, and recipes have more than one ingredient, we need an intermediate model to model the many-to-many relationship. Recipe Amount might also have extra data like the weight of the ingredient to be used, or say a relationship to alternative ingredients.

System fields

Each model in Gadget has three fields managed by the Gadget platform that is necessary for Gadget to function. These fields are:

  • ID: This field captures a unique ID for each of your records within a table

Gadget will automatically handle ID assignments. To ensure performance and stability, Gadget auto-increments IDs on every new mutation regardless of the status of the mutation. As a result, your saved records will always have a unique ID, but the IDs are not guaranteed to be sequential. As an example, your IDs may increment from 11 to 13, as the mutation to create the 12th record failed.

  • Created At: This field captures the timestamp at record creation
  • Updated At: This field captures the timestamp of the last event which changed the record

System fields cannot be given a different name / API identifier.

Field defaults

Most field types in Gadget can have a default value configured in the Schema Editor. This default value will be saved to a record when no value is given for any action that will create a record. The default value for a field can be changed at any time, but the new value only applies to records created after the value is changed.

States in Gadget

Some Gadget models have a record state field which stores a specific status for each record. Using record state to implement business logic is optional, but it can be a great way to express who is allowed to do what when. Instead of layering all your business logic into generic and overloaded operations like create, update, and delete, Gadget allows you to express rich verbs like publish, fulfill or complete that describe something specific and useful. Each of these verbs can move a record from one state to another, so publish might move a record from the Unpublished state to the Published state, or you might have a complete verb that moves a todo from the Not Started or Doing state to the Done state.

The record state field stores the current state for each record. The values of this field correspond to states defined in the field configuration panel for the given record state field.

JavaScript
1// fetch an example Post record that is unpublished
2let record = await api.post.findOneOrThrow(1);
3console.log(record.state);
4//=> "unpublished"
5
6// call an action which sets the state to published
7record = await api.post.publish(1);
8console.log(record.state);
9//=> "published"

Session models

Each new application in Gadget has a Session model generated on app creation. This model is a starting point for an authentication system for your application. You can change this model's fields and actions to build authentication. The Set User and Unset User effects can be used to implement login and logout on the default Session model.