Building models

Every Gadget project comes with a built-in database powered by PostgreSQL for storing data. The Gadget database is relational, schema-ish, and stores your data in the schema you create by enforcing your rules. Gadget has a built in set of field types, relationships, and validations that can be used to build the correct data models for your application.

Data Modeling

Gadget's database is relational, which really just means that it stores data in rows, and then there is a different bucket for each shape of row you might want. Gadget calls rows records and the tables models. So, to store data, you create a model which defines the shape of the data, and then create records of that model using the API. You can think of the Gadget database like a big, slightly more strict spreadsheet: many rows coming and going that all share the same set of columns, with different spreadsheet tabs for each kind of data you want to store.

This is very similar to traditional SQL based relational databases like PostgreSQL or MySQL, and different than document stores like MongoDB or Firebase. Gadget developers should be able to teach their applications what valid data looks like, and so Gadget's modeling system has tools to only allow data of a certain type into a column, or to validate that data in complicated, problem-specific ways to accept or deny it.

If you don't want to validate data in order to go fast, or you can't because the data is constantly changing shape, Gadget has a JSON field type that allows you to store whatever data you like once formatted as JSON. We call this schemaish, in that you can choose to use schema-ful, enforced rules when it suits you, and relax those rules when it doesn't.

Schema Management

Models stored in Gadget's database are easy to change. The Gadget platform takes care of creating and migrating the underlying data storage system as you change the shape of a model. This means you don't need to manually run something like CREATE TABLE or ALTER TABLE SQL statements as you change the shape of your model -- Gadget just keeps on working as you change your models.

This also means that all the schema change operations in Gadget are non-destructive and copy-on-write. For more information, see Changing a field's type and the Data Migrations section.

Field Types

Gadget offers many field types to choose from as you build out your models. Gadget fields represent all sorts of different data: scalar values like String and Number, lists of values like StringOptions, schemaless values like JSON, and relationships between models like BelongsTo or HasMany.

Here's all the field types Gadget currently supports:

Field typeDescription
StringStores an arbitrary length string with UTF-8 encoding
NumberStores a numerical value, optionally with a certain precision. Used to store both integers, floats, and arbitrary precision values.
IDStores a unique identifier for each record in this model. IDs are unique within the model, but not across models
StringOptionsStores one string or a list of strings picked from a global list of allowed values. Similar to an enum in traditional SQL databases
RichTextStores an enhanced-Markdown formatted string. Can produce formatted HTML for the markdown.
BooleanStores a true or false value
DateTimeStores a date or date and time value with a timezone. Accepts and serves data as an ISO-8601 string.
EmailStores a email as a string
URLStores a URL as a string
MoneyStores an amount of money with a specific currency configured on the field.
ComputedProduces a value according to a Gelly code snippet using other field values

Each field type in Gadget has specific configuration values that govern its behavior. You can change how a field behaves by changing this configuration on the Storage page for the model. For example, some field types (such as string), allow you to input a default value which gets passed to the API unless it is overridden by the user.

Validations

Validations allow you to prevent misshaped or incoherent data from entering your system. When running actions, Gadget ensures that any new data entering the system passes your validations. Invalid data isn't persisted and instead structured error messages are returned to the client.

Gadget offers a selection of common validations for each field type. These vary from one field type to another:

Field typeApplicable Validations
StringRequired, Uniqueness, String Length Range, Run JavaScript, Match RegExp
NumberRequired, Uniqueness, Number Min/Max Range, Run JavaScript
IDNo applicable validations
StringOptionsRequired, Uniqueness,Run JavaScript
RichTextRequired, Uniqueness, String Length Range, Run JavaScript, Match RegExp
BooleanRequired, Uniqueness,Run JavaScript
DateTimeRequired, Uniqueness,Run JavaScript, Match RegExp,
EmailRequired, Uniqueness, String Length Range, Run JavaScript, Match RegExp
URLRequired, Uniqueness, String Length Range, Run JavaScript, Match RegExp
MoneyRequired, Uniqueness,Run JavaScript
ComputedNo applicable validations

The Run JavaScript and Match RegExp validations allow you to write custom validation logic in the form of a JavaScript function or regular expression. If you cannot find a field type that matches your expectations, you can build your own by adding a Run JavaScript or Match RegExp validator to the field type that is closest to the definition you are looking for.

Gadget uses the RegExp engine built into node.js to run Match Regexp validations. For information on how to write JavaScript regular expressions, please refer to MDN's reference

Relationships

Gadget allows you to build relationships between your different models by using the relationship field types. Relationship fields allow traditional normalized data modeling in Gadget, where different classes of objects can reference each other while existing independently. Relationships are useful because they allow you to model real world objects that have distinct lifecycles from one another. For example, in a blog, posts get created at different times and by different people than comments, so it is simple and powerful to model posts and comments as two separate models that are related to one another. That way, each can have their own permissions, their own effects, and records of each model can be created or destroyed independently, while staying linked together and being easy to fetch together in the API. Gadget relationships are powered by storing the identifier for one record in the other record's data, like a post_id column in a comments table in a SQL database.

There are four relationship types available:

  • BelongsTo relationships are used to express that a record references zero or one other parent record. Examples: books belong to one author, blog comments belong to one blog post. BelongsTos denote the inverse of a HasMany or HasOne relationship.
  • HasMany relationships are for many-to-one relationships where one parent record is referenced by many children records. Examples: blog posts have many comments, authors have many books
  • HasOne relationships are for one-to-one relationships between two models.
  • HasManyThrough relationships are used when describing a many-to-many association between two models. Examples: students can have many courses, and courses have many students; or in e-commerce, a single product can be categorized into multiple categories, while each category holds many products. A HasManyThrough requires an in-between join model that has one row per unique combination of the two related models.

You are not required to setup any foreign key columns in Gadget -- you just specify the relationships, and Gadget stores the necessary identifiers. The identifiers remain accessible via the GraphQL API. You can change the value of relationship fields using the API in the same way you would change other fields. Gadget has extra facilities in the API generated for relationship types to do the extra things you might want to do with relationships, like move a record between parent records or remove a relationship between records without deleting either.

BelongsTo Relationships

Belongs To relationships set up a connection between one record of a model to one record of another model. BelongsTo relationships should be used anytime one record is directly owned or categorized by a record of another model. Adding a BelongsTo to a Gadget model is like adding a something_id column and a FOREIGN KEY constraint to a table in an SQL database.

For example, if you have a Blog Post model representing each post to a blog, you might also have a Comment model that allows readers of the blog to discuss each post. In this case, each comment pertains directly to one post, and so we say the comment belongs to the post, and create a BelongsTo relationship from the Comment model to the Post model.

Each Comment record will then have an associated Post object. Different comments can have different posts, but each comment has only one post.

BelongsTo relationships create a one way relationship from the model they are on to the related model. If you want to access the relationship from the other way around, to say access all the comments for a given post, you would need to create a relationship field on the other model. BelongsTo powers one model's side of a one-to-many or one-to-one relationship, so you can use a HasOne or HasMany on the other model to govern how many related records the other model can have.

BelongsTo relationships actually store an ID value in the Gadget database under the hood. This powers the BelongsTo relationship itself and allows Gadget to go fetch the related record when asked, and it also powers the HasMany or HasOne relationships that might be on the other side.

HasMany Relationships

Has Many relationships set up a connection between one record of a model to many records of another model. HasMany relationships should be used anytime one record owns or categorizes several records of another model. Adding a HasMany doesn't have a direct equivalent in SQL because it doesn't store anything -- it just sets up useful API fields for accessing and mutating the list of related records.

In the example of a blog app, we might have a Blog Post model and a Comment model where each comment belongs to a post, and so the Comment model would have a BelongsTo relationship pointing to the Post model. In this case, we'd want each post record to have many comments so that several different users could come and write comments. We say that the post has many comments, and create a HasMany relationship on the Post model linking the Comment model, so that in the API each post's list of comments can be fetched easily.

HasMany relationships create a one way relationship from the model the are on to the related model, and they require a BelongsTo relationship on the related model to work. This is because they are powered by the data stored in the other model's relationship. For example, the list of comments for a particular blog post is determined by looking at all the comments, and seeing which ones have a stored reference to the post in question in their BelongsTo relationship field. HasMany and BelongsTo together power a one-to-many relationship.

HasOne Relationships

Has One relationships setup a connection between one record of a model to one record of another model. HasOne relationships should be used any time there are two models that are always siblings, and always only have zero or one records on the other side of the relationship.

HasOne relationship fields represent one side of a one-to-one relationship while BelongsTo represent the other. You must use a BelongsTo on one model -- you can't use two HasOne fields, or two BelongsTo fields. This is because Gadget needs to know which side of the relationship to store the ID reference data in.

For example, we might have a User model in an application that may or may not have set up a billing relationship with the company running the app. In this instance, each user can only have one Billing Account, and each billing account corresponds to exactly one user. We have a one-to-one relationship, and so we add a BelongsTo to the Billing Account model and a HasOne to the User model:

HasOne relationships tend to be somewhat rare. If the two models that have a one-to-one relationship have the same lifecycle such that they are created and deleted at the same time, it can be annoying to manage them as two separate models, and so it's often easier to just combine them into the same model and not have any relationship at all. If the two models in a one-to-one relationship have different lifecycles, HasOne can be useful to allow one model to come and go independently of the other.

It can also be hard to determine which side of the relationship should get the HasOne and which side should get the BelongsTo. Usually, it is best to select the model that tends to exists for longer as the owner of the HasOne. That way, when the model that gets deleted more often gets deleted, you don't have to go update the other model that is still around, because the BelongsTo field's value has just been deleted. If there isn't really a pattern between when the two models get created and deleted, then it tends to not matter which side gets which relationship type.

HasManyThrough Relationships

Has Many Through relationships set up a connection between two models where records on each side of the relationship can have many related records on the other side. HasManyThrough relationships should be used when both sides of a relationship have many records on the other side, or there's no clear owner in a relationship between two models.

Becauese they represent many-to-many relationships, there is no correct place to put an ID reference on either of the related models, as that would only let us model a relationship to one record instead of to many. For this reason, Gadget uses a third, intermediate model, where each record of this third model represents one instance of the relationship between the first two models. Because of this, HasManyThrough relationships require a bit more configuration that usual to set up.

For example, let's consider a school registration system, where we have a Student model representing each person enrolled at the school, and a Course model representing each subject being taught at the school. Students need to enroll in many courses because they take more than one course at once, and courses are taught by teachers who can teach more than one student at once. Each side of this relationship has more than one related record on the other side, so we call it a many-to-many relationship and use a HasManyThrough to represent it. We create a third model that sits inbetween Student and Course called Registration that will have one record for course that each student has enrolled in.

If one student enrolls in three courses, that will generate three Registration records representing those three enrollments. If a second student then enrolls in the same three courses, that will generate three more Registration records. These Registration records have independent lifecycles, so to model a student dropping a course, we'd delete the relevant Registration record, and leave the Student and Course records as they were.

System fields

Each model in Gadget has four fields managed by the Gadget platform that are necessary for Gadget to function. These fields are:

  • ID: This field captures a unique ID for each of your records within a table
  • Created At: This field captures the timestamp at record creation
  • Updated At: This field captures the timestamp of the last event which changed the record
  • State: This field captures and stores the state of each record

States in Gadget

Gadget always creates a State field within each new model. The State field captures structured information about the current status of a record. Using State to implement your business logic is optional, but it can be a great way to express who is allowed to do what when. Instead of layering all your business logic into generic and not very descriptive operations like create, update, and delete, Gadget allows you to express verbs like publish that moves a record from the Unpublished state to the Published state, or a series of states like Todo, Doing and Done with a bunch of transitions between them.

The State field stores the current state for each record. The values of this field correspond to states defined in the behavior chart on the model's Behaviour page. The value of this field is often a string, but since states can be nested, the value of this field can also be an object with nested keys expressing the value of nested state.

JavaScript
1const record =
2 await api.widgets.findOneOrThrow(1);
3console.log(record.state);
4//=> "created"
5
6const other =
7 await api.fancyRecord.findOneOrThrow(
8 1
9 );
10console.log(record.state);
11//=> { "created": "unapproved" }

By default, each record in your database will be in the created state. This is because Gadget requires a state for every record, and the default state machine has a Created state that each record starts in.

Nested States

States can be nested within one another, which allows you to represent inner state machines that are only active when a record is in an outer state. This happens most often by adding new substates to the Created state.

The value of the state field on a Gadget record is a string when there are no substates, and an object with the outer state name as the key and the inner state value as the value. This nests arbitrarily deeply.

Here are some examples:

JavaScript
1// for a record with the default state machine
2record.state; // => "created"
3
4// for a record with two states Unapproved and Approved nested inside Created
5record.state; // => { created: "approved" } or { created: "unapproved "}
6
7// for a record with two states Unapproved and Approved nested inside a Paid state which is itself nested in Created
8record.state; // => { created: { paid: "approved" } } or { created: { paid: "unapproved "} }

If you have taught Gadget that your users come in two forms, active or churned, and had modeled these as sub-states of ‘created', like so, any record in the “active” substate would have its state recorded as “created.active”. Conversely, if you were looking at a user record pertaining to a churned user, you would see the record's state as “created.churned”.

Field Defaults

Most field types in Gadget can have a default value configured. This default value will be saved to a record when no value is passed for that field during any model action. Field defaults can be changed over time or added retroactively, but they won't affect existing data.

Default Models - User and Session

Each new application in Gadget has a User and a Session model generated on app creation. These models are a starting point for an authentication system system for your application and come with preconfigured behavior diagrams to power logging in and logging out. You can change them or delete them if you don't need them.

User Model

The Gadget user model is intended to house the users of your backend. By default, Gadget requires that each user have an email, a password, and a role. You can modify the information needed by adding or removing fields from the model.

Session Model

The session model that is pre-generated for every Gadget app governs the login system. When a user is logged in, a user is set on the browser's session. Conversely, a logged out user is represented by a session cookie with no user_id set on it. The Set User and Unset User effects can be used to implement login and logout on the default Session model.

The User model also comes with a Sign Up action. In addition to creating a new User record in your database, this action assigns this new user's id to their current session's user_id. This means users can be created and signed into your app in a single event.

Changing a field's type

You can always change the type of a field in a model. If the old field type can be converted losslessly into the new field type (like say a number into a string), Gadget will record the change in the shape of the model and not change any stored data.

If the old field type can't be converted to the new field type, Gadget will start writing new data with the new type to a new spot on disk, without destroying the old data. Currently, the old data is not accessible, so if you want to migrate the old data to a new place, we recommend creating a new column, and then writing a script to migrate the data from the old column to the new column via API calls.