Computing in Gadget with Gelly

Gelly is the data access language powering advanced features in Gadget like Computed fields and Filtered Model Permissions. Gelly is a superset of GraphQL that follows the same principles of declarative queries returning predictable results.

Computed field snippets look like this:

field OrderCount on ShopifyShop {
count(orders.id)
}

and filtered model permissions look like this:

fragment PostFilter($session: Session) on Post {
*
[where !isSpam || authorId == $session.userId]
}

Gelly snippets look like this:

gelly
1fragment Details on Product {
2 # select the name field
3 name
4
5 # select a field named inStock, which is true if the
6 # inventory count is greater than 0
7 inStock: inventoryCount > 0
8
9 # sort the returned list of Product by the id field descending
10 [order by id desc]
11
12 # return some fields the first three variants of each product,
13 # sorted by most recently created
14 variants {
15 id
16 name
17 price
18 [limit 3 order by createdDate desc]
19 }
20
21 # and return an aggregate count of the images for the product
22 count(images.id)
23}

Why a different language?

Plain old JavaScript code works well for adding functionality to Gadget applications when implementing behavior, but some parts of applications have different needs. Gelly computations are declarative such that Gadget can pre-compute or re-compute the values of Gelly snippets very efficiently across a high number of rows. This means you don't need to determine and store a value each time a record changes in a String or Number, and instead you can create a Computed field that's easier to change and better performing.

Currently, Gadget compiles Gelly into high performance SQL to execute inside the database with a custom caching layer on top, which makes Gelly fast while remaining consistent. Gadget does this in order to power features like filtering or sorting on a computed field, and conditional permissions that decide if a row is visible using data from related rows. Neither of these is possible at scale if the computation is defined in JavaScript because Gadget would have to run the computation for every row in the database to know which ones to return, which can be too slow at scale.

Gelly for SQL lovers

Gelly is similar to, and inspired by SQL! A Gelly query is also an up front, declarative ask for some data based on the relational algebra, but it's different in that it allows relationship traversal, fragments, and more ergonomic expressions. Gelly eliminates a few annoyances from plain old SQL like having to manually specify join conditions all the time, having to execute multiple queries or complicated json_agg functions to retrieve related data, and plain old quality of life stuff like not requiring trailing commas or a super specific statement order.

Gelly for GraphQL lovers

Gelly is very similar to GraphQL, but extends it with the ability to do a bunch of fancy stuff normal GraphQL APIs can't. Gelly snippets can contain arbitrary computations that build new expressions with normal-looking code, as well as universally available commands to filter, sort, or aggregate a list of data.

Computed fields

Computed fields are read-only fields defined by a Gelly snippet. A Computed field is accessed the same ways as any other field through the API. Gadget guarantees the value to be up to date as input data referenced in the computed field's definition changes, and Gadget takes responsibility for efficiently computing and caching the value. You don't have to spend time worrying about counter caches, TTLs or refresh intervals, query optimization, or input row counts. Gadget implements Gelly with on-demand SQL queries for lightweight Gelly snippets and incremental dataflow recomputation for heavy weight Gelly snippets behind the scenes.

Computed fields are defined using a Gelly field fragment. A field fragment defines one named field on a model and an expression to produce the value for each row.

An example field fragment for a Computed might define the fullName field for a Customer model, which might comprise the first name plus a space plus a last name:

gelly
field fullName on Customer {
firstName + ' ' + lastName
}

Another Computed might define the count of all the comments on a BlogPost model using a count of a HasMany relationship:

gelly
field commentCount on BlogPost {
count(comments.id)
}

Note that Computed fields are defined by only one expression, and can't produce more than one value for one row. This is why Computed fields use Field Fragments and not normal Fragments. You can't use a normal Fragment to define a computed field like so:

gelly
# won't work, must use a Field Fragment, not a normal Fragment
fragment CommentCount on BlogPost {
commentCount: count(comments.id)
otherField: upper(title)
}

Examples

Produce a fullName field that concatenates the firstName and lastName string for a Customer model:

gelly
field fullName on Customer {
firstName + ' ' + lastName
}

GraphQL differences

Gelly is similar to and inspired by GraphQL -- it's a really good idea. There are some key differences between the two systems thought:

  • Gelly queries can contain arbitrary expressions which are evaluated on the server side

This is so that folks on the front end can push as much work to the server as possible, and so that they don't need to manually add new, bespoke API endpoints for each separate thing they want to do. Sometimes it is simpler to just ask the server to upcase a string, or filter on an expression, or group by an arbitrary field that the user has selected. So, Gelly has math, function calls, aggregations, and relational commands built in.

  • Gelly has relational commands available on every relation instead of some GraphQL fields having arguments

In Gelly, the relational commands [where], [order by], [limit] and [group by] are universally available on every relation. This allows developers to work quickly and leverage the automatic query compilation and execution that Gadget does for each Gadget application. Some GraphQL APIs expose similar facilities to Gelly's relational commands, but as bespoke field arguments, and that often require backend work to set up and make performant. Aggregations in Gelly aren't exposed as other strange fields in a schema, and are instead available as aggregate functions

  • Gelly doesn't have Relay style Connections, and instead has a formal idea of a relation

Gelly is a syntax around a fundamentally very powerful concept called the relational algebra. Just like in the algebra, relations in Gelly are things you can manipulate, and so we don't work with them as just more types in the system, and instead bake the idea in deeply. There's no edges { node { ... }} selections in Gelly, there's just subselections of relations. We have big plans for enhancing Gelly with support for the full relational algebra including joins as well as a relational chaining operator for implementing subqueries.

  • The Gelly schema can be extended with Gelly itself, which allows for great performance

Sometimes, it is valuable to teach the server about a new field that should be present on the model and work the same way for all clients, which Gelly supports with Field Fragments and Computed. These schema extensions are defined in the Gelly language itself, which allows the platform to push that work into high performance database queries, instead of shuffling a lot of data to the client for client side computation.

  • Gelly doesn't support Mutations

GraphQL supports writes and Gelly does not (yet). If you're building a Gadget app, you can use the GraphQL API to execute writes to your application.

  • Gelly doesn't support Subscriptions or @live queries

Some GraphQL servers support subscribing to a specific field, and some even support asking for arbitrary changes to a query result over time, both in a push style. Gelly doesn't yet support these facilities.

  • Gelly null works like SQL NULL, not like JavaScript's null, where null == null gives null.

Gelly is powered by SQL underneath the hood, and so inherits SQL's null semantics. This means that null == null returns null in Gelly, not true. To check if something is null, you should use the isNull function.