Use Gadget and OpenAI to build a chatbot that generates custom movie scenes 

Topics covered: AI + vector embeddings, Actions, HTTP routes, React frontends
Time to build: ~30 minutes

Learn about Gadget's built-in AI features such as the OpenAI connection, vector databases, and cosine similarity search, and use them to build a chatbot that generates custom movie scenes.

You can fork this Gadget project and try it out yourself.

After forking, you will still need to:

• run the ingestData global action to ingest some sample data and generate embeddings for the movie quotes

Create a new Gadget app 

Before we get started we need to create a new Gadget app. We can do this at gadget.new. When selecting an app template, make sure you select the AI app template.

Now that we have a new Gadget app, let's start building!

Step 1: Create a movie model 

The first thing we need to do is store some movie quotes in our Gadget app. We're going to make use of Gadget's data models, which are similar to tables in a Postgres database, to store this information. To fetch data, we will make use of a global action.

Start by creating a new model in Gadget:

1. Click the + button in the DATA MODELS section of the sidebar
2. Enter movie as the model's API identifier

Now add some fields to your model. Fields are similar to columns in a database table, and allow you to define what kind of data is stored in your model. For our movie model, we'll add the following fields:

3. Click + in the movie model's FIELDS section
4. Enter title as the field's API identifier
5. Click on the + Add Validations drop-down and select Required to make the title field Required

Adding a Required validation to title means that an error will be thrown if a movie is added without a title. Now let's add a field to store the movie's quotes:

6. Click + in the movie model's FIELDS section
7. Enter quote as the field's API identifier
8. Click on the + Add Validations drop-down and select Required to make the quote field Required

Now we have a place to store movie quotes! This is all the raw required for our app, but we also need a field used to store vector embeddings. Vector embeddings are a way of representing text as a vector of numbers. Gadget has built-in methods to perform similarity searches using vector embeddings - for this app, we'll use these vectors to find similar movie quotes when compared to text entered by users. To learn more about vector embeddings, check out our docs on building AI apps.

9. Click + in the movie model's FIELDS section
10. Enter embedding as the field's API identifier
11. Select vector as the field's type

That is all that we need to store data for our app! Now we need a way to generate embeddings. Luckily, OpenAI has an API that we can use to pass in text and get back a vector embedding.

Step 2: Data ingestion 

We need some test data for our app. We're going to make a request to an open data model hosted on Hugging Face. Hugging Face is an AI community and a great resource for AI models, datasets, and great community-provided AI content. We will then use the OpenAI connection to generate embeddings for our movie quotes.

To ingest data into our app, we can make use of a Gadget global action. Unlike model actions, such as the default CRUD actions generated for our movie model, global actions serve as ways to apply updates to more than a single record.

1. Click on Global Actions in the sidebar
2. Click the + Add Action button or + next to the ACTIONS section title to create a new global action
3. Name the action's API Identifier to ingestData

Our OpenAI connection is already set up for us using Gadget-managed credentials. Each team gets $50 in OpenAI credits to use when building in development. To learn more about how to set up your own OpenAI connection, check out our OpenAI connection docs.

With our OpenAI client set up and ready to be used, we can add some code to create embeddings for our movie quotes when they are added to our Gadget database.

4. Enter the following code in the generated code file (replace the entire file):

globalActions/ingestData.js
js
Copy
1import { IngestDataGlobalActionContext } from "gadget-server";
2
3/**
4 * @param { IngestDataGlobalActionContext } context
5 */
6export async function run({ params, logger, api, connections }) {
7  // use Node's fetch to make a request to https://huggingface.co/datasets/ygorgeurts/movie-quotes
8  const response = await fetch(
9    "https://datasets-server.huggingface.co/first-rows?dataset=ygorgeurts%2Fmovie-quotes&config=default&split=train",
10    {
11      method: "GET",
12      headers: { "Content-Type": "application/json" },
13    }
14  );
15
16  const responseJson = await response.json();
17
18  // log the response
19  logger.info({ responseJson }, "here is a sample of movies returned from hugging face");
20
21  if (responseJson?.rows) {
22    // get the data in our record's format
23    const movies = responseJson.rows.map((movie) => ({ title: movie.row.movie, quote: movie.row.quote, embedding: [] }));
24    // also get input data for the OpenAI embeddings API
25    const input = responseJson.rows.map((movie) => `${movie.row.movie} from the movie ${movie.row.quote}`);
26    const embeddings = await connections.openai.embeddings.create({
27      input,
28      model: "text-embedding-ada-002",
29    });
30    // append embeddings to movies
31    embeddings.data.forEach((movieEmbedding, i) => {
32      movies[i].embedding = movieEmbedding.embedding;
33    });
34
35    // use the internal API to bulk create movie records
36    await api.internal.movie.bulkCreate(movies);
37  }
38}

This code:

• uses fetch to pull in a small sample dataset that stores movie quotes hosted on Hugging Face
• loops through the returned data and creates a new movie record for each movie quotes
• uses the OpenAI connection to generate embeddings for each movie quote
• uses your Gadget app's internal API to bulk create movie records with the generated embeddings

Now we can run our global action to ingest the data:

5. Click on the Run Action button to open your action in the API Playground
6. Run the action

The action will be run and we see a success message once data has been pulled in from the dataset. We can also see the data in our Gadget database by:

7. Clicking on the movie model in the sidebar
8. Clicking on Data

You should see movie records, complete with title, quote, and embedding data!

Now that we have data in our database, we are ready to build the user-facing portion of our app.

Step 3: Use a global action to find similar movie quotes 

Our app will allow users to enter some text and will find movie quotes that are similar to the entered text using a similarity search on the embeddings. We will use a global action to find the top 4 most similar movie quotes, and then present these movies to the user.

We can create a new global action that we will call from our frontend:

1. Click on Global Actions in the sidebar
2. Click the + next to the ACTIONS section title to create a new global action
3. Name action's API Identifier to findSimilarMovies
4. Enter the following code in the generated code file (replace the entire file):

globalActions/findSimilarMovies.js
js
Copy
1import { FindSimilarMoviesGlobalActionContext } from "gadget-server";
2
3/**
4 * @param { FindSimilarMoviesGlobalActionContext } context
5 */
6export async function run({ params, logger, api, connections, scope }) {
7  const { quote } = params;
8
9  // create an embedding from the entered quote
10  const response = await connections.openai.embeddings.create({ input: quote, model: "text-embedding-ada-002" });
11
12  // get the 4 most similar movies that match your quote, and return them to the frontend!
13  const movies = await api.movie.findMany({
14    sort: {
15      embedding: {
16        cosineSimilarityTo: response.data[0].embedding,
17      },
18    },
19    first: 4,
20  });
21
22  // remove duplicates!
23  const filteredMovies = movies.filter((movie, index) => movies.findIndex((m) => m.title === movie.title) === index);
24
25  // add the movies to the global action's response
26  scope.result = filteredMovies;
27}
28
29// define custom params to pass values to your global action
30module.exports.params = {
31  quote: { type: "string" },
32};

This code creates an embedding from the text passed in by the user and then finds the 4 most similar movies to the entered text. We then remove any duplicate movies and add the movies to the global action's response.

In our global action, we define a custom param at the bottom of the code snippet so the user's quote can be passed in as a string through the params object.

globalActions/findSimilarMovies.js
js
Copy
// define custom params to pass values to your global action
module.exports.params = {
  quote: { type: "string" },
};

To return values from a global action in Gadget, we need to set the scope.result variable to the value we want to return. In this case, we want to return the movies that are similar to the user's entered text:

globalActions/findSimilarMovies.js
js
Copy
// add the movies to the global action's response
scope.result = filteredMovies;

Finding similar vectors with cosine similarity 

This piece of code, included in the snippet above, is the key to finding similar movies:

globalActions/findSimilarMovies.js
js
Copy
1const movies = await api.movie.findMany({
2  sort: {
3    embedding: {
4      cosineSimilarityTo: response.data[0].embedding,
5    },
6  },
7  first: 4,
8});

Gadget has built-in vector distance sorting which we use to get the most similar vectors to the user's entered text. We use the cosineSimilarityTo operator to find the cosine similarity between the user's entered text and the movie quotes in our database.

Build a UI to display the results 

Now let's start building our frontend to call this global action and display the recommended movies to the user.

We're going to use the Chakra UI library to style our frontend. We can install it by running the following commands:

1. Open the Gadget command palette with P or Ctrl P
2. Type > to enable terminal mode
3. Run the following command:

yarn
Copy
yarn add @chakra-ui/react @emotion/react @emotion/styled framer-motion

Once installation is complete, we can start building our UI:

4. Open the frontend/main.jsx file
5. Import the ChakraProvider component at the top of the file:

frontend/main.jsx
jsx
Copy
import { ChakraProvider } from "@chakra-ui/react";

6. Wrap the <App /> component with the ChakraProvider:

frontend/main.jsx
jsx
Copy
1ReactDOM.createRoot(root).render(
2  <React.StrictMode>
3    <ChakraProvider>
4      <Provider api={api} auth={{ signOutActionApiIdentifier: "signOut", signInPath: "/" }}>
5        <App />
6      </Provider>
7    </ChakraProvider>
8  </React.StrictMode>
9);

7. Replace the contents of frontend/routes/index.jsx with the following code (replace the entire file):

frontend/routes/index.jsx
jsx
Copy
1import { useGlobalAction } from "@gadgetinc/react";
2import { api } from "../api";
3import { useCallback, useState } from "react";
4import {
5  Box,
6  Button,
7  Container,
8  Input,
9  Flex,
10  FormControl,
11  FormLabel,
12  Heading,
13  Radio,
14  RadioGroup,
15  Spinner,
16  Stack,
17  Text,
18} from "@chakra-ui/react";
19
20export default function () {
21  const [quote, setQuote] = useState("");
22  const [movie, setMovie] = useState("");
23
24  const [{ data, fetching, error }, findSimilar] = useGlobalAction(api.findSimilarMovies);
25
26  const submitQuote = useCallback(
27    (e) => {
28      e.preventDefault();
29      void findSimilar({ quote });
30    },
31    [quote]
32  );
33
34  const pickMovie = useCallback((movie) => {
35    setMovie(movie);
36  }, []);
37
38  return (
39    <Container maxH="2xl" maxW="5xl" py="15">
40      <Box pb="8">
41        <Heading py="1">Make-a-movie scene (with AI)</Heading>
42        <Text>Write a fake movie quote, pick a suggested movie, and let OpenAI generate a new scene!</Text>
43      </Box>
44
45      <FormControl>
46        <form onSubmit={submitQuote}>
47          <Flex gap="2">
48            <Input placeholder="Enter a fake movie quote!" value={quote} onChange={(event) => setQuote(event.target.value)} />
49            <Button type="submit">Send</Button>
50          </Flex>
51        </form>
52      </FormControl>
53
54      {error && <Text color="red.500">There was an error fetching similar movies. Check your Gadget logs for more details!</Text>}
55      {fetching && <Spinner />}
56
57      {data && (
58        <FormControl>
59          <FormLabel>Movies with similar quotes:</FormLabel>
60          <form>
61            <Flex direction="column" gap="2">
62              <RadioGroup onChange={pickMovie} value={movie}>
63                <Stack direction="row">
64                  {data?.map((movie, i) => (
65                    <Radio key={`movie_option_${i}`} value={movie.title}>
66                      {movie.title}
67                    </Radio>
68                  ))}
69                </Stack>
70              </RadioGroup>
71            </Flex>
72          </form>
73        </FormControl>
74      )}
75    </Container>
76  );
77}

Most of this code is just normal React! The only Gadget-specific code is the useGlobalAction hook, which we use to call our global action. Let's take a closer look at how it works:

frontend/routes/index.jsx
jsx
Copy
const [{ data, fetching, error }, findSimilar] = useGlobalAction(api.findSimilarMovies);

We provide our findSimilarMovies global action as input to the useGlobalAction hook. The imported api object, which is your automatically-created Gadget app's API client, is how we can access our Gadget app API from the frontend.

The useGlobalAction hook returns a response object containing:

• data - the data returned from running the action,
• fetching - a boolean that is true when the action is running
• error - an object that details any errors that occurred while running the action

These objects can be used to display returned data, a loading spinner, error banners or messages, when building your UI.

The findSimilar function returned by the hook is what we call to run the action. We pass in the quote as input to the action, and the data returned from the useGlobalAction hook is an array of movies with similar quotes.

frontend/routes/index.jsx
jsx
Copy
1const submitQuote = useCallback(
2  (e) => {
3    e.preventDefault();
4    void findSimilar({ quote }); // this will call the action!
5  },
6  [quote]
7);

Update permissions 

By default, newly added actions are not callable by just anyone. We need to grant unauthenticated users permission to call findSimilarMovies.

1. Click on Settings in the left sidebar
2. Click on Roles & Permissions
3. Check the box next to findSimilarMovies under the unauthenticated role

You should now be able to enter a quote and see similar movies! Let's test it out.

Test out the frontend 

To view our development app:

1. Click on the app name at the top of the left sidebar
2. Hover over Go to app and click Development

Try it out! Enter a (fake) movie quote and movies with 'similar' quotes (as decided by our vector similarity search) should be returned.

We're at the last step: let's use OpenAI to generate fake movie scenes using our quote and the selected movie. We can also add some chatbot capabilities to our app so that we can interact with the generated scene.

Step 4: Add a route to generate a scene 

Now for the final step in our development process: adding an HTTP route to our Gadget app that will be called by the frontend to generate a scene. We'll use the Vercel AI SDK to make it easy to handle the array of chat responses in our frontend.

1. Start by modifying the routes/POST-chat.js HTTP route file in your Gadget app (replace the entire file):

routes/POST-chat.js
js
Copy
1import { openAIResponseStream } from "gadget-server/ai";
2
3/**
4 * Route handler for POST chat
5 *
6 * @param { import("gadget-server").RouteContext } request context - Everything for handling this route, like the api client, Fastify request, Fastify reply, etc. More on effect context: https://docs.gadget.dev/guides/extending-with-code#effect-context
7 *
8 * @see {@link https://www.fastify.dev/docs/latest/Reference/Request}
9 * @see {@link https://www.fastify.dev/docs/latest/Reference/Reply}
10 */
11export default async function route({ request, reply, api, logger, connections }) {
12  const { messages } = request.body;
13
14  // use the OpenAI connection to create a new chat completion (and write a movie scene!)
15  const stream = await connections.openai.chat.completions.create({
16    model: "gpt-3.5-turbo",
17    messages,
18    stream: true,
19  });
20
21  await reply.send(openAIResponseStream(stream));
22}
23
24route.options = {
25  schema: {
26    body: {
27      type: "object",
28      properties: {
29        messages: {
30          type: "array",
31        },
32      },
33      required: ["messages"],
34    },
35  },
36};

This modifies the sample POST route for our app at /chat.

A custom queryparam is defined in the schema object. This queryparam is used to pass the messages array to the route. The messages array contains the prompt and the generated scene.

We reply using Gadget's openAiResponseStream helper which assists with streaming responses that are returned from the OpenAI connection.

Update the UI to add chatbot functionality 

Now that we have defined our route, we can add support to call this route from the frontend. First, install the Vercel AI SDK npm package:

1. Open the Gadget command palette with P or Ctrl P
2. Type > to enable terminal mode
3. Run the following command:

yarn
Copy
yarn add ai

4. Once again, paste the following code into frontend/routes/index.jsx:

frontend/routes/index.jsx
jsx
Copy
1import { useChat } from "ai/react";
2import { useGlobalAction } from "@gadgetinc/react";
3import { api } from "../api";
4import { useCallback, useState } from "react";
5import {
6  Box,
7  Button,
8  Card,
9  CardBody,
10  CardFooter,
11  Container,
12  Input,
13  Flex,
14  FormControl,
15  FormLabel,
16  Heading,
17  Radio,
18  RadioGroup,
19  Spinner,
20  Stack,
21  StackDivider,
22  Text,
23} from "@chakra-ui/react";
24
25export default function () {
26  const [quote, setQuote] = useState("");
27  const [notes, setNotes] = useState("");
28  const [movie, setMovie] = useState("");
29
30  const { messages, setMessages, setInput, handleSubmit } = useChat({ api: "/chat", headers: { "content-type": "application/json" } });
31  const [{ data, fetching, error }, findSimilar] = useGlobalAction(api.findSimilarMovies);
32
33  const submitQuote = useCallback(
34    (e) => {
35      e.preventDefault();
36      void findSimilar({ quote });
37      setMessages([]);
38    },
39    [quote]
40  );
41
42  const pickMovie = useCallback(
43    (movie) => {
44      setMovie(movie);
45      setInput(
46        `Here is a fake movie quote: "${quote}" and a movie selected by a user: "${movie}". Can you write a fake scene for that movie that makes use of the quote? Use a maximum of 150 words.`
47      );
48    },
49    [quote]
50  );
51
52  return (
53    <Container maxH="2xl" maxW="5xl" py="15">
54      <Box pb="8">
55        <Heading py="1">Make-a-movie scene (with AI)</Heading>
56        <Text>Write a fake movie quote, pick a suggested movie, and let OpenAI generate a new scene!</Text>
57      </Box>
58
59      <FormControl>
60        <form onSubmit={submitQuote}>
61          <Flex gap="2">
62            <Input placeholder="Enter a fake movie quote!" value={quote} onChange={(event) => setQuote(event.target.value)} />
63            <Button type="submit">Send</Button>
64          </Flex>
65        </form>
66      </FormControl>
67
68      {error && <Text color="red.500">There was an error fetching similar movies. Check your Gadget logs for more details!</Text>}
69      {fetching && <Spinner />}
70
71      {data && (
72        <FormControl>
73          <FormLabel>Movies with similar quotes:</FormLabel>
74          <form onSubmit={handleSubmit}>
75            <Flex direction="column" gap="2">
76              <RadioGroup onChange={pickMovie} value={movie}>
77                <Stack direction="row">
78                  {data?.map((movie, i) => (
79                    <Radio key={`movie_option_${i}`} value={movie.title}>
80                      {movie.title}
81                    </Radio>
82                  ))}
83                </Stack>
84              </RadioGroup>
85              <Button type="submit" isDisabled={!movie}>
86                Generate scene
87              </Button>
88            </Flex>
89          </form>
90        </FormControl>
91      )}
92
93      {messages?.length > 0 && (
94        <Card maxH="md" overflowY="scroll">
95          <CardBody>
96            <Stack divider={<StackDivider />} spacing="4">
97              {messages.map((m) => (
98                <Box key={m.id}>
99                  <Heading size="xs" textTransform="uppercase">
100                    {m.role === "user" ? "You" : "AI Screenwriter"}
101                  </Heading>
102                  <Text textAlign="left">{m.content}</Text>
103                </Box>
104              ))}
105            </Stack>
106          </CardBody>
107          <CardFooter>
108            <FormControl>
109              <form
110                onSubmit={(e) => {
111                  handleSubmit(e);
112                  setNotes("");
113                }}
114              >
115                <Flex gap="2">
116                  <Input
117                    placeholder="Enter your editor's notes"
118                    value={notes}
119                    onChange={(event) => {
120                      const notes = event.target.value;
121                      setNotes(notes);
122                      setInput(notes);
123                    }}
124                  />
125                  <Button type="submit">Send</Button>
126                </Flex>
127              </form>
128            </FormControl>
129          </CardFooter>
130        </Card>
131      )}
132    </Container>
133  );
134}

The useChat hook is imported from Vercel's AI SDK (ai/react) and sends a request to our /chat route when handleSubmit is called.

frontend/Screenwriter.jsx
jsx
Copy
const { messages, setMessages, setInput, handleSubmit } = useChat({ api: "/chat", headers: { "content-type": "application/json" } });

The useChat hook also provides us with:

• a messages array which we can use to display the chat history
• a setMessages function which we can use to send messages to the chatbot
• a setInput function we use to seed the initial prompt passed into our route

For more details on Vercel's AI SDK, check out the documentation.

None of this added code is Gadget-specific! It is, once again, plain old React, making use of different libraries and packages (Chakra UI and Vercel's AI SDK) to build a simple chatbot.

Test your chatbot 

We are done building! Let's test out the AI screenwriter. Enter some text, select a movie, and watch as the AI screenwriter generates a new scene! You can also enter some editor's notes to give the AI screenwriter some feedback.

The final step is deploying to production.

Step 5 (Optional): Deploy to Production 

If you want to deploy a Production version of your app, you can do so in just a couple of clicks!

First, you need to use your own OpenAI API key in the OpenAI connection:

1. Click on the Connections tab in the left sidebar
2. Click on the OpenAI connection
3. Edit the connection and use your API key for the Production environment

Now, deploy your app to Production:

4. Click on the Deploy button in the bottom right corner of the Gadget UI
5. Click Deploy Changes

That's it! Your app will be built, optimized, and deployed!

You can preview your Production app:

6. Click on the app name at the top of the left sidebar
7. Hover over Go to app and click Production

Alternatively, you can remove --development from the domain of the window you were using to preview your frontend changes while developing.

Next steps 

Congrats! You've built a fullstack web app that makes use of generative AI and vector embeddings! 🎉

In this tutorial, we learned:

• How to create and store vector fields in Gadget
• How to stream chat responses from OpenAI to a Gadget frontend using Vercel's AI SDK
• When to use global actions vs routes in Gadget

This app also contains Gadget's built-in authentication helpers, which we did not cover in this tutorial. If you want to learn more about auth in Gadget, check out the tutorial:

Questions? 

If you have any questions, feel free to reach out to us on Discord to ask Gadget employees or the Gadget developer community!