Use Gadget and OpenAI to build a chatbot that generates custom movie scenes
Topics covered: AI + vector embeddings, Actions, HTTP routes, React frontends
Time to build: ~30 minutes
Learn about Gadget's built-in AI features such as the OpenAI connection, vector databases, and cosine similarity search, and use them to build a chatbot that generates custom movie scenes.
Create a new Gadget app
Before we get started we need to create a new Gadget app. We can do this at gadget.new. When selecting an app template, make sure you select the Web app type.
Now that we have a new Gadget app, let's start building!
Step 1: Create a movie model
The first thing we need to do is store some movie quotes in our Gadget app. We're going to make use of Gadget's data models, which are similar to tables in a Postgres database, to store this information.
Start by creating a new model in Gadget:
- Click the + button in the
api/models
folder of the sidebar - Enter
movie
as the model's API identifier
Now add some fields to your model. Fields are similar to columns in a database table, and allow you to define what kind of data is stored in your model. For our movie model, we'll add the following fields:
- Select schema in the
api/models/movie
folder - Click + in the FIELDS section
- Enter
title
as the field's API identifier - Click on the + Add Validations drop-down and select Required to make the
title
field Required
Adding a Required validation to title
means that an error will be thrown if a movie is added without a title. Now let's add a field to store the movie's quotes:
- Select schema in the
movie
model's folder - Click + in the FIELDS section
- Enter
quote
as the field's API identifier - Click on the + Add Validations drop-down and select Required to make the
quote
field Required
Now we have a place to store movie quotes! We also need a field used to store vector embeddings. Vector embeddings are a way of representing text as a vector of numbers. To learn more about vector embeddings, check out our docs on building AI apps.
- Select schema in the
movie
model's folder - Click + in the FIELDS section
- Enter
embedding
as the field's API identifier - Select vector as the field's type
That is all that we need to store data for our app! Now we need a way to generate embeddings. Luckily, OpenAI has an API that we can use to pass in text and get back a vector embedding.
Step 2: Add the OpenAI connection
Gadget has built-in connections to popular APIs, including OpenAI. You can use these connections to interact with external services in your app.
- Click on Settings in the sidebar
- Click on Plugins
- Select OpenAI from the list of plugins
- Use the Gadget development keys so you can start using the OpenAI API without an API key
- Click Add connection
You can now use the OpenAI connection in your app.
Step 3: Data ingestion
We need some test data for our app. We're going to use a global action to fetch an open data source hosted on Hugging Face. We will then use the OpenAI connection to generate embeddings for our movie quotes.
- Click the + next to the
api/actions
folder to create a new global action - Name the action file
ingestData.js
Our OpenAI connection is already set up for us using Gadget-managed credentials. To learn more about how to set up your own OpenAI connection, check out our OpenAI connection docs.
Teams in Gadget get free OpenAI credits to use for experimenting during development! Using Gadget-managed OpenAI credentials automatically draws from this credit pool.
- Enter the following code in the generated code file (replace the entire file):
1type Dataset = {2 rows: {3 row: {4 quote: string;5 movie: string;6 type: string;7 year: number;8 };9 }[];10};1112export const run: ActionRun = async ({ logger, api, connections }) => {13 // use Node's fetch to make a request to https://huggingface.co/datasets/ygorgeurts/movie-quotes14 const response = await fetch(15 "https://datasets-server.huggingface.co/first-rows?dataset=ygorgeurts%2Fmovie-quotes&config=default&split=train",16 {17 method: "GET",18 headers: { "Content-Type": "application/json" },19 }20 );2122 const responseJson: Dataset = await response.json();2324 // log the response25 logger.info(26 { responseJson },27 "here is a sample of movies returned from hugging face"28 );2930 // get the data in our record's format31 const movies = responseJson.rows.map((movie) => ({32 title: movie.row.movie,33 quote: movie.row.quote,34 embedding: [] as number[],35 }));3637 // also get input data for the OpenAI embeddings API38 const input = responseJson.rows.map(39 (movie) => `${movie.row.movie} from the movie ${movie.row.quote}`40 );41 const embeddings = await connections.openai.embeddings.create({42 input,43 model: "text-embedding-ada-002",44 });4546 // append embeddings to movies47 embeddings.data.forEach((movieEmbedding, i) => {48 movies[i].embedding = movieEmbedding.embedding;49 });5051 // use the internal API to bulk create movie records52 await api.internal.movie.bulkCreate(movies);53};
1type Dataset = {2 rows: {3 row: {4 quote: string;5 movie: string;6 type: string;7 year: number;8 };9 }[];10};1112export const run: ActionRun = async ({ logger, api, connections }) => {13 // use Node's fetch to make a request to https://huggingface.co/datasets/ygorgeurts/movie-quotes14 const response = await fetch(15 "https://datasets-server.huggingface.co/first-rows?dataset=ygorgeurts%2Fmovie-quotes&config=default&split=train",16 {17 method: "GET",18 headers: { "Content-Type": "application/json" },19 }20 );2122 const responseJson: Dataset = await response.json();2324 // log the response25 logger.info(26 { responseJson },27 "here is a sample of movies returned from hugging face"28 );2930 // get the data in our record's format31 const movies = responseJson.rows.map((movie) => ({32 title: movie.row.movie,33 quote: movie.row.quote,34 embedding: [] as number[],35 }));3637 // also get input data for the OpenAI embeddings API38 const input = responseJson.rows.map(39 (movie) => `${movie.row.movie} from the movie ${movie.row.quote}`40 );41 const embeddings = await connections.openai.embeddings.create({42 input,43 model: "text-embedding-ada-002",44 });4546 // append embeddings to movies47 embeddings.data.forEach((movieEmbedding, i) => {48 movies[i].embedding = movieEmbedding.embedding;49 });5051 // use the internal API to bulk create movie records52 await api.internal.movie.bulkCreate(movies);53};
This code:
- uses
fetch
to pull in a small sample dataset that stores movie quotes hosted on Hugging Face - loops through the returned data and creates a new
movie
record for each movie quotes - uses the OpenAI connection (
connections.openai
) to generate embeddings for each movie quote - uses your Gadget app's internal API to bulk create
movie
records with the generated embeddings
Now we can run our global action to ingest the data:
- Click on the Run Action button to open your action in the API Playground
- Run the action
The action will be run and a success message is returned once data has been added to the database.
We can also see the data in our Gadget database by:
- Clicking on
api/models/movie/data
You should see movie
records, complete with title, quote, and embedding data!
Now that we have data in our database, we are ready to build the user-facing portion of our app.
Step 4: Use a global action to find similar movie quotes
Our app will allow users to enter a fake movie quote and find movie quotes that are similar to the entered text using a similarity search on the embeddings. We will use a global action to find the top 4 most similar movie quotes, and then present these movies to the user.
We can create a new global action:
- Click the + next to the
api/actions
folder to create a new global action - Name the action file
findSimilarMovies.js
- Enter the following code in the generated code file (replace the entire file):
1export const run: ActionRun = async ({ params, api, connections }) => {2 const { quote } = params;34 // throw an error if a quote wasn't passed in5 if (!quote) {6 throw new Error("Missing quote!");7 }89 // create an embedding from the entered quote10 const response = await connections.openai.embeddings.create({11 input: quote,12 model: "text-embedding-ada-002",13 });1415 // get the 4 most similar movies that match your quote, and return them to the frontend16 const movies = await api.movie.findMany({17 sort: {18 embedding: {19 cosineSimilarityTo: response.data[0].embedding,20 },21 },22 first: 4,23 select: {24 id: true,25 title: true,26 },27 });2829 // remove duplicates30 const filteredMovies = movies.filter(31 (movie, index) => movies.findIndex((m) => m.title === movie.title) === index32 );33 return filteredMovies;34};3536// define custom params to pass values to your global action37export const params = {38 quote: { type: "string" },39};
1export const run: ActionRun = async ({ params, api, connections }) => {2 const { quote } = params;34 // throw an error if a quote wasn't passed in5 if (!quote) {6 throw new Error("Missing quote!");7 }89 // create an embedding from the entered quote10 const response = await connections.openai.embeddings.create({11 input: quote,12 model: "text-embedding-ada-002",13 });1415 // get the 4 most similar movies that match your quote, and return them to the frontend16 const movies = await api.movie.findMany({17 sort: {18 embedding: {19 cosineSimilarityTo: response.data[0].embedding,20 },21 },22 first: 4,23 select: {24 id: true,25 title: true,26 },27 });2829 // remove duplicates30 const filteredMovies = movies.filter(31 (movie, index) => movies.findIndex((m) => m.title === movie.title) === index32 );33 return filteredMovies;34};3536// define custom params to pass values to your global action37export const params = {38 quote: { type: "string" },39};
Finding similar vectors with cosine similarity
This api.movie.findMany
call from the above function is the key to finding similar movies:
1// get the 4 most similar movies that match your quote, and return them to the frontend2const movies = await api.movie.findMany({3 sort: {4 embedding: {5 cosineSimilarityTo: response.data[0].embedding,6 },7 },8 first: 4,9 select: {10 id: true,11 title: true,12 },13});
1// get the 4 most similar movies that match your quote, and return them to the frontend2const movies = await api.movie.findMany({3 sort: {4 embedding: {5 cosineSimilarityTo: response.data[0].embedding,6 },7 },8 first: 4,9 select: {10 id: true,11 title: true,12 },13});
Gadget has built-in vector distance sorting which we use to get the most similar vectors to the user's entered text. We use the cosineSimilarityTo
operator to find the cosine similarity between the user's entered text and the movie quotes in our database.
Step 5: Add a route to generate a scene
Now for the final backend development step: adding an HTTP route to our Gadget app that will be called by the frontend to generate a scene. We make use of Gadget's OpenAI connection to generate a scene using the user's entered text and a movie quote.
We used a global action to ingest data and find similar movies, but we're using a route to generate a scene. You might be asking yourself why?
There are two main reasons:
- Global actions do not support streaming responses, and we want to stream the text returned from OpenAI to the frontend
- The
openAIResponseStream
helper we are using integrates seamlessly with HTTP routes
In general, we suggest you use global actions over HTTP routes whenever possible. But when streaming or integrating with external systems or packages, HTTP routes can be a better choice. To read more about when to use each, see the Actions guide.
- Add a
api/routes/POST-chat.js
HTTP route file to your app and paste the following code:
1import { RouteHandler } from "gadget-server";2import { openAIResponseStream } from "gadget-server/ai";34/**5 * Route handler for POST chat6 *7 * See: https://docs.gadget.dev/guides/http-routes/route-configuration#route-context8 */9const route: RouteHandler<{10 Body: { quote: string; movie: string };11}> = async ({ request, reply, api, logger, connections }) => {12 const prompt = `Here is a fake movie quote: "${request.body.quote}" and a movie selected by a user: "${request.body.movie}". Write a fake scene for that movie that makes use of the quote. Use a maximum of 150 words.`;1314 // get streamed response from OpenAI15 const stream = await connections.openai.chat.completions.create({16 model: "gpt-3.5-turbo",17 messages: [18 {19 role: "system",20 content: `You are an expert, hilarious AI screenwriter tasked with generating funny, quirky movie scripts.`,21 },22 { role: "user", content: prompt },23 ],24 stream: true,25 });2627 await reply.send(openAIResponseStream(stream));28};2930export default route;
1import { RouteHandler } from "gadget-server";2import { openAIResponseStream } from "gadget-server/ai";34/**5 * Route handler for POST chat6 *7 * See: https://docs.gadget.dev/guides/http-routes/route-configuration#route-context8 */9const route: RouteHandler<{10 Body: { quote: string; movie: string };11}> = async ({ request, reply, api, logger, connections }) => {12 const prompt = `Here is a fake movie quote: "${request.body.quote}" and a movie selected by a user: "${request.body.movie}". Write a fake scene for that movie that makes use of the quote. Use a maximum of 150 words.`;1314 // get streamed response from OpenAI15 const stream = await connections.openai.chat.completions.create({16 model: "gpt-3.5-turbo",17 messages: [18 {19 role: "system",20 content: `You are an expert, hilarious AI screenwriter tasked with generating funny, quirky movie scripts.`,21 },22 { role: "user", content: prompt },23 ],24 stream: true,25 });2627 await reply.send(openAIResponseStream(stream));28};2930export default route;
The OpenAI connection is used to call the chat completions endpoint, which generates a scene from the user's selected movie and entered quote.
Now we can call this route from our frontend to generate a scene!
Step 6: Build the frontend
Now that we have defined our global actions and HTTP route, we can add support to call them from the frontend.
Gadget's React frontends are built on top of Vite, and include support for email/password auth as well as Google Auth. Our frontend code lives in the web
folder. We will only be making changes to a single frontend route, web/routes/signed-in.jsx
, which is the route accessed when a user is signed in to our app.
- Paste the following code into
web/routes/signed-in.jsx
:
1import { useGlobalAction, useFetch, useMaybeFindFirst, useActionForm } from "@gadgetinc/react";2import { api } from "../api";3import { useState } from "react";45export default function () {6 // see if movie data exists in the database!7 const [{ data: movieDataIngested, fetching: checkingIfMovieDataExists, error: errorCheckingForData }, retry] = useMaybeFindFirst(8 api.movie9 );10 // fire action used to ingest data from HuggingFace model11 const [{ fetching: ingestingData, error: errorIngestingData }, ingestData] = useGlobalAction(api.ingestData);1213 return (14 <main style={{ width: "60vw", position: "absolute", top: 40 }}>15 <div>16 <h1>Make-a-movie scene (with AI)</h1>17 <p>Write a fake movie quote, pick a suggested movie, and let OpenAI generate a new scene!</p>18 </div>1920 <br />2122 {checkingIfMovieDataExists && <div className="loader"></div>}2324 {ingestingData && (25 <div className="row">26 <h2>Fetching movie data...</h2>27 </div>28 )}2930 {errorCheckingForData && <p className="error">{errorCheckingForData.message}</p>}3132 {errorIngestingData && <p className="error">{errorIngestingData.message}</p>}3334 {!movieDataIngested && !checkingIfMovieDataExists && (35 <button36 onClick={async () => {37 await ingestData();38 await retry();39 }}40 disabled={ingestingData}41 >42 Ingest data43 </button>44 )}4546 {movieDataIngested && <MovieQuoteForm />}47 </main>48 );49}5051const MovieQuoteForm = () => {52 // used to track the reset state of the form53 const [isReset, setIsReset] = useState(true);54 // state for the currently selected movie55 const [movie, setMovie] = useState("");56 // action for finding movies with similar quotes57 const { submit, register, actionData, error, formState, watch, reset } = useActionForm(api.findSimilarMovies);58 const similarMovies = actionData as { id: string; title: string }[] | undefined;5960 // watch changes to the quote state in our form, and store in a variable61 const quote = watch("quote");6263 return (64 <>65 <form66 onSubmit={async (e) => {67 e.preventDefault();68 await submit();69 setIsReset(true);70 }}71 style={{ maxWidth: "100%" }}72 autoComplete="off"73 >74 <div className="row" style={{ display: "flex" }}>75 <input76 style={{ flex: "1", marginRight: "4px" }}77 placeholder="Enter a fake movie quote! Ex. Here's a toast to you, my dear."78 {...register("quote")}79 disabled={formState.isSubmitting}80 />81 <button type="submit" disabled={formState.isSubmitting} style={{ marginRight: "4px" }}>82 Find quotes83 </button>84 <button85 disabled={!formState.isDirty}86 onClick={() => {87 // reset the form88 reset();89 setIsReset(false);90 }}91 >92 Reset93 </button>94 </div>95 </form>9697 {error && <p className="error">There was an error fetching similar movies. Check your Gadget logs for more details!</p>}9899 {quote && similarMovies && isReset && (100 <>101 <div className="row" style={{ textAlign: "left", paddingBottom: 8 }}>102 <b>Movies with similar quotes:</b>103 </div>104 <form>105 <div106 className="row"107 style={{108 display: "flex",109 flexWrap: "wrap",110 textAlign: "left",111 gap: 16,112 }}113 >114 {similarMovies.map((movieRecord, i) => (115 <span key={`movie_option_${i}`}>116 <input117 type="radio"118 checked={movieRecord.title == movie}119 value={movieRecord.title}120 onChange={(e) => setMovie(e.target.value)}121 id={movieRecord.id}122 />123 <label htmlFor={movieRecord.id}>{movieRecord.title}</label>124 </span>125 ))}126 </div>127 </form>128 </>129 )}130 {movie && quote && isReset && <SceneGenerator movie={movie} quote={quote} />}131 </>132 );133};134135const SceneGenerator = ({ movie, quote }: { movie: string; quote: string }) => {136 // call HTTP route and stream response from OpenAI137 const [{ data, fetching, error }, sendPrompt] = useFetch("/chat", {138 method: "post",139 body: JSON.stringify({ movie, quote }),140 headers: {141 "content-type": "application/json",142 },143 stream: "string",144 });145146 return (147 <section>148 <button onClick={() => void sendPrompt()}>Generate scene</button>149 {error && <p className="error">{error.message}</p>}150 {fetching && <div className="loader" />}151 {data && (152 <pre153 style={{154 border: "dashed black 1px",155 background: "#f5f5f5",156 padding: "10px",157 maxHeight: "45vh",158 overflowY: "scroll",159 whiteSpace: "pre-wrap",160 textAlign: "left",161 }}162 >163 {data}164 </pre>165 )}166 </section>167 );168};
1import { useGlobalAction, useFetch, useMaybeFindFirst, useActionForm } from "@gadgetinc/react";2import { api } from "../api";3import { useState } from "react";45export default function () {6 // see if movie data exists in the database!7 const [{ data: movieDataIngested, fetching: checkingIfMovieDataExists, error: errorCheckingForData }, retry] = useMaybeFindFirst(8 api.movie9 );10 // fire action used to ingest data from HuggingFace model11 const [{ fetching: ingestingData, error: errorIngestingData }, ingestData] = useGlobalAction(api.ingestData);1213 return (14 <main style={{ width: "60vw", position: "absolute", top: 40 }}>15 <div>16 <h1>Make-a-movie scene (with AI)</h1>17 <p>Write a fake movie quote, pick a suggested movie, and let OpenAI generate a new scene!</p>18 </div>1920 <br />2122 {checkingIfMovieDataExists && <div className="loader"></div>}2324 {ingestingData && (25 <div className="row">26 <h2>Fetching movie data...</h2>27 </div>28 )}2930 {errorCheckingForData && <p className="error">{errorCheckingForData.message}</p>}3132 {errorIngestingData && <p className="error">{errorIngestingData.message}</p>}3334 {!movieDataIngested && !checkingIfMovieDataExists && (35 <button36 onClick={async () => {37 await ingestData();38 await retry();39 }}40 disabled={ingestingData}41 >42 Ingest data43 </button>44 )}4546 {movieDataIngested && <MovieQuoteForm />}47 </main>48 );49}5051const MovieQuoteForm = () => {52 // used to track the reset state of the form53 const [isReset, setIsReset] = useState(true);54 // state for the currently selected movie55 const [movie, setMovie] = useState("");56 // action for finding movies with similar quotes57 const { submit, register, actionData, error, formState, watch, reset } = useActionForm(api.findSimilarMovies);58 const similarMovies = actionData as { id: string; title: string }[] | undefined;5960 // watch changes to the quote state in our form, and store in a variable61 const quote = watch("quote");6263 return (64 <>65 <form66 onSubmit={async (e) => {67 e.preventDefault();68 await submit();69 setIsReset(true);70 }}71 style={{ maxWidth: "100%" }}72 autoComplete="off"73 >74 <div className="row" style={{ display: "flex" }}>75 <input76 style={{ flex: "1", marginRight: "4px" }}77 placeholder="Enter a fake movie quote! Ex. Here's a toast to you, my dear."78 {...register("quote")}79 disabled={formState.isSubmitting}80 />81 <button type="submit" disabled={formState.isSubmitting} style={{ marginRight: "4px" }}>82 Find quotes83 </button>84 <button85 disabled={!formState.isDirty}86 onClick={() => {87 // reset the form88 reset();89 setIsReset(false);90 }}91 >92 Reset93 </button>94 </div>95 </form>9697 {error && <p className="error">There was an error fetching similar movies. Check your Gadget logs for more details!</p>}9899 {quote && similarMovies && isReset && (100 <>101 <div className="row" style={{ textAlign: "left", paddingBottom: 8 }}>102 <b>Movies with similar quotes:</b>103 </div>104 <form>105 <div106 className="row"107 style={{108 display: "flex",109 flexWrap: "wrap",110 textAlign: "left",111 gap: 16,112 }}113 >114 {similarMovies.map((movieRecord, i) => (115 <span key={`movie_option_${i}`}>116 <input117 type="radio"118 checked={movieRecord.title == movie}119 value={movieRecord.title}120 onChange={(e) => setMovie(e.target.value)}121 id={movieRecord.id}122 />123 <label htmlFor={movieRecord.id}>{movieRecord.title}</label>124 </span>125 ))}126 </div>127 </form>128 </>129 )}130 {movie && quote && isReset && <SceneGenerator movie={movie} quote={quote} />}131 </>132 );133};134135const SceneGenerator = ({ movie, quote }: { movie: string; quote: string }) => {136 // call HTTP route and stream response from OpenAI137 const [{ data, fetching, error }, sendPrompt] = useFetch("/chat", {138 method: "post",139 body: JSON.stringify({ movie, quote }),140 headers: {141 "content-type": "application/json",142 },143 stream: "string",144 });145146 return (147 <section>148 <button onClick={() => void sendPrompt()}>Generate scene</button>149 {error && <p className="error">{error.message}</p>}150 {fetching && <div className="loader" />}151 {data && (152 <pre153 style={{154 border: "dashed black 1px",155 background: "#f5f5f5",156 padding: "10px",157 maxHeight: "45vh",158 overflowY: "scroll",159 whiteSpace: "pre-wrap",160 textAlign: "left",161 }}162 >163 {data}164 </pre>165 )}166 </section>167 );168};
The frontend has 3 components: the default export
for the route, the MovieQuoteForm
component, and the SceneGenerator
component. These 3 components all make use of different @gadgetinc/react
hooks that help us make requests and manage our form state. The hooks simplify the management of response and form state, and let us interact with responses and forms in a React-ful way through the returned data
, fetching
, and error
objects.
- the route's
default export
is responsible for calling theingestData
global action (if you haven't already done so!) using theuseGlobalAction
hook (more info onuseGlobalAction
) - the
MovieQuoteForm
component manages and submits the input form for the entered quote, and calls thefindSimilarMovies
global action using theuseActionForm
hook (more info onuseActionForm
) which then allows users to select a movie from the returnedactionData
- the
SceneGenerator
component makes a request to the/chat
HTTP route using theuseFetch
hook (more info onuseFetch
) and displays a streamed response
Remove background-image
You can clean up the appearance of your project by removing the background-image
from the .app
CSS class set in web/components/App.css
:
Search for and remove this line from the .app class in web/components/App.csscssbackground-image: url("./assets/default-background.svg");
Test your screenwriter
We are done building! Let's test out the AI screenwriter.
- Click Preview in the top right corner of the Gadget UI
Sign-up and sign-in to your app, enter a fake movie quote, select a recommended movie, and watch as the AI screenwriter generates a new scene!
The final step is deploying to production.
Step 7 (Optional): Deploy to Production
If you want to deploy a Production version of your app, you can do so in just a couple of clicks!
First, you need to use your own OpenAI API key in the OpenAI connection:
- Click on the Settings section in the left sidebar
- Click on the Plugins tab
- Select the OpenAI connection
- Edit the connection and use your API key for the Production environment
Now, deploy your app to Production:
- Click on the Deploy button in the top right corner of the Gadget UI
- Click Deploy Changes
That's it! Your app will be built, optimized, and deployed!
You can preview your Production app:
- Click on the app name at the top of the left sidebar
- Click the environment selector in the left corner and from the dropdown click on Production
Alternatively, you can remove --development
from the domain of the window you were using to preview your frontend changes while developing.
Next steps
Congrats! You've built a full-stack web app that makes use of generative AI and vector embeddings! 🎉
In this tutorial, we learned:
- How to create and store vector fields in Gadget
- How to stream chat responses from OpenAI to a Gadget frontend using Vercel's AI SDK
- When to use global actions vs routes in Gadget
Questions?
If you have any questions, feel free to reach out to us on Discord to ask Gadget employees or the Gadget developer community!