Runtime environment
Gadget automatically manages a serverless runtime environment for your application. This environment is a Linux-based virtual machine that runs vanilla Node.js, and is optimized for running server-side Gadget code. Gadget manages all aspects of the environment, including starting and stopping workers, security upgrades, deployment, and handling requests.
Your application's runtime environment is used by only your application, and is securely isolated from other applications on the platform.
Environment details
Your application runs in a Debian Linux environment, using an unmodified version of Node.js. The runtime environment has a variety of common system packages installed at fixed versions, including:
- Node.js itself
- npm
- yarn
- Google Chromium (and the dependencies necessary to run it)
- Firefox ESR (and the dependencies necessary to run it)
- Common linux utilities including curl, vim, nano, htop, iotop, lsof, ping
The specific versions of these packages depend on which Gadget framework version your application is using. See the Gadget framework guide for exact details.
If you'd like to see new tools added to the runtime environment, please join the Gadget Discord and we'll be happy to discuss your needs.
Request processing
After request validation and authorization, Gadget routes requests to your application's runtime environment for processing. Your application's worker then processes the request, and sends the response back to the user via Gadget. If your worker crashes during request processing, Gadget will respond on your behalf with a 500 error, and restart the worker to resume processing further requests.
Request and worker isolation
Individual requests are run on a small, shared set of workers for your application. Each worker will process many requests simultaneously within the same Node.js process, as is Node.js best practice. Each request does not get assigned its own worker, and the same global state will be re-used between requests. Requests are load balanced across all available workers to ensure even distribution of load, and there's no way to control which worker a request is run on.
Because requests are run within the same Node.js process, the same global state will be re-used between requests, you must be careful to not accidentally leak state between requests. Using global variables is a common way to leak state between requests, so be careful to avoid it.
Individual workers are run on independent, isolated virtual machines that don't share files or memory with the other workers. If you write a file to the filesystem, it will only be visible to that one worker, and not other applications on the platform. If you need to persist some data between requests, you should your application's built in database or file storage to do so.
Workers for different environments of the same application are fully isolated from each other, and cannot access each other's data.
Worker lifecycle
When your application receives a request that needs a worker, Gadget automatically starts a serverless worker to process the request if there aren't any available, and then sends the request to the worker. The worker runs for as long as there are requests that need processing. After a cool-down period of receiving no new requests, the worker is stopped gracefully. Workers are considered ephemeral such that they can be started and gracefully stopped at any time.
Gadget monitors worker load by measuring concurrent requests and CPU usage, and will start more workers if there are too many requests, or if CPU usage rises above a 75% to spread out the load. If your workers are processing too few requests, or have too little CPU usage, Gadget will slowly and gracefully scale down the number of workers to reduce resource usage until the load is balanced.
Occasionally, Gadget may also choose to start and stop workers to deploy new versions of the base environment, or to recycle the underlying hardware out of the Gadget fleet. This is a non-disruptive operation, where Gadget will gracefully shutdown some workers, and start new ones in their place elsewhere.
Worker health
Gadget monitors three health metrics for each of your workers and shows them on the Operations dashboard. Read more about each metric in the Operations dashboard section.
For CPU usage, Gadget monitors the metric and will scale your worker count up or down as needed to keep CPU usage below 75%.
For event loop utilization, Gadget monitors the metric, but doesn't take action based on it.
For memory usage, Gadget monitors the metric, but doesn't take action based on it.
Worker crashes
If your worker crashes during request processing, Gadget will respond on your behalf with a 500 error, and restart the worker to resume processing further requests.
Crashes can happen for a variety of reasons, including:
- Your application's code throws an unhandled exception or an unhandled promise rejection
- Your worker runs out of memory and Node.js throws an Out of Memory error
Note that when a worker crashes, all requests that were being processed by that worker will fail with a 500 error. This can make debugging request errors difficult, as failed requests might be the cause of a crash, or just an innocent victim of a crash caused by a different request running on the same worker. This is sadly a fact of life when working with event-loop based systems like Node.js that process multiple requests simultaneously in the same process. For debugging these crashes, Gadget recommends reviewing all the logs for a worker that crashed in time order to try to identify what went wrong when, instead of reviewing logs on a request-by-request basis.
Worker-less requests
Some requests to your application don't need a worker to be processed, and can instead be handled entirely by Gadget's services. This includes requests to your application's Internal API, requests for CDN assets, and read requests to your application's Public API. Avoiding workers for these requests allows Gadget to reduce costs and improves performance by avoiding any cold starts.