Deploy a model. Get a URL.
Speaks OpenAI.
Point the SDK you already use at your endpoint. Nothing else changes.
Sleeps when traffic does.
Set min workers to 0 and an idle endpoint costs storage, not GPUs.
Any open model.
Name a Hugging Face repo, or an Ollama tag. Gated models take your HF token.
What is a serverless GPU endpoint?
A serverless endpoint is a model with a URL and no machine to think about. You name the model and the GPU; we run a pool of workers that grows when requests queue and shrinks when they stop. At zero traffic the pool can sleep entirely, and while it sleeps you pay for stored weights, not for waiting hardware.
The other two products rent you time on a box. This one rents you an answer to a request. That is why it bills differently: per minute of worker runtime, metered per worker, instead of per minute of a machine you keep.
Three engines, your model.
Bring the model as a Hugging Face repo or an Ollama tag. Custom containers are not part of the beta.
vLLM
Fast LLM inference and serving. The default for chat and completion workloads.
SGLang
Fast serving for large language and vision-language models.
Ollama
Lightweight runner for open LLMs. Name a tag like gemma3:4b and go.
Four steps, then it answers for itself.
Pick a framework and a model
vLLM, SGLang or Ollama; a Hugging Face repo or an Ollama tag.
Shape the pool
GPU type, 1 to 8 GPUs per worker, min and max workers. Min 0 means it can sleep.
Get the URL
OpenAI-compatible, authenticated with your JarvisLabs API token.
Let it breathe
Workers start when requests queue and stop after your idle timeout.
Every knob, in the open.
Serverless platforms tend to hide their scaling behavior. Ours is a form you fill in.
| Knob | Range | What it does |
|---|---|---|
| GPU type | L4 · H100 · H200 · RTX Pro 6000 | One type per deployment |
| GPUs per worker | 1 · 2 · 4 · 8 | Size one worker to fit the model |
| Min workers | 0 to 100 | 0 = the pool may sleep entirely |
| Max workers | 1 to 100 | The ceiling autoscaling can reach |
| Concurrent requests | 1 to 100 (Ollama: 4) | Per worker, before the queue grows |
| Idle timeout | 0s to 24h | How long a quiet worker stays warm |
| Wait time | 0s to 10m | How long a request waits for a worker before failing |
| File storage | 50 GB to 1 TB | Holds the model weights, billed around the clock |
After deploy, name, idle timeout and wait time can change on a running deployment. Changing workers, GPU or model means recreating it.
The bill breathes with the pool.
While a worker runs
Its GPUs bill per minute of runtime, plus the worker's 50 GB root disk at $0.00014/GB/hour. Each worker is metered on its own.
While the pool sleeps
Nothing runs, so no GPU charges. File storage keeps your weights warm at $0.00014/GB/hour, around the clock.
What you will not find
No request fees, no per-token markup, no minimums. The dashboard shows the live running cost per worker as it accrues.
Watch it work.
Live logs
Streamed from every worker, filterable per worker, tail included.
Worker state
Healthy counts, queue depth, model-download progress, and each worker's scale-down countdown.
Running cost
Per-worker cost as it accrues, and the deployment total, on the same screen.
Serving here. Everything else, next door.
The three products share GPUs and a per-minute meter. They differ in what you are renting.
| ServerlessThis page | Template | On-Demand VM | |
|---|---|---|---|
| Built for | Serving requests | Training and notebooks | Owning the machine |
| What you get | An OpenAI-compatible URL | A running environment in 1.8s | Root on a box in about 90s |
| Scales | 0 to 100 workers, automatically | One instance, your call | One machine, your call |
| Bills | Per minute of worker runtime | Per minute while running | Per minute while running |
| Pick this when | you're serving, not training | you want to start training right away | a container is not a machine |
The product is young, not gated. You sign up and deploy from the dashboard today. It also means no SLA yet, three frameworks rather than custom containers, and rates that may change as the product settles.
Yes. It serves the chat-completions API, so the OpenAI SDK works by changing the base URL and using your JarvisLabs API token as the key.
A worker cold-starts. The request waits up to your configured wait time (as long as 10 minutes) for a worker to pick it up, then fails if none does. Large models take longer because the weights load first; the dashboard shows download progress.
With min workers at 0, only file storage, at $0.00014/GB/hour. Workers bill only while running.
L4, H100, H200, RTX Pro 6000, at 1, 2, 4, 8 GPUs per worker.
Yes. Add your Hugging Face token as an environment variable on the deployment and name the repo as usual.
Name, idle timeout and wait time can change on a running deployment. Changing workers, GPU or model means recreating it, which the beta keeps deliberately simple.
India today, in two regions. More regions follow the beta.
Not on Serverless yet. vLLM, SGLang and Ollama are the beta engines. For arbitrary containers, an On-Demand VM is the right box.