BentoML Cheat Sheet

BentoML cheat sheet

BentoML turns a model-inference script into a production REST API with a couple of decorators. Wrap a class in @bentoml.service, mark methods with @bentoml.api, and type-hint the I/O — BentoML gives you a server with validation, docs, adaptive batching, and streaming. Package everything (code + models + env) into a versioned Bento, then containerize to Docker or deploy to BentoCloud. Targets the modern 1.4.x service API (the class-based successor to the old Runner API), Python 3.9+.

service & API serving features models & config package & ship deploy & ecosystem gotcha ★ most common

Verified 2026-08-31 against the official docs at docs.bentoml.com (BentoML 1.4.39, released 2026-05-07). This sheet uses the current @bentoml.service class API; the pre-1.2 bentoml.Runner/Service(api=...) style is legacy.

Outline

Write a @bentoml.service class, wire models & config, then build a Bento and containerize or deploy it.

1Install & first serviceservice.py

pip install bentoml★
Python 3.9+. Installs the framework + the bentoml CLI. Add framework extras as needed (e.g. bentoml[triton]).
import bentoml @bentoml.service class Summarizer: @bentoml.api def summarize(self, text: str) -> str: return do_summary(text)★
A whole service in one class. Save as service.py; the class name is the service.
bentoml serve service:Summarizer★
Starts a dev server on :3000 with Swagger UI at /. Syntax is <file>:<ClassName>.
curl -X POST localhost:3000/summarize -H "Content-Type: application/json" \ -d '{"text":"long text..."}'
Each @api method becomes a POST /<method> endpoint that takes JSON matching its params.

2@service & @apithe two decorators

@bentoml.service(name="summarizer") class S: def __init__(self): self.model = load() # runs once per worker at startup★
__init__ loads models/state once when the service starts — not per request.
@bentoml.api def predict(self, x: int) -> int: ...★
Only @bentoml.api methods are exposed; helpers stay private. One service can have many endpoints.
@bentoml.api(route="/v1/generate")
Override the URL path. Default route is the method name.
with bentoml.importing(): import torch # heavy/optional import, only at runtime
bentoml.importing() defers imports that shouldn't run at build time (e.g. GPU libs).

3Inputs & outputstype hints = schema

@bentoml.api def f(self, name: str, n: int = 1) -> dict: ...★
Type hints define & validate the request/response schema automatically (via Pydantic under the hood).
from pydantic import BaseModel class In(BaseModel): prompt: str; temp: float = 0.7 @bentoml.api def gen(self, params: In) -> str: ...★
Use a Pydantic model for structured/validated input with defaults & constraints.
from PIL import Image import numpy as np def classify(self, img: Image.Image) -> np.ndarray: ...
Rich types just work: PIL images, numpy arrays, pandas DataFrames, and pathlib.Path for file uploads/downloads.
from bentoml import File def transcribe(self, audio: File) -> str: ...
Accept raw file uploads with the File / Path types; return files the same way.

6Model storeversion models

import bentoml bentoml.sklearn.save_model("iris_clf", clf)★
Save a trained model to the local store with a name; each save gets an auto version tag (iris_clf:<hash>). Framework modules: pytorch, sklearn, xgboost, transformers, onnx…
bento_model = bentoml.models.get("iris_clf:latest") model = bento_model.load_model()★
Reference a stored model by tag; :latest resolves to the newest. Load it in __init__.
@bentoml.service class S: bento_model = bentoml.models.get("iris_clf:latest")
Declaring a model as a class attribute links it to the service so it's packaged into the Bento automatically.
bentoml models list · bentoml models pull · bentoml models push
Manage the store from the CLI; push/pull models to/from BentoCloud or an S3-backed store.

7Service configresources / scaling

@bentoml.service( resources={"gpu": 1, "memory": "4Gi"}, traffic={"timeout": 60})★
Declare the runtime footprint & request timeout right on the decorator — used when deploying.
@bentoml.service(workers=4)
Number of service worker processes (default: match CPU/GPU). workers="cpu_count" for one per core.
@bentoml.service(traffic={"concurrency": 32})
Max concurrent requests per replica — the key signal for BentoCloud autoscaling.
bentoml.serve(..., env={"HF_TOKEN": bentoml.env(...)}) # envs
Configure environment variables & secrets for the service; also settable in the build config.

8Composition (depends)multi-service

@bentoml.service class API: embedder = bentoml.depends(EmbeddingService)★
bentoml.depends() injects another service. Call it like a local object: await self.embedder.embed(x).
# each @service scales independently in one deployment graph
Compose an inference graph: a GPU model service + a lightweight API service, scaled separately. This replaces the old Runner model.
embedder = bentoml.depends(url="http://remote:3000")
Depend on an already-deployed remote service by URL instead of co-packaging it.

9Build config (bentofile)reproducible env

# pyproject.toml [tool.bentoml.build] service = "service:Summarizer"★1.4
Modern config lives in pyproject.toml. It names the service and the build environment.
[tool.bentoml.build.python] packages = ["torch", "transformers"]★
Pin Python deps (or point to a requirements.txt/lockfile) so the Bento rebuilds identically anywhere.
# bentofile.yaml (classic equivalent) service: "service:Summarizer" python: {packages: [torch]} docker: {python_version: "3.11"}
The YAML bentofile.yaml is still fully supported. Set base image, system packages, CUDA, etc. under docker:.
include: ["*.py"] · envs: [{name: HF_TOKEN}]
Control which files are packaged and declare required env vars.

10serve, build & containerizeCLI flow

bentoml serve service:Summarizer --reload★
Local dev with hot reload. Add --port/--host as needed.
bentoml build★
Package code + models + env into a versioned Bento in the local store (~/bentos). Reads the build config.
bentoml list · bentoml get summarizer:latest · bentoml export/import
Manage Bentos; export a Bento to a .bento archive to move between machines.
bentoml containerize summarizer:latest★
Turn a Bento into an OCI/Docker image (auto-generated Dockerfile). Run it anywhere: docker run -p 3000:3000 summarizer:latest.

Worth memorizing

@bentoml.serviceclass = a deployable service

@bentoml.apimethod = a validated HTTP endpoint

type hintsdefine request/response schema (Pydantic)

__init__load models once per worker

yieldgenerator return = streaming (SSE)

batchable=Trueadaptive batching for throughput

bentoml.models.getload versioned model from the store

bentoml.depends()compose services (replaces Runners)

resources / traffic@service(...) sizing + autoscale signal

bentoml buildcode+models+env -> versioned Bento

bentoml containerizeBento -> Docker image

bentoml deployship to BentoCloud w/ autoscaling

BentoML cheat sheet

Outline

Build a service

Serving features

Models & config

Package & ship

Deploy & ecosystem

Build a Service

Serving Features

Models & Config

Package & Ship

Deploy & Ecosystem

Worth memorizing