Quick Reference · the ML & GenAI lifecycle platform

mlflow cheat sheet

An experiment holds three kinds of record, and MLflow 3 is the release that made the second one first-class. A Run is one execution — params, metrics, tags, artifacts. A LoggedModel is one model with its own id, metrics and artifacts, no longer buried inside a run's artifact folder. A Trace is one request through an LLM app, broken into spans. The Registry sits on top and promotes a model by moving an alias.

setup · CLI · projects runs logging models & flavors registry & serving tracing & evaluation removed / gotcha ★ most common

Built against the official docs at mlflow.org/docs/latest — the ML track (tracking, tracking APIs, model, model registry, evaluation) and the LLMs & Agents track (tracing, scorers, judges) — plus the MLflow 3 Migration Guide, the Breaking Changes page, the full CLI reference, and the 3.x release notes.

One experiment end to end · and the three records it holds
A · THE PATH OF ONE EXPERIMENT set_experiment() where records land set_tracking_uri first start_run() a context manager creates the Run log_params / metrics or just autolog() params write once log_model(name=) makes a LoggedModel returns model_uri register & serve alias points at a version mlflow models serve log_model no longer needs an open run — models are first-class in MLflow 3 B · WHAT AN EXPERIMENT HOLDS Experiment the container Run params · metrics · tags LoggedModel own id · own metrics Trace spans of one request backend store metadata · files or a database artifact store the large files themselves the registry needs the database-backed one Model Registry name · version · alias register_model() models:/<model_id> one logged model models:/<name>/3 a pinned version models:/<name>@champion whatever the alias points at runs:/… still works, deprecated
A tracked, registered, reloadable model in seventeen lines
import mlflow
from mlflow.models import infer_signature

mlflow.set_tracking_uri("http://127.0.0.1:5000")   # without this it writes to ./mlruns
mlflow.set_experiment("fraud-detection")         # created if it does not exist
mlflow.autolog()                                 # params, metrics and the model, for free

with mlflow.start_run(run_name="rf-baseline") as run:
    model.fit(X_train, y_train)
    mlflow.log_params({"n_estimators": 300, "max_depth": 8})
    mlflow.log_metric("val_auc", auc, step=epoch)      # step is optional, metrics are a series

    info = mlflow.sklearn.log_model(
        model,
        name="model",                              # MLflow 3: name=, NOT artifact_path=
        signature=infer_signature(X_train, model.predict(X_train)),
        input_example=X_train[:5],
        registered_model_name="fraud-rf",           # log and register in one step
    )

loaded = mlflow.pyfunc.load_model(info.model_uri)   # use the returned URI, not get_artifact_uri()
Part I

Setup, runs & logging

point MLflow somewhere, open a run, and record what happened
01Install & the packagesthree of them
02The CLImlflow --help
03Where it writesURI & the two stores
04Experimentsthe container
05Start a run★context manager
06Params, metrics & tags★the run record
07Autologone line, most of it
08Artifacts & datasetsfiles and inputs
09Nested runs & parallelismmany runs at once
Part II

Models, flavors & the registry

where MLflow 3 changed most — and where old tutorials break
10Log a model★name=, not artifact_path=
11Model URIs★four shapes
12Signatures & examplesthe contract
13Flavorsmlflow.<flavor>
14pyfunc & custom modelsanything at all
15Load & predictgetting it back
16The Model Registry★needs a database
17Aliases, not stagesthe 2.9 migration
18Serving & deploymentmlflow models
Part III

Tracing, evaluation & operations

the GenAI half of MLflow 3, plus querying, projects and the sharp edges
19Tracing★since 2.14, flagship in 3
20Spans by handfiner control
21GenAI evaluation★mlflow.genai
22Scorers & LLM judgeshow quality is graded
23Classical evaluationmlflow.models.evaluate
24Search & querygetting results back
25ProjectsMLproject
26What the tutorials still teach★MLflow 2 habits
27When something goes wrongthe usual suspects

Four things worth seeing

Almost every MLflow 3 error traces back to one of these four pictures being out of date in your head.

Where model artifacts moved

Models left the run's artifact folder and got a directory of their own. This one relocation is why old load paths return ResourceNotFound.

MLflow 2 — the model lived under the run experiments/ ∟ <experiment_id>/ ∟ <run_id>/ ∟ artifacts/ ∟ model/ get_artifact_uri("model") resolved here — which is why the old snippets worked MLflow 3 — the model has its own home experiments/ ∟ <experiment_id>/ ∟ models/ ∟ <model_id>/ ∟ artifacts/ use the model_uri that log_model returned list_artifacts() on the run no longer shows models either

The model URI decoder ring

Six shapes, four of them current. Picking the wrong one is the most common MLflow error, because the deprecated forms still appear in nearly every tutorial.

CURRENT models:/<model_id> one logged model — what log_model hands back as info.model_uri models:/fraud-rf/3 a pinned registered version — immutable, so reproducible models:/fraud-rf@champion whatever the alias points at — promote without redeploying ./out · s3://bucket/model a saved model directory, no tracking server involved STALE — BUT EVERYWHERE ONLINE runs:/<run_id>/model deprecated — still resolves today, slated for removal models:/fraud-rf/Production went with stages — replaced by the @alias form above

Stages became aliases

Four fixed stages were too rigid, so MLflow split the idea in two: a registered model per environment, and aliases naming the version that serves traffic inside each.

BEFORE — one model, four fixed stages None Staging Production Archived deprecated in 2.9 — the stage also had to stand in for the environment AFTER — a registered model per environment, aliases inside dev.fraud-rf experiments land here staging.fraud-rf CI promotes into it prod.fraud-rf serves real traffic @champion → v7 @challenger → v9 Serving targets the alias: models:/prod.fraud-rf@champion Promotion = move the alias. No redeploy, and rollback is the same one-line operation.

What a trace looks like

A trace is one request; spans are the nested steps inside it. Autolog produces them for known libraries, @mlflow.trace for everything you wrote yourself.

ONE TRACE · ONE REQUEST chat_app AGENT · root span · @mlflow.trace ∟ retrieve RETRIEVER ∟ generate LLM · from mlflow.openai.autolog() ∟ lookup TOOL 0 ms latency → decorator order matters @mlflow.trace goes outermost — but BELOW @app.route / @app.post traces are queryable mlflow.search_traces() feeds straight into mlflow.genai.evaluate()

Worth memorizing

name=not artifact_path= — the defining MLflow 3 change
info.model_urithe only reliable way to load back what you logged
models:/<model_id>the new URI shape; runs:/ is deprecated
@championaliases replaced the four fixed stages in 2.9
registryneeds a database backend — the file store cannot serve it
set_tracking_urifirst call, or everything silently lands in ./mlruns
the CLIreads MLFLOW_TRACKING_URI, never your Python config
paramsimmutable and stored as strings
metricsappend-only — a metric is a series, not a value
step=omit it and every point lands on step 0
timestamp=milliseconds; seconds put your point in 1970
autolog()opens and closes its own run if none is active
log_models=Falsestops autolog uploading a model every fit
@mlflow.traceoutermost — except below a route decorator
models are first-classlog_model no longer needs an open run
deletes are softspace comes back only with mlflow gc
Recipesremoved outright in MLflow 3
fastai / mleapflavors dropped, with gluon and diviner
greater_is_betterrenamed from higher_is_better
default judgegpt-4o-mini — needs a key, and costs per row