Hugging Face Transformers v5 Cheat Sheet

1Install & verifytransformers 5.x

pip install "transformers[torch]"★
v5 is PyTorch-only. Add accelerate for multi-GPU/device_map, datasets for data, bitsandbytes for 4/8-bit.
import transformers; transformers.__version__
Requires Python 3.10+, PyTorch 2.4+. Python 3.9 is rejected.
export HF_HOME=/data/hf
Cache location for models/datasets (default ~/.cache/huggingface). HF_TOKEN supplies auth non-interactively.
transformers env
CLI (replaces transformers-cli) — prints versions/CUDA for bug reports; transformers chat / serve also exist.

2pipeline() — one-liner inferencehigh-level

from transformers import pipeline pipe = pipeline("text-generation", model="Qwen/Qwen3-8B")★
Handles tokenize → forward → decode. Task string picks the head; omit model for a task default.
pipe("Hello, ", max_new_tokens=50)
Call with text (or a list to batch). Kwargs pass through to generate() / the model.
pipeline(task, model=m, device_map="auto", dtype="auto")★
Spread across GPUs and load in the checkpoint's native precision.
pipeline("automatic-speech-recognition") # asr, image-classification, # zero-shot-classification, feature-extraction, image-text-to-text ...
Same API across modalities. image-text-to-text is the modern vision-LLM task.

3Auto classes & from_pretrainedexplicit control

from transformers import AutoModelForCausalLM, AutoTokenizer★
Auto classes pick the right architecture from the checkpoint config. Also AutoModelForSequenceClassification, ...ForTokenClassification, ...ForImageTextToText, AutoProcessor.
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B")★
Downloads + caches weights (safetensors) and builds the model. Pair with AutoTokenizer.from_pretrained(same_id).
from_pretrained(id, dtype="auto", device_map="auto")★
dtype="auto" uses the saved precision (the v5 default). Pass revision="..." to pin a commit, token=... for gated repos.
model.save_pretrained("out/") # writes model.safetensors + config.json tok.save_pretrained("out/")
Reload with from_pretrained("out/"). Serialization is safetensors-only in v5.
model.config · model.generation_config
config is architecture; generation_config holds default decoding params shipped with the model.

4dtype, device_map & attentionhow it loads

dtype=torch.bfloat16 # or "auto" (default), torch.float16, torch.float32★
v5: the old torch_dtype= name is superseded by dtype=. Default changed from fp32 to "auto" — pin torch.float32 for the old behavior.
device_map="auto" # or "cuda", {"": 0}, a custom dict★
Needs accelerate. Shards a big model across GPUs/CPU/disk. For a single device just .to("cuda").
attn_implementation="flash_attention_2" # or "sdpa" (default), "eager"
sdpa (PyTorch scaled-dot-product) is the default; flash_attention_2 needs the flash-attn package + supported GPU.
model.eval() · with torch.inference_mode(): ...★
Disable dropout and autograd for inference — faster and less memory.

5generate() & decodingtext generation

inputs = tok("The capital of France is", return_tensors="pt").to(model.device) out = model.generate(**inputs, max_new_tokens=64)★
Decode with tok.decode(out[0], skip_special_tokens=True) (or batch_decode).
generate(..., do_sample=True, temperature=0.7, top_p=0.9, top_k=50)★
Sampling controls. do_sample=False = greedy; num_beams=4 = beam search. repetition_penalty, min_new_tokens also common.
generate(..., streamer=TextStreamer(tok))
Token-by-token streaming to stdout; TextIteratorStreamer for a Python iterator (e.g. a web stream).
generate(..., cache_implementation="static")
Static KV-cache unlocks torch.compile speedups for decoding.
set pad_token & padding_sidegotcha
For batched generation set tok.pad_token = tok.eos_token and left-padding (tok.padding_side="left") or outputs misalign.

6Chat templates & messagesinstruct models

msgs = [{"role":"user", "content":"Hi"}]★
The universal message format. Each model's chat template turns it into the exact prompt string it was trained on.
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)★
Renders + tokenizes in one call. add_generation_prompt=True appends the assistant turn opener.
pipe(msgs, max_new_tokens=256) # pipeline accepts messages directly
A text-generation pipeline applies the chat template for you when given a message list.
tools=[...] · apply_chat_template(msgs, tools=tools)
Tool/function-calling: pass Python callables or JSON schemas; capable models emit tool calls.

7Vision / audio / multimodalbeyond text

pipe = pipeline("image-text-to-text", model="Qwen/Qwen3-VL-8B") pipe(text=msgs_with_image)★
Vision-language models: messages can carry {"type":"image","url":...} content parts.
asr = pipeline("automatic-speech-recognition", model="openai/whisper-large-v3") asr("audio.mp3")
Audio in, text out; return_timestamps=True for word/segment times.
proc = AutoProcessor.from_pretrained(id) inputs = proc(text=..., images=..., return_tensors="pt")★
AutoProcessor bundles the tokenizer + image/feature extractor for multimodal models.

10Trainer & TrainingArgumentsthe training loop

from transformers import Trainer, TrainingArguments args = TrainingArguments(output_dir="out", per_device_train_batch_size=8, learning_rate=2e-5, num_train_epochs=3, bf16=True)★
Config object; hundreds of fields (schedulers, logging, eval strategy, gradient accumulation).
trainer = Trainer(model, args, train_dataset=ds, processing_class=tok, data_collator=collator) trainer.train()★
v5: pass the tokenizer/processor as processing_class= — the old tokenizer= arg is removed.
args = TrainingArguments(..., eval_strategy="epoch", gradient_accumulation_steps=4, gradient_checkpointing=True)
eval_strategy (renamed from evaluation_strategy); accumulation + checkpointing stretch small GPUs.
trainer.evaluate() · trainer.push_to_hub() · trainer.save_model()
Multi-GPU is automatic via accelerate; launch with accelerate launch train.py.
compute_metrics, label namesgotcha
Pass compute_metrics for eval scores; ensure your dataset has a labels column (the model computes loss when labels is present).

11PEFT / LoRA / TRLcross-reference

pip install peft from peft import LoraConfig, get_peft_model model = get_peft_model(model, LoraConfig(r=16, target_modules="all-linear"))★
Parameter-efficient fine-tuning — train small adapters, not the full model. Fits a Trainer directly.
model.print_trainable_parameters() · model.save_pretrained("adapter/")
Saves just the adapter (MBs). Reload onto the base with PeftModel.from_pretrained(base, "adapter/").
pip install trl # SFTTrainer, DPOTrainer, GRPOTrainer, PPOTrainer
TRL builds on Trainer for instruction tuning (SFT) and preference/RL training (DPO/GRPO). See its own docs.

12huggingface_hub — download/uploadhub client · v1.x

pip install huggingface_hub hf auth login # CLI login (or set HF_TOKEN)★
The hf CLI (v1.x) replaces huggingface-cli. Auth is needed for gated/private repos and uploads.
from huggingface_hub import hf_hub_download path = hf_hub_download("org/model", "config.json")★
Download one file to the cache and return its local path.
from huggingface_hub import snapshot_download snapshot_download("org/model", allow_patterns="*.safetensors")★
Download a whole repo; filter with allow_patterns/ignore_patterns to skip large or duplicate files.
model.push_to_hub("me/my-model") · upload_file(...) · create_repo(...)★
Push weights/adapters/datasets. HfApi() exposes the full REST surface (repos, files, branches).

13InferenceClient — remote modelsno local weights

from huggingface_hub import InferenceClient client = InferenceClient(model="meta-llama/Llama-3.1-8B-Instruct")★
Call hosted models via Inference Providers — no GPU needed locally. Reads your token automatically.
client.chat_completion(messages=msgs, max_tokens=200)★
OpenAI-style chat interface; also text_generation, feature_extraction, text_to_image, etc.
provider="auto" # route to an available provider; AsyncInferenceClient for async
Pick or auto-route across providers; drop-in OpenAI-compatible base URL supported too.

14safetensors — safe weightsno pickle

from safetensors.torch import save_file, load_file save_file(model.state_dict(), "model.safetensors")★
Safe (no arbitrary code on load), fast, zero-copy. The default — and in v5 the only — format Transformers reads/writes.
state = load_file("model.safetensors", device="cuda")
Loads straight to a device. safetensors.numpy / .flax variants also exist.
from safetensors import safe_open with safe_open("m.safetensors", framework="pt") as f: t = f.get_tensor("layer.weight")★
Lazy, per-tensor access — read one weight without loading the whole file (great for sharded checkpoints).
metadata must be str→strnote
Only tensors + a flat string metadata dict are stored — no Python objects. That restriction is the whole point (no pickle RCE).

15Quantization & performancefit + speed

from transformers import BitsAndBytesConfig q = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4") from_pretrained(id, quantization_config=q)★
4-bit (QLoRA-style) or 8-bit via bitsandbytes — run big models on small GPUs.
quantization_config= # also: GPTQConfig, AwqConfig, TorchAoConfig, ...
Load pre-quantized GPTQ/AWQ checkpoints, or quantize with TorchAO. Method depends on the checkpoint.
model = torch.compile(model) # + cache_implementation="static"
Compile the forward pass for a decode speedup; combine with a static KV-cache.
attn_implementation="flash_attention_2" · bf16 · batching
The reliable wins: bf16 + SDPA/FlashAttention + real batching. Profile before reaching for exotic tricks.

16v5 migration & gotchaswhat changed from v4

TensorFlow & Flax backends REMOVEDv5
v5 is PyTorch-only. TFAutoModel* / FlaxAutoModel* are gone — port to the PyTorch classes.
dtype="auto" is the new defaultv5
Models load in their saved precision, not fp32. Pin dtype=torch.float32 to restore v4 behavior. The arg is dtype=, not torch_dtype=.
safetensors onlyv5
safe_serialization is gone; passing safe_serialization=False now errors. .bin/pickle saving is dropped.
Trainer(tokenizer=) → processing_class=v5
Also evaluation_strategy→eval_strategy, and various long-deprecated TrainingArguments removed.
min: Python 3.10+, PyTorch 2.4+requirements
Python 3.9 is unsupported. Read MIGRATION_GUIDE_V5.md for the full list before upgrading a v4 codebase.

transformers cheat sheet

Outline

Get running

Inference

Tokenizers & data

Training

Hub & serialization

Scale & migrate

Get Running

Inference & Generation

Tokenizers & Data

Training & Fine-tuning

Hub & Serialization

Scale & Migrate

Worth memorizing