Quick Reference · gradient boosting on decision trees · v2 — comprehensive

CatBoost cheat sheet v2

CatBoost trains symmetric (oblivious) trees with ordered boosting and ordered target statistics, so raw categorical, text, embedding and NaN-laden columns go in as-is — no encoding, no imputation. One pipeline throughout: raw data → Pool → train with an overfitting detector → best model → predict / quantify uncertainty / explain / ship.

data & pool categorical / text params & structure training predict & explain tuning utility / ecosystem watch out ★ most common

Distilled & cross-checked against: catboost.ai/docs (Python reference · training parameters · parameter tuning · uncertainty) · github.com/catboost RELEASE.md · NeurIPS 2018 paper & SGLB paper · Yandex team tutorials (TDS) · kdnuggets · deep-and-shallow · datascienceletter

The pipeline & the calls that move you through it
Raw Data DataFrame / array / file num + cat + text + emb NaNs OK · no one-hot Pool cat/text/embedding_features weight · group_id · baseline .quantize() to reuse fast Train + OD fit(eval_set=…) · snapshots early_stopping_rounds use_best_model=True Best Model predict · proba · uncertainty SHAP · interactions · fstr .cbm / onnx / coreml / json Pool(X, y,…) .fit() best iter cv() / grid_search() / randomized_search() / Optuna — retry with new params YOUR DATASET DEPLOYABLE ARTIFACT
01Install & Importget going
02Build a Poolthe native dataset object
03Categorical & Textthe signature feature
04Core Paramsconstructor arguments
05Tree Structure & Growthgrow_policy family
06Boosting Scheme & Samplinghow each tree is fit
07Regularization & Constraintsfight overfitting
08Class Imbalanceweights & priors
09Train, Resume & Snapshotfit & friends
10Overfitting Controlstop at the right tree
11Predict & Scoreuse the trained model
12Uncertainty Estimationvirtual ensembles · SGLB
13Explainabilityfstr · SHAP · statistics
14Cross-Validation & Tuningsearch strategies
15GPU & Performancescale up
16Save, Load & Exportship the model
17catboost.utilshelper toolbox
18Custom Objectives & MetricsPython plug-ins
19Ecosystem & Extensionsbeyond the Python lib
20loss_function Menupicks the task for you

Mathematical backbone

The four formulas that explain most of CatBoost's behavior — from the official references and the NeurIPS 2018 paper.

Boosting update

Each tree ht fits the negative gradient of the loss at the current prediction; the learning rate η shrinks its contribution.

Ft(x) = Ft−1(x) + η·ht(x) ht ≈ −∂L(y, Ft−1(x)) / ∂Ft−1(x) η = learning_rate · t = 1 … iterations

Ordered target statistic (with prior)

Category value for row k uses only rows before k in a random permutation σ; the prior P smooths rare categories.

x̂k = ( Σj<k 𝟙[xj=xk]·yj + a·P ) ( Σj<k 𝟙[xj=xk] + a ) P = prior (mean target) · a = prior weight · j<k in permutation σ

Uncertainty decomposition

Virtual ensembles split total predictive uncertainty into what the data can't tell you and what the model hasn't seen.

Total = Knowledge + Data clf: H(p̄) − (1/N)·Σ H(pi) = Knowledge reg: Vari(μi) + mean(σ²i)

Balanced class weights

auto_class_weights='Balanced' multiplies each class k by the largest class total over its own total.

CWk = maxc( Σt=c w ) / Σt=k w SqrtBalanced: CWk = √( maxc(Σw) / Σt=kw ) rarest class ↑ heaviest — probabilities become uncalibrated

What makes CatBoost different

Six mechanisms — same boosting idea as XGBoost/LightGBM, different machinery.

Oblivious (symmetric) tree

Every node at a depth splits on the same feature & threshold — fast inference, built-in regularization.

age < 30 ? city = NY ? city = NY ? L1 L2 L3 L4

grow_policy alternatives

Depthwise (XGBoost-like) and Lossguide (LightGBM-like) grow asymmetric trees — flexible, but slower to apply.

Depthwise level by level, all leaves Lossguide best leaf first

Ordered Target Statistics

Encoding for row r4 uses targets of r1–r3 only — its own target never leaks into its own encoding.

r1 r2 r3 r4 r4's encoding ← targets of r1, r2, r3 only encoding r4

Ordered boosting

Residuals for row k+1 come from a model that never saw row k+1 — removing the prediction-shift bias of classic GBDT.

model Mₖ trained on rows 1 … k scores row k+1 unbiased residual boosting_type='Ordered' — 'Plain' is the classic scheme

Overfitting detector

Eval loss tracked per iteration; training halts after od_wait stale rounds and keeps trees up to the optimum.

train loss eval loss best_iteration early stop →

Virtual ensemble

One trained model is truncated at several tree counts; disagreement between the truncations estimates knowledge uncertainty — no retraining.

full model — T trees m₁ = trees 1…T/2 m₂ = trees 1…3T/4 m₃ = trees 1…T Var(m₁, m₂, m₃ predictions) → knowledge uncertainty

Worth memorizing

cat_features ≠ one-hotmanual encoding kills CTRs & feature combinations
NaNs are finenan_mode handles them; no imputation pipeline needed
use_best_modelsilently ignored without an eval_set
defaults are strongtune iterations+lr first; depth & l2_leaf_reg next
Ordered vs PlainOrdered wins on small data; Plain is faster at scale
symmetric trees~10× faster prediction than Depthwise/Lossguide trees
class weights skew probsrecalibrate predict_proba after weighting classes
eval_metric ≠ lossoptimize Logloss, monitor AUC/F1 — all at once
snapshotssave_snapshot=True resumes training after a crash
quantize() the Poolbin once, then iterate on params much faster
GPU ≠ CPUsome params differ per device; results not bit-identical
.cbm is nativeonnx/coreml/pmml only for cross-platform serving