Quick Reference · gradient boosting on decision trees

CatBoost cheat sheet

CatBoost's whole pitch: hand it raw categorical and text columns as-is — no manual encoding — and it builds symmetric (oblivious) trees with built-in ordered boosting to avoid target leakage. Everything below moves through the same pipeline: raw data → Pool → train with an overfitting detector → best model → predict / explain / ship.

data & pool categorical / text core params training predict & explain tuning utility watch out ★ most common

Distilled & cross-checked against: catboost.ai/docs (Python reference, training parameters, parameter tuning) · github.com/catboost/catboost · original NeurIPS 2018 CatBoost paper · geeksforgeeks.org · coderzcolumn.com

The pipeline & the calls that move you through it
Raw Data DataFrame / array / list num + cat + text cols no one-hot needed Pool cat_features=[...] weight / group_id text_features=[...] Train + OD model.fit(eval_set=...) early_stopping_rounds use_best_model=True Best Model predict / predict_proba SHAP · feature importance save_model(.cbm/.onnx) Pool(X, y,…) .fit() best iter cv() / grid_search() / randomized_search() — retry with new params YOUR DATASET DEPLOYABLE ARTIFACT
01Install & Importget going
02Build a Poolthe native dataset object
03Categorical & TextCatBoost's signature feature
04Core Paramsconstructor arguments
05Train the Modelscikit-learn-style API
06Overfitting Controlstop at the right iteration
07Predict & Scoreuse the trained model
08Explainabilityfeature importance & SHAP
09Cross-Validation & Tuningfind good hyperparameters
10GPU & Performancescale up training
11Save, Load & Exportship the model
★Choosing loss_functionpicks the task for you

What makes CatBoost different

Same boosting idea as XGBoost/LightGBM — four mechanisms give it its name and its edge on categorical data.

Oblivious (symmetric) tree

Every node at a given depth splits on the same feature & threshold. Fast to evaluate, and acts as built-in regularization.

age < 30 ? city = NY ? city = NY ? same condition → ← same condition L1 L2 L3 L4

Ordered Target Statistics

To encode a category, CatBoost uses the target of only rows seen earlier in a random permutation — never the row's own target, so there's no leakage.

r1 r2 r3 r4 r4's category → average target of r1, r2, r3 only encoding r4

Ordered boosting

Standard boosting reuses the same rows to fit trees and to evaluate them — a subtle bias. CatBoost trains on a permutation prefix and scores the next unseen row instead.

model Mₖ trained on rows 1 … k scores row k+1 unbiased residual row k+1 only enters training at step k+1 — never scored by its own model

Overfitting detector

Eval-set loss is tracked every iteration; training halts once it stops improving for od_wait rounds, and only the trees up to the best point survive.

train loss eval loss best_iteration early stop →

Worth memorizing

cat_features ≠ one-hotnever manually encode columns you list there
Pool is optionalfit() takes raw X, y directly for simple cases
use_best_modelsilently ignored without an eval_set
learning_rate=Noneauto-picked from iterations & dataset size
od_type IncToDecneeds od_pval; 'Iter' just counts od_wait rounds
symmetric treesfast & regularized, less flexible than leaf-wise growth
border_count=254best GPU quality; lower it to train faster
.cbm is nativeuse onnx/pmml/coreml only for cross-platform export