In plain English
Machine learning is fitting flexible patterns to data and checking, honestly, whether they hold on data the model has never seen.
The advanced view
The bias–variance trade-off governs model choice: flexible models reduce bias and raise variance. Regularisation (ridge, lasso), cross-validation and ensembling manage that trade-off. Tree ensembles dominate tabular business problems; interpretability tools (feature importance, SHAP) matter because decisions must be defended.
ML is not a replacement for statistics; it is an extension that trades interpretability for predictive power when the prediction problem demands it. Supervised learning predicts labels: decision trees split recursively on the most informative feature and overfit aggressively, random forests average many bootstrapped trees to fix that, and gradient boosting (XGBoost, LightGBM) builds trees sequentially, each correcting the last. These are the best tabular models in practice — credit scoring, fraud detection, churn.
Regularisation penalises complexity: ridge (L2) shrinks coefficients toward zero, lasso (L1) can zero them out and so selects variables, elastic net combines both, and λ is tuned by cross-validation. Unsupervised learning has no labels: k-means partitions by minimising within-cluster variance, and PCA finds directions of maximum variance — applied to a return correlation matrix, the first component is usually "the market". For time series use walk-forward validation; never train on future data.
Essential vocabulary
- Bias–variance trade-off
- Simple models underfit, complex models overfit. The sweet spot minimises total error.
- Feature engineering
- Building informative inputs from raw data. Often more impactful than model choice.
- Confusion matrix
- TP, FP, TN, FN — and from them precision, recall, F1 and AUC-ROC.
- Hyperparameters
- Settings not learned from data (depth, learning rate, λ). Tuned on validation data.
Why it works
Cross-validation works because in-sample error is optimistically biased — the model has already seen the answers. Holding out folds estimates the generalisation error, which is the only quantity that predicts business performance. This is also why a model that beats a baseline in-sample but not out-of-sample has learned noise.
Common pitfalls
- ×Leaking target information through features built after the prediction moment.
- ×Judging an imbalanced classifier by accuracy instead of precision/recall or AUC.
- ×Tuning on the test set until it becomes a second training set.
- ×Reporting model lift without translating it into money.
How it is used — value a churn model
Step 1 of 4
- 1Base churn 10% of 100,000 customers = 10,000 leavers, each worth 500 of margin.
Deeper
Deeper: bias, variance and the cost of a wrong prediction
Total error = bias² + variance + irreducible noise. Simple models are biased but stable; flexible models fit the training data and swing wildly on new data. Cross-validation exists to measure the swing, and regularisation (ridge, lasso) buys variance reduction by accepting a little bias.
For business use, accuracy is the wrong headline metric on imbalanced problems: a churn model that predicts 'no churn' for everyone is 95% accurate and worthless. Choose the threshold from the cost matrix — what a false positive costs in wasted retention spend versus what a false negative costs in lost lifetime value.
Must know cold
- ✓Always hold out data; never tune on the test set.
- ✓Precision = TP/(TP+FP); recall = TP/(TP+FN); they trade off through the threshold.
- ✓Class imbalance makes accuracy meaningless.
- ✓Interpretable models win when a human must defend the decision.
Exercises
Try each one on paper before revealing the worked solution.
Exercise 1
Churn model: 5% of 100,000 customers churn. At the chosen threshold, it flags 8,000 with 2,000 true positives. Compute precision and recall, and value the campaign at 200 retention cost and 1,500 LTV saved with a 30% save rate.
Exercise 2
Training accuracy 98%, validation accuracy 71%. Diagnose and give two fixes.
A linear model averages over curvature and misses the turning point; trees and splines capture it. The trade is interpretability and overfitting risk, which is why the flexible model must be judged on out-of-sample error, never on fit.