Random Forest Algorithm Explained with Examples

Quick answer: Learn how the Random Forest algorithm works, why it reduces overfitting compared to a single decision tree, feature importance, and its strengths and weaknesses.

What is Random Forest?

Random Forest is an ensemble Machine Learning algorithm that builds many decision trees, each trained on a random subset of the data and a random subset of features, then combines their predictions through majority vote (classification) or averaging (regression).

Why Not Just Use One Decision Tree?

A single decision tree tends to overfit, memorising the specific noise and quirks of its training data rather than the genuine underlying pattern. Random Forest reduces this problem by averaging across many trees, each of which sees slightly different data, so their individual errors tend to cancel out.

How Random Forest Works

1. Create multiple random samples of the training data, drawn with replacement, a technique called bootstrapping

2. Train a separate decision tree on each sample

3. At each split within each tree, consider only a random subset of features, not all of them

4. Combine all trees' predictions through majority vote for classification, or averaging for regression

from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier(n_estimators=200, max_depth=10, random_state=42)
model.fit(X_train, y_train)
predictions = model.predict(X_test)

Key Hyperparameters

Feature Importance

Random Forest can rank which features contributed most to its predictions, based on how much each feature reduced impurity across all the trees' splits, a useful tool for understanding a model's behaviour even though the forest as a whole is less interpretable than a single tree.

import pandas as pd

importances = pd.Series(model.feature_importances_, index=X_train.columns)
print(importances.sort_values(ascending=False))

Strengths and Weaknesses

Random Forest vs a Single Decision Tree

A single decision tree is fast, fully interpretable, but prone to overfitting. Random Forest sacrifices some interpretability and speed in exchange for meaningfully better generalisation to new, unseen data.

Common Interview Questions

Why does Random Forest generalise better than a single decision tree?

Each tree sees a different random subset of data and features, so any single tree's tendency to overfit its specific subset gets averaged out across the many trees in the forest.

What is the difference between bagging in general and Random Forest specifically?

Random Forest is a specific implementation of bagging applied to decision trees, with the additional step of randomly selecting a subset of features to consider at each split, which further decorrelates the individual trees.