Decision Tree in Machine Learning: A Visual Guide with Python (2026)

You already think in decision trees.

"Is it raining? Yes. Do I have an umbrella? No. Take an auto."

A decision tree in machine learning does exactly that, except it learns the questions from data. In this guide you'll see how it picks those questions, why it overfits so easily, and what our tests showed when we limited its depth.

You'll also get:

• A loan-approval tree you can explain to anyone
• Gini impurity worked out with real numbers
• The benchmark where a tree hit 100% on training data and failed on test data

Let's get started.

A decision tree in machine learning is a supervised learning algorithm that predicts an outcome by asking a series of yes/no questions about the input features, arranged like an upside-down tree. Each internal node tests one feature, each branch is an answer, and each leaf node gives the final prediction, a class or a number.

Quick answer: decision tree in machine learning (November 2026)

• A decision tree splits data with yes/no questions, from a root node through decision nodes to leaf nodes.
• Scikit-learn's DecisionTreeClassifier uses an optimised version of CART and splits on Gini impurity by default.
• In our test on the breast cancer dataset, a tree with no depth limit scored 100% on training data but 91.2% on test data, a clear sign of overfitting.
• Limiting depth to 5 raised 5-fold accuracy from 91.0% to 92.8%, and a random forest reached 95.6%.
• Decision trees are popular because they're a white box model: you can read every rule.

Contents

1. What Is a Decision Tree in Machine Learning?
2. A Decision Tree Example: Loan Approval
3. How a Decision Tree Chooses Splits
4. Classification Trees vs Regression Trees
5. Decision Tree in Machine Learning: Python Code We Tested
6. Overfitting and Pruning: Our Depth Test
7. Decision Tree vs Random Forest
8. Advantages and Disadvantages of Decision Trees
9. FAQ

What Is a Decision Tree in Machine Learning?

A decision tree is a supervised learning model that works for both classification and regression. The decision tree algorithm is one of the few models you can explain to a non-technical manager in two minutes. Scikit-learn's user guide describes the goal as predicting a target by learning simple decision rules inferred from the data features.

Every tree has three kinds of nodes:

Decision trees are one of several <a href="/blogs/types-of-machine-learning/">types of machine learning</a> models every beginner should know, alongside linear regression in machine learning.

A Decision Tree Example: Loan Approval

A decision tree example makes the idea click instantly. Here's a simple tree a bank could use to sort loan applications:

Read any path as a sentence:

"If income is above ₹40,000 and credit score is above 700, approve."

That's why decision trees are loved in business. A manager can check every rule, and you can explain any single prediction in one line.

Note: This tree is a teaching illustration, not a real bank's credit policy.

How a Decision Tree Chooses Splits

A decision tree chooses each split by testing every feature and every possible cut-off, then picking the one that makes the resulting groups the most "pure", meaning each group is mostly one class. Purity is measured with Gini impurity or entropy.

Gini Impurity, Worked Out

Gini impurity is calculated as Gini = 1 minus the sum of p², where p is the share of each class in a group.

• A group with 5 "approve" and 5 "reject": Gini = 1 − (0.5² + 0.5²) = 1 − 0.5 = 0.5 (most mixed for two classes)
• A group with 10 "approve" and 0 "reject": Gini = 1 − (1² + 0²) = 0 (perfectly pure)
• A group with 8 "approve" and 2 "reject": Gini = 1 − (0.8² + 0.2²) = 1 − 0.68 = 0.32

The tree picks the split that lowers the weighted Gini the most, then repeats on each branch. Scikit-learn's documentation says it uses an optimised version of the CART algorithm (Classification and Regression Trees) to do this.

Entropy and information gain do the same job using a log-based formula. Scikit-learn's DecisionTreeClassifier uses criterion='gini' by default and also accepts "entropy" and "log_loss". In practice, the two usually give very similar trees.

Classification Trees vs Regression Trees

Classification trees predict a category at each leaf, while regression trees predict a number, usually the average of the training values that land in that leaf.

Scikit-learn's guide warns that tree predictions are piecewise constant approximations that are not good at extrapolation. That's why, in our benchmark, a default regression tree scored an R² of minus 0.133 on the diabetes dataset, while linear regression scored 0.478.

Decision Tree in Machine Learning: Python Code We Tested

A decision tree in Python takes about 10 lines with scikit-learn. We ran this with scikit-learn 1.9.1 on the built-in breast cancer dataset (569 samples, 30 features).

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier, export_text

data = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(
    data.data, data.target, test_size=0.2,
    random_state=42, stratify=data.target
)

tree = DecisionTreeClassifier(max_depth=3, random_state=42)
tree.fit(X_train, y_train)

print("Test accuracy:", tree.score(X_test, y_test))
print(export_text(tree, feature_names=list(data.feature_names)))

The export_text line prints every rule as plain text, so you can paste the logic straight into a report. Use plot_tree to draw it, and check tree.feature_importances_ to see which features drove the splits; feature importance is a favourite interview topic.

Overfitting and Pruning: Our Depth Test

Overfitting happens when a decision tree keeps splitting until it memorises the training data, including its noise. Scikit-learn's guide warns that tree learners can create over-complex trees that do not generalize the data well.

Our benchmark shows it clearly. A tree with no depth limit grew to depth 7 with 19 leaves:

• Training accuracy: 100%
• Test accuracy: 91.2%

Then we tested different depths with 5-fold stratified cross-validation (scikit-learn 1.9.1, random_state=42):

Takeaway: a little restraint beats unlimited growth. Here are the main pruning controls:

1. max_depth: cap how many questions deep the tree can go.
2. min_samples_leaf: require a minimum number of samples in each leaf.
3. ccp_alpha: cost complexity pruning, which trims branches that add little value after the tree is grown.

Pro tip: Tune max_depth with cross-validation, never by staring at test accuracy. Otherwise you're secretly fitting the test set.

Decision Tree vs Random Forest

A random forest is an ensemble: it trains many decision trees on random samples of rows and features, then lets them vote. It trades some interpretability for much better accuracy and stability.

In our benchmark on the same breast cancer data:

Scikit-learn's guide notes that trees can be unstable because small variations in the data might result in a completely different tree, a problem ensembles reduce.

Want to go deeper? Read our guide on <a href="/blogs/ensemble-learning-techniques-in-machine-learning/">ensemble learning techniques</a>.

Advantages and Disadvantages of Decision Trees

Decision trees are easy to explain and need little data preparation, but they overfit easily and can be unstable. Here's the balanced view:

No scaling needed is a real advantage. In the same benchmark, logistic regression and SVM lost accuracy without StandardScaler, while the tree didn't care. See our logistic regression in machine learning and support vector machine in machine learning guides for those numbers.

Now It's Your Turn

A decision tree is the most explainable model you'll learn: questions, branches and leaves, with rules anyone can read. Just remember to prune it.

Run the code above with max_depth from 1 to 6 and watch where accuracy peaks.

Want guided projects and placement support? Explore the <a href="/courses/data-science-certification-nagpur/">Data Science certification at Fireblaze</a>, then practise with these machine learning interview questions.

What will your first decision tree in machine learning decide?

About the Author

Aniruddha Abhay Kalbande is the founder of Fireblaze AI School, which he started in Nagpur on 31 October 2017. He is an Electronics and Telecommunication Engineer from Yeshwantrao Chavan College of Engineering (YCCE), Nagpur, with 10 years of experience in IT consulting and training. He has trained more than 20,000 students. The school is a NASSCOM partner for its Post Graduate Program in Data Science & Analytics and runs classroom batches at 3 Nagpur centers alongside live online batches across Maharashtra. He writes here on career roadmaps, interview preparation and how to evaluate a training institute honestly.

Sources

1. Decision Trees user guide, scikit-learn documentation, 2026
2. DecisionTreeClassifier, scikit-learn documentation, 2025
3. load_breast_cancer, scikit-learn documentation, 2025
4. Fireblaze ML Benchmark: scikit-learn 1.9.1, 5-fold stratified cross-validation, random_state=42, run November 12, 2026