Polynomial Regression in Python: Complete Guide

Quick answer: Learn polynomial regression including when to use it over linear regression, implementing it in Python with Scikit-Learn, and how to avoid overfitting with high-degree polynomials.

What is Polynomial Regression?

Polynomial regression models a non-linear relationship between a feature and a target by adding polynomial terms, such as x squared and x cubed, allowing a curved line to fit the data instead of a straight one, while still using the same linear regression machinery underneath.

When Plain Linear Regression Fails

If the true relationship between a feature and the target genuinely curves, a straight line will systematically underfit, missing the pattern at both extremes of the data. Polynomial regression addresses this by transforming the feature before fitting a standard linear model to the transformed features.

The Polynomial Regression Equation

y = b0 + b1*x + b2*x² + b3*x³ + ... + bn*xⁿ

Implementing Polynomial Regression in Python

from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
import numpy as np

X = np.array([[1], [2], [3], [4], [5]])
y = np.array([2, 8, 18, 32, 50])

poly = PolynomialFeatures(degree=2)
X_poly = poly.fit_transform(X)   # creates columns for x and x²

model = LinearRegression()
model.fit(X_poly, y)

prediction = model.predict(poly.transform([[6]]))

PolynomialFeatures transforms the original feature into multiple polynomial terms, then an ordinary LinearRegression model is fit on those transformed features, which is why polynomial regression is technically still considered a form of linear regression, linear in its coefficients even though the resulting curve is not a straight line.

Choosing the Right Degree

A degree that is too low underfits a genuinely curved relationship. A degree that is too high overfits, producing a wildly oscillating curve that memorises the training points rather than capturing a genuine pattern. The right degree is typically chosen using cross validation across a few candidate degrees.

from sklearn.model_selection import cross_val_score

for degree in [1, 2, 3, 4]:
    poly = PolynomialFeatures(degree=degree)
    X_poly = poly.fit_transform(X)
    scores = cross_val_score(LinearRegression(), X_poly, y, cv=5)
    print(f"Degree {degree}: {scores.mean():.3f}")

The Overfitting Risk with High Degrees

As the polynomial degree increases, the model gains enough flexibility to pass through every single training point exactly, which usually means it has stopped learning a genuine pattern and started memorising noise. This is a classic, very visible example of overfitting.

Common Interview Questions

Is polynomial regression a type of linear regression?

Yes, technically. It is linear in its coefficients, even though the resulting relationship between the original feature and the target is a curve, since the polynomial terms are simply additional transformed features fed into an otherwise standard linear model.

How do you decide the right polynomial degree to use?

Through cross validation, comparing performance across a few candidate degrees and choosing the one that generalises best to validation data, rather than the one that fits the training data most closely.