Categorical Data in Python: Complete Guide

Quick answer: Learn how to handle categorical data in Python with Pandas, including nominal vs ordinal data, one-hot encoding, label encoding, and the category dtype.

What is Categorical Data?

Categorical data represents values that fall into a limited, fixed set of categories, such as colour, city or customer segment, rather than continuous numbers.

Nominal vs Ordinal Data

The category dtype in Pandas

import pandas as pd

df = pd.DataFrame({'city': ['Nagpur', 'Pune', 'Nagpur', 'Mumbai']})
df['city'] = df['city'].astype('category')

print(df['city'].cat.categories)   # Index(['Mumbai', 'Nagpur', 'Pune'])

Converting a text column to category can significantly reduce memory usage when the column has many repeated values, and speeds up certain operations like grouping.

One-Hot Encoding

One-hot encoding converts a categorical column into multiple binary columns, one per category, which most Machine Learning algorithms require since they cannot work directly with text labels.

df_encoded = pd.get_dummies(df, columns=['city'])
print(df_encoded)

This works well for nominal data with a manageable number of categories, but can create too many columns if a category has hundreds of unique values.

Label Encoding

Label encoding assigns each category an integer, which is more compact but introduces an artificial numeric order that can mislead a model if the data is genuinely nominal.

from sklearn.preprocessing import LabelEncoder

encoder = LabelEncoder()
df['city_encoded'] = encoder.fit_transform(df['city'])

Label encoding is more appropriate for ordinal data, where the assigned numeric order can be made to reflect the real underlying order of the categories.

Encoding Ordinal Data Correctly

from pandas.api.types import CategoricalDtype

grade_order = CategoricalDtype(categories=['Low', 'Medium', 'High'], ordered=True)
df['satisfaction'] = df['satisfaction'].astype(grade_order)

Common Interview Questions

What is the difference between one-hot encoding and label encoding?

One-hot encoding creates a separate binary column per category with no implied order, suited to nominal data. Label encoding assigns a single integer per category, which is more appropriate for ordinal data where an order genuinely exists.

Why is label encoding risky for nominal categorical data?

It introduces an artificial numeric relationship, such as implying city A is somehow less than city B, which some models may misinterpret as meaningful when it is not.