Image Segmentation Using OpenCV: Complete Guide

Quick answer: Learn image segmentation techniques in OpenCV including thresholding, contour detection and watershed segmentation, with practical Python code examples.

What is Image Segmentation?

Image segmentation divides an image into distinct regions or objects, going a step further than simple object detection by identifying the exact pixels belonging to each region rather than just a bounding box around it.

Simple Thresholding

The most basic segmentation approach separates an image into foreground and background based on pixel intensity.

import cv2

image = cv2.imread('document.jpg', cv2.IMREAD_GRAYSCALE)
_, thresholded = cv2.threshold(image, 127, 255, cv2.THRESH_BINARY)

cv2.imshow('Thresholded', thresholded)
cv2.waitKey(0)

Every pixel above the threshold value becomes white, and everything below becomes black, useful for simple cases with clear contrast, such as scanned text documents.

Adaptive Thresholding

adaptive = cv2.adaptiveThreshold(
    image, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
    cv2.THRESH_BINARY, 11, 2
)

Unlike a single global threshold, adaptive thresholding calculates a different threshold for different regions of the image, working better under uneven lighting conditions.

Contour Detection

contours, _ = cv2.findContours(thresholded, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

image_with_contours = image.copy()
cv2.drawContours(image_with_contours, contours, -1, (0, 255, 0), 2)

Contours trace the boundaries of connected regions in a thresholded image, commonly used to isolate and count distinct objects.

Watershed Segmentation

The watershed algorithm treats an image like a topographic surface and floods it from marked starting points, useful for separating touching or overlapping objects that simple thresholding cannot distinguish from each other.

import numpy as np

markers = cv2.connectedComponents(sure_foreground)[1]
markers = cv2.watershed(image, markers)
image[markers == -1] = [0, 0, 255]   # mark boundaries in red

Practical Use Cases

• Isolating a specific product in a retail image for background removal

• Segmenting cells in medical microscopy images for counting or measurement

• Separating text regions from background in document scanning

• Preprocessing step before further analysis like defect detection in manufacturing

Classical Techniques vs Deep Learning Segmentation

The methods above are classical computer vision techniques, fast and interpretable but limited on genuinely complex scenes. Deep learning approaches like U-Net can learn much more sophisticated segmentation directly from labelled examples, at the cost of needing substantially more training data and compute.

Common Interview Questions

What is the difference between image segmentation and object detection?

Object detection draws a bounding box around a detected object. Segmentation identifies the exact pixels belonging to that object, giving a far more precise outline of its shape.

Why might adaptive thresholding be preferred over simple global thresholding?

Simple thresholding uses one fixed value across the whole image, which fails under uneven lighting. Adaptive thresholding calculates different thresholds for different regions, handling varying lighting conditions much better.