CSC4040: Computer Vision

Image convolution is the mathematical operation of sliding a small matrix (called a kernel or filter) across an image and computing the weighted sum of pixel values at each position. The kernel determines what the operation does: a Gaussian kernel smooths the image by averaging neighboring pixels (weighted by distance from center), a Sobel kernel detects edges by computing the gradient (rate of intensity change), and a sharpening kernel enhances edges by subtracting a blurred version from the original. Convolution is fundamental because nearly every image processing operation can be expressed as a convolution or a sequence of convolutions. Blurring, edge detection, embossing, and feature extraction all use different kernels applied through the same convolution operation. This is also why convolutional neural networks (CNNs) are so powerful for vision tasks: instead of hand-designing kernels, CNNs learn the optimal kernels directly from training data. The first layer might learn simple edge detectors, the second layer learns combinations of edges (corners, textures), and deeper layers learn complex patterns (object parts, faces). In CSC4040, students implement convolution from scratch to understand the mathematics, then use OpenCV's optimized functions for practical applications.

The Canny edge detector uses a multi-stage pipeline that produces thin, well-localized, connected edge lines. Stage one applies Gaussian smoothing to reduce noise (noise produces false gradients that look like edges). Stage two computes the gradient magnitude and direction at every pixel using Sobel operators in the x and y directions. Stage three performs non-maximum suppression: at each pixel, it checks whether the gradient magnitude is the local maximum along the gradient direction, and suppresses (sets to zero) any pixel that is not a local maximum. This produces edges that are exactly one pixel wide, unlike raw Sobel output which produces thick, blurry edges. Stage four applies hysteresis thresholding with two thresholds: pixels above the high threshold are definitely edges, pixels below the low threshold are definitely not edges, and pixels between the two thresholds are edges only if they connect to a definite edge pixel. This connectivity requirement eliminates isolated noise pixels that happen to have strong gradients while preserving weak but genuine edge segments that connect to strong edges. The result is cleaner, more connected edge maps than any single-stage method can produce. In OpenCV, cv2.Canny(image, low_threshold, high_threshold) implements the full pipeline in one call.

Transfer learning means taking a model that was trained on one task (usually a large, general dataset like ImageNet with 1.2 million images across 1,000 categories) and adapting it to a different, often smaller and more specific task. For CNNs, this works because the features learned by early layers (edges, textures, color gradients) are general-purpose visual features useful for virtually any image task, while later layers learn task-specific features (e.g., dog breed characteristics for a dog classification model). In practice, transfer learning involves loading a pre-trained model (ResNet-50, VGG-16, EfficientNet), removing or replacing the final classification layer(s) to match the new task's number of classes, and fine-tuning the model on the new dataset. Fine-tuning can freeze early layers (keeping their general features fixed) and only train later layers, which works well when the new dataset is small; or it can train all layers with a small learning rate, which works better when the new dataset is large enough to support full model updates. Transfer learning matters because training a deep CNN from scratch requires millions of labeled images and significant compute time. With transfer learning, you can achieve strong performance on a custom task with hundreds or thousands of images instead of millions. CSC4040 projects frequently use transfer learning to build classifiers for domain-specific image datasets.

These three tasks represent increasing levels of visual understanding. Image classification assigns a single label to the entire image: "this image contains a cat." It answers the question "what is in this image?" but says nothing about where. Standard architectures include VGGNet, ResNet, and EfficientNet. Object detection finds and localizes multiple objects within an image: "there is a cat at coordinates (x1,y1,x2,y2) and a dog at (x3,y3,x4,y4)." It draws bounding boxes around each detected object and classifies them. Standard architectures include YOLO (You Only Look Once), SSD (Single Shot Detector), and Faster R-CNN. Image segmentation assigns a class label to every pixel in the image rather than just drawing boxes. Semantic segmentation labels each pixel with a class (all "cat" pixels, all "background" pixels) but does not distinguish between different instances of the same class. Instance segmentation (Mask R-CNN) goes further by distinguishing individual objects: "this group of pixels is cat #1, this other group is cat #2." The computational cost increases from classification (process image once, output one label) through detection (process image, output multiple boxes and labels) to segmentation (process image, output a label map the same size as the input). CSC4040 covers all three tasks, starting with classical methods (sliding window + HOG for detection, watershed for segmentation) before introducing CNN-based approaches.