Class-Agnostic Object Counting AI

Few-shot object counting in dense scenes without training or fine-tuning

Project Overview

Counting objects in dense or complex environments usually requires training specialized models from scratch. This project explores few-shot object counting: given just 3 visual samples of any object in a scene, the system accurately counts all occurrences without any model training or fine-tuning. By refining foundation model feature matching and spatial density estimation, this approach accurately counts repetitive objects across complex, crowded scenes while staying completely training-free.

Few-Shot Object Counting Zero Training & Fine-Tuning DINO Feature Filtering 2D Gaussian Splatting Dense Scene Enumeration

The Challenge

Counting objects from only a few visual samples with zero model training presents several core vision hurdles:

  • Strict Zero-Training Requirement: The model must recognize and enumerate completely unfamiliar objects using only 3 visual samples, without any weight fine-tuning or category-specific dataset training.
  • Color Sensitivity Distractions: Pre-trained foundation models naturally encode strong color features. If the sample objects differ in color from other items in the scene, standard feature matching fails to group them together.
  • Scale and Rotation Variations: Objects appear at varying angles, orientations, and distances, causing similarity matches to degrade and leading to severe undercounting.
  • Dense Scene Overlapping: In tightly packed scenes, overlapping objects blend together in raw activation maps, making it difficult to isolate individual instances.

The Solution & Technical Approach

To make few-shot counting reliable across diverse environments without retraining, a targeted pipeline enhancements was implemented:

  • Filtering Color Channels in DINO: Removed color-sensitive feature channels from the pre-trained DINO backbone, allowing the model to match objects based on fundamental shape and structure rather than superficial color differences.
  • Fourier Frequency Shift Processing: Analyzed rotation and scale shifts in the frequency domain using log-polar representations and Fourier shift theorems to handle variations between visual samples and target objects.
  • 2D Gaussian Splatting Inference: Fixed count inference by converting peak activations into smooth 2D Gaussian density distributions, accurately separating overlapping objects in dense scenes without retraining.

Interactive Density Estimation Showcase

Drag the slider to compare the raw input scene with 3 visual sample exemplars against the predicted 2D Gaussian Splatting density map:

Raw Scene with 3 Exemplars
Gaussian Splatting Density Map
Gaussian Splatting Density 3 Input Exemplars

Benchmark Metrics & Key Results

Experimental evaluations across dense object benchmarks demonstrated substantial accuracy gains over baseline models:

14.91
Mean of Absolute Error in Counting
Lowest error achieved via 2D Gaussian Splatting, down from 31.28 baseline
52%
Error Reduction
More than halved the mean of absolute error in counting across dense scenes
100%
Training-Free
Operates purely on 3 visual samples with zero model weight updates

Technologies

Few-Shot Object Counting DINO Foundation Models 2D Gaussian Splatting Fourier Frequency Analysis Density Map Estimation PyTorch Zero-Shot Vision

Open Source Repository

Class-Agnostic Counting Repository
Training-Free Few-Shot Object Counting with Fourier Processing and 2D Gaussian Splatting
View GitHub Repository