Object detection is a computer vision task that finds each object of interest in an image, draws a box around it and labels it. It answers two questions at once: what is in the picture, and where is it? From those answers, software can count, check and raise an alert.
Imagine a forklift setting a pallet down at loading bay 4. Before the truck leaves, someone has to check it. Are all 24 cartons there? Is the shipping label on? Today that check means a person with a clipboard. Object detection lets a camera do the first pass.
This guide builds on our introduction to computer vision and focuses on one task. We use the loading bay as a running example.
Key takeaways
- Object detection returns a class, a box and a confidence score for each object. It does not understand intent or the wider situation.
- The value comes from what happens next: counting, checking against a rule or alerting a person.
- A strong benchmark score says little about your own cameras. Test on your own images, taken on different days.
- Start with one narrow task and a measured baseline, and try an existing model before paying for a custom one.
What is object detection?
Object detection tells you which objects of known types appear in an image and where each one sits. Those types are called classes. A model trained on the public COCO dataset knows 80 everyday classes, such as person, truck and bottle. A loading-bay model might need only three: pallet, carton and shipping label.
For every object it finds, the detector returns a class, a bounding box and a confidence score. The box is four numbers that mark a rectangle around the object, and the score runs from 0 to 1.
One frame at bay 4 might produce a pallet at 0.97, 23 cartons between 0.71 and 0.95, and a shipping label at 0.88. The detector decides nothing. A business rule compares 23 cartons with the 24 on the order and flags the pallet.
Recognising an object is not understanding the scene. The detector sees a carton, but not that it holds the wrong product or belongs on another truck. That context comes from other systems, such as your warehouse management system.
Confidence scores need care too. A score of 0.9 does not mean the box is right nine times out of ten. Research on confidence calibration for object detection found detection models to be systematically miscalibrated. Treat the score as a ranking signal and set thresholds by testing them on your own images.
How is object detection different from other computer vision tasks?
Object detection sits between image classification, which labels a whole image, and image segmentation, which outlines objects pixel by pixel. Object tracking adds time. The table applies all four to our loading bay.
| Task | Question it answers at the loading bay | What it returns | Enough when you need |
|---|---|---|---|
| Image classification | Is there a pallet in this frame? | One or more labels for the whole image | To know whether something is present |
| Object detection | How many cartons are on the pallet, and where? | A class, box and confidence score per object | Counts, positions or specific items |
| Image segmentation | Which pixels show the torn shrink wrap? | An outline (mask) per object | Shape, area or a precise edge |
| Object tracking | Is this the same pallet that arrived two minutes ago? | A persistent ID per object across video frames | Movement, waiting time or counting things that pass |
Boxes are enough when count and position matter. When area or shape matters, you need segmentation.
Detection also has no memory. It treats every frame on its own, so a pallet in view for ten seconds at 25 frames per second is detected 250 times. To count pallets passing a door, you need a tracker that gives each object an ID across frames.
How does object detection work?
A detection system turns a camera frame into an action in five steps, and the model is only one of them.
- Capture: a camera takes a frame when a sensor sees the pallet arrive.
- Prepare: software resizes and normalises the image for the model.
- Run the model: the detector returns boxes, each with a class and a score.
- Filter and interpret: low scores and duplicate boxes are dropped, and business rules are applied.
- Act: the system passes the pallet, alerts a supervisor or logs the image.
Two phases matter. Training is when a model learns from labelled images, in which people have drawn a box around every object. Inference is when the trained model processes new images. Inference runs on every frame, so its speed and hardware decide your running costs and response time.
Few companies train a detector from scratch. A pretrained model has already learned general visual features from large public datasets. Fine-tuning continues that training on your own labelled images, so it learns your carton types and camera angle.
Which model families are used for object detection?
YOLO (You Only Look Once) looks at the whole image in one pass. The original YOLO paper from 2015 showed detection fast enough for live video. Ultralytics released YOLO26 in January 2026, built for edge devices. Check the licence first: Ultralytics offers AGPL-3.0 or a paid enterprise licence.
Faster R-CNN works in two stages: it proposes regions, then classifies each one. It is usually slower, but it remains a well-understood baseline. DETR, from Facebook AI researchers in 2020, uses a transformer to predict all objects at once. The original trained slowly, and later variants such as RT-DETR target real-time speed.
Newest does not mean best for you. What counts is accuracy on your images, speed on your hardware, the licence and whether your team can maintain it.
Which object detection models should you evaluate first?
There are hundreds of detection architectures and thousands of trained variants, so start from your goal rather than from a leaderboard. The table lists common starting points as of October 2026. Some of them classify, segment or track rather than detect, and in practice they are combined with a detector.
| Your goal | Models worth evaluating | Good to know |
|---|---|---|
| Fast detection on a camera or edge device | YOLO26, YOLO11 | YOLO26 is the current Ultralytics release. YOLO27 was announced in September 2026 but is not yet available. |
| High accuracy where speed matters less | RT-DETR, larger YOLO variants, Faster R-CNN | Compare them on your own images, not on COCO scores |
| Find objects from a text description | Grounding DINO, YOLO-World, YOLOE | Useful for feasibility tests and pre-labelling |
| Recognise objects without custom training | Grounding DINO, OWLv2, Florence-2 | Usually less reliable than a fine-tuned model on niche items |
| Precise outlines instead of boxes | SAM 3, Mask2Former, YOLO segmentation models | SAM 3 also accepts text prompts and works on video |
| Follow objects through video | A detector plus ByteTrack or BoT-SORT, or SAM 3 | Trackers add IDs; they do not find objects on their own |
| Label the whole image | ConvNeXt, EfficientNet, ViT | Classification only: no boxes |
| Run on small edge devices | Nano and small YOLO variants, MobileNet-based models | Check export support for your hardware first |
Licences differ as well. Ultralytics models come under AGPL-3.0 or a paid licence, SAM 3 under Meta’s own SAM License, and many research models under permissive licences such as Apache 2.0 or MIT. Check the licence before you build a product on a model.
Where does object detection create business value?
Object detection pays off where people check by hand at volume and a miss has a clear cost. In each case below, the value comes from the rules and systems around the detector.
Manufacturing: are all components present?
A detector finds each expected component on an assembly, and a rule compares the result with the parts list for that product variant. Missing items send the unit to rework. Visible defects such as scratches vary more than components do, so they often need segmentation or anomaly detection on top.
Logistics: pallets, parcels and labels
At our loading bay, the detector finds pallets, cartons and labels, and a rule checks them against the order. Reading the label needs OCR or a barcode reader. One limit is physical: a camera cannot see cartons hidden inside a stacked pallet.
Retail: is the product on the shelf?
Detection on shelf images finds products and gaps, and software compares them with the planned layout. With thousands of products, a second classification step usually identifies the exact item, and new packaging means new training images.
Transportation: road users and traffic flows
Detection finds cars, cyclists and pedestrians at a junction, and tracking turns them into counts and routes for traffic planners. Cameras in public space film people, so privacy design belongs in the project from day one.
Agriculture: weeds, fruit and crops
On John Deere’s See & Spray, cameras on the spray boom identify weeds and switch individual nozzles on only where needed. John Deere reports that customers used it on more than five million acres in 2025 and cut non-residual herbicide use by nearly half on average. These are company-reported figures. The pattern still holds: detection alone sprays nothing, and the value comes from linking it to machine control.
What determines object detection accuracy and reliability?
Reliability depends more on the images and the setup than on the model.
Image conditions, data and labels
At the bay, afternoon sun floods in when the dock door opens. Shrink wrap reflects light, a moving forklift blurs the frame and its mast hides part of the stack. A label six metres away covers only a few dozen pixels. Each of these cuts accuracy, and they often come together.
Training images must cover every shift, season, product variant and error type. Labels must be consistent: if one person boxes each carton and another boxes the whole stack, the model gets confused. New packaging or a moved camera changes conditions after launch, so keep monitoring.
False positives and false negatives
A false positive flags a complete pallet, and a supervisor loses a few minutes. A false negative misses a pallet that really lacks a carton, and a customer gets a short delivery. The second usually costs more. A lower confidence threshold catches more real problems but raises more false alarms, so set it based on what each mistake costs.
Precision, recall, IoU and mAP in plain language
Precision asks: of everything the model flagged, how much was right? Recall asks: of everything really there, how much did it find? IoU (intersection over union) measures how well a predicted box overlaps the true box. A detection counts as correct only above a set overlap, often 0.5.
mAP (mean average precision) rolls precision and recall into one number, averaged across classes. The COCO benchmark also averages it over ten overlap thresholds from 0.5 to 0.95. It helps compare models, but for a business decision, ask for precision and recall at your own threshold.
Why benchmark scores do not transfer
Grounding DINO scored 52.5 AP on COCO without any COCO training data. In the same zero-shot setting, it averaged 26.1 AP on ODinW, a collection of real-world datasets. The benchmarks differ in difficulty, but the gap shows how much results vary by domain, and your loading bay is a domain of its own.
Speed, real time and human review
Larger models are usually more accurate and slower. A waiting pallet needs one good result within seconds, while a parcel sorter needs an answer in milliseconds. Define the response time first.
Send uncertain or consequential results to a person. A borderline pallet appears on the supervisor’s screen with the boxes drawn in, and each correction becomes new training data.
How do you start an object detection project?
Start with one narrow, measurable question, and prove the model on your own images before integrating anything.
- Define one problem, such as missing labels at bays 1 to 4, and measure how often it happens today.
- Set acceptance criteria. A hypothetical target: catch 95 percent of missing labels, with at most five false alarms per shift.
- Fix the camera setup and collect images across shifts and product mix, including real errors.
- Evaluate an existing model first, then decide whether fine-tuning is needed.
- Test on unseen images from different days, not frames from the same video clip.
- Run a pilot alongside the current process for several weeks, flagging pallets without blocking them.
- Plan integration, monitoring and maintenance, including who owns the model.
Should the model run at the edge or in the cloud?
Edge deployment runs the model on a device next to the camera. It responds fast, works offline and keeps images on site, but you maintain hardware everywhere. The cloud is easier to update and scale, but it needs a stable connection and sends images off site. Many systems detect at the edge and train in the cloud. Our comparison of cloud, edge and local AI covers the trade-offs.
If cameras also capture people, the footage can be personal data. The EDPB guidelines on video devices explain how the GDPR applies. In practice: aim cameras at the goods, blur people where possible and keep footage short.
What does object detection cost, and when does it pay off?
The model is rarely the largest cost. Cameras, data, integration and maintenance usually weigh more.
| Cost item | What it covers | What drives the amount |
|---|---|---|
| Cameras and lighting | Cameras, mounts, lighting, cabling | Number of positions, dust, cold or outdoor use |
| Data and labelling | Collecting images, drawing boxes, a labelling guide | Number of classes, objects per image, rare errors |
| Development | Model choice, fine-tuning, business rules, testing | How specific the classes are, the accuracy target |
| Computing | Edge devices, GPU servers or cloud inference | Frame rate, model size, number of cameras |
| Integration | Links to warehouse or production systems, alerts | Number of systems, degree of automation |
| Maintenance | Monitoring, retraining, camera upkeep, licences | How often products and conditions change |
Compare the total cost over several years with measurable benefits, such as checking hours saved and errors avoided.
A hypothetical example: a warehouse ships 100,000 pallets a year. Assume 0.5 percent leave with a missing carton or label, so 500 pallets, each costing €120 in credit notes and redelivery. That is €60,000 a year. If the pilot shows the system catches 80 percent of them, it avoids €48,000 a year. Replace every assumption with your own figures.
Sometimes a camera is the wrong tool. If every carton is scanned at picking, a scan count is cheaper and more reliable. Object detection makes sense when objects and positions vary and no barcode or sensor answers the question.
Frequently asked questions about object detection
What is the difference between object detection and image recognition?
Image recognition usually identifies what an image shows, often with one label per image. Object detection goes further: it finds every object of the classes it knows, marks each one with a bounding box and adds a confidence score. Recognition says a pallet is present. Detection says how many there are, and where.
Can object detection work in real time?
Object detection can run in real time, and compact models process dozens of video frames per second on suitable hardware. Whether that is fast enough depends on your process: a parked pallet allows seconds, while a fast conveyor needs milliseconds. Define the response time first, then choose the model and hardware.
How much training data do you need?
No fixed amount works for every project. Fine-tuning a pretrained model often starts with hundreds to a few thousand labelled images, depending on the classes. Coverage matters more than volume: include every shift, lighting condition, product variant and error type, and keep separate images for testing.
Can a model detect objects it was not originally trained to recognise?
A standard detector only finds the classes it was trained on. Open-vocabulary detectors such as Grounding DINO take a text prompt and can find new object types without extra training. They suit quick feasibility tests and pre-labelling, but on specialised items a fine-tuned model is usually more reliable.
How do you measure object detection accuracy?
Teams measure precision (how many detections are right), recall (how many real objects were found) and mAP, which summarises both across classes. For a business decision, look at precision and recall at the threshold you will actually use, on unseen images from your own cameras.
Do you need a custom model?
You do not always need a custom model. For common objects such as people, vehicles or parcels, a pretrained model may perform well enough, so test one first. Fine-tuning is usually needed when your classes are specific, such as your own components or packaging, or when your camera view differs from public datasets.
Can object detection run on an existing camera system?
Object detection often runs on existing cameras, as long as they show the objects clearly. Resolution, angle, lighting, frame rate and video compression decide whether the footage is good enough. Test on recorded images first. A poor angle or very small objects may still call for an extra camera or better lighting.
What does an object detection project cost?
Costs depend mainly on cameras, labelling, integration and maintenance, more than on the model. A pilot on existing cameras with a pretrained model costs far less than a multi-site system with custom classes. Scope one narrow task, then compare the total cost of ownership with measured benefits before you scale.
Where should you start with object detection?
Object detection lets software locate and count what your people now check by eye. It works best on a narrow task, with your own images and a clear rule for each detection.
Pick one visual check that is repetitive, measurable and costly when it fails, and test it on your own images. DataNorth’s computer vision team can help you assess feasibility and run a small pilot alongside your current process.
