Manufacturing annotation

Manufacturing Defect Annotation for Quality Control and Factory AI

Labels for inspection cameras, assembly lines and shop-floor video, written so that a scratch, a missing part and a normal surface variation never end up in the same class.

Start a Pilot Explore Services

Compatible workflows

Tools we can work with

Roboflow
SuperAnnotate
Darkhorse
Roboflow
SuperAnnotate
Darkhorse

TL;DR

Defect data is only useful when defect type, location and severity are labelled separately and normal variation has its own rules.
Assembly verification labels say which part should be present, which part is present and whether it is seated correctly.
PPE compliance labels are tied to the person and the zone, so a helmet in a hand is not counted as a helmet on a head.
Egocentric worker video is tagged with hand–object actions and step boundaries, the approach behind our 1M+ frame Factory project.

Defect classes

Defect taxonomy and severity rules for visual inspection

A visual inspection model learns whatever the taxonomy tells it. If scratches, dents and dust share one class, or severity is guessed per annotator, the model inherits the confusion. We fix the taxonomy in writing before production starts.

Defect type, location and extent

Each defect is labelled with its type, such as scratch, dent, crack, porosity, burr, contamination, discoloration or misprint, and with geometry that matches the model. Detection models get tight boxes, while segmentation models get polygons or masks that follow the visible edge of the defect, not the part around it. Thin defects like hairline cracks follow a line or thin-polygon rule written into the guideline.

Where a part shows several defects, each is labelled on its own. Touching defects of different types are split at the visible change in texture or shape, and the rule for when two nearby marks count as one defect is written down with example images.

Severity tiers and pass or fail

Severity is a separate attribute from type. Your quality team defines the tiers, for example cosmetic, functional and critical, and the visible evidence for each one, such as length relative to a reference feature or whether the defect sits on a sealing surface. Annotators apply the tiers from the written definitions and example sheets, never from personal judgement.

Normal variation, such as machining marks, reflections, glare, dust on the lens and acceptable colour shifts, gets explicit not-a-defect rules. Borderline cases are flagged for your engineers rather than forced into a class, and their decisions are added to the guideline as new examples.

Box tightness, polygon and mask rules are explained on our image annotation page.

Assembly line

Assembly verification and production event labelling

Many line-side models do not look for damage at all. They check that the right part is in the right place, in the right order, at the right station.

Presence, position and orientation

For each station, the guideline lists the expected parts and what correct seating looks like. Annotators label each part as present, missing, wrong part or misaligned, and mark orientation for parts that can be fitted backwards. Fasteners, connectors and labels can be labelled as small objects with their own visibility flags when they are partly hidden by fixtures or hands.

Production events in video

In line video, events such as part placed, tool engaged, station cleared or line stop get start and end frames under written boundary rules. Events tied to one product unit carry the same unit identifier across stations when the footage allows it, so a missed step can be traced to the item it affected.

Event boundaries and track checks are covered on our video annotation page.

Safety

PPE compliance and zone rules for factory safety models

Safety models must decide whether a person is protected, not just whether safety equipment appears somewhere in the frame.

Person-linked PPE attributes

Each worker gets a box or track, and PPE items such as helmets, high-visibility vests, gloves, safety glasses, ear protection and masks are labelled as worn, not worn, worn incorrectly or not visible. A helmet carried in a hand or hanging from a belt is labelled as not worn. Not visible is kept separate from not worn, because a hidden glove is not a missing glove.

Zone rules are written per site. A person inside a marked hazard area, near moving machinery or on a forklift route can carry a zone attribute, so the model learns that the same missing item matters more in some places than others.

Worker actions

Egocentric worker action tagging on the shop floor

Head-mounted and chest cameras show the task from the worker's own view: hands, tools, parts and the order of steps. That is the data robotics and task-guidance models need.

Hand–object actions and step boundaries

Actions are tagged as verb and object pairs such as pick bolt, insert connector or tighten screw, each with a start and end frame. Boundary rules state when an action begins, for example the first frame the hand moves toward the object, and when it ends. Hands, tools and parts can also be labelled with boxes or keypoints so the model links each action to the right object.

Our Factory project applied this to more than 1M frames of egocentric factory footage, with action tags, object labels and reviewed step boundaries. Motion blur, gloves and parts hidden inside fixtures are handled with visibility flags and written fallback rules.

Read the Factory egocentric case study for how this worked on more than 1M frames.

Quality

Calibration pilot and two-stage QA for manufacturing data

Every project starts with a calibration pilot on your own images or clips, where we agree the taxonomy, severity tiers and boundary rules before scaling.

Self-check, then independent review

Annotators check their own work against the written guideline. An independent reviewer then audits a sample of each batch, looking for mixed-up defect types, severity drift, missed small defects and actions with loose boundaries. Errors are sent back with the guideline rule they break, and recurring disagreements become new guideline examples under a new version number.

Delivery follows your pipeline: COCO JSON or YOLO for detection and segmentation, CVAT XML for tracks and attributes, and temporal action JSON for worker video.

For hand keypoints and Ego4D-style action phases, see our egocentric video annotation page.

Inspect Sample Deliverable & QA Rubric

Illustrative sample

// Illustrative sample, not client data
{
  "images": [{ "id": 31, "file_name": "station_04_000031.png", "width": 2048, "height": 1536 }],
  "categories": [{ "id": 1, "name": "scratch" }, { "id": 2, "name": "dent" }],
  "annotations": [{
    "id": 901, "image_id": 31, "category_id": 1,
    "bbox": [812, 604, 140, 18],
    "segmentation": [[812, 610, 950, 604, 952, 614, 814, 622]],
    "attributes": { "severity": "cosmetic", "surface": "non_sealing", "borderline": false }
  }]
}

Stage 1 — Annotator self-check

Every object in the label set is tagged
Boxes/segments are tight to visible edges
Class names match the guideline exactly
Occlusion and truncation flags set

Stage 2 — Independent reviewer audit

Independent reviewer re-checks a sampled batch
Errors logged against the rubric by type
Batch returned if it misses the accuracy threshold
Edge cases added back to the guideline

Manufacturing defect annotation FAQ

How do you keep defect severity consistent between annotators?

Severity is a separate attribute defined by your quality team with visible evidence and example sheets for each tier. Reviewers audit severity on its own, and borderline cases are flagged for your engineers instead of guessed.

How do you stop reflections and machining marks being labelled as defects?

The guideline has explicit not-a-defect rules with example images for normal variation such as glare, dust, tool marks and acceptable colour shifts. Disputed examples from the calibration pilot become new reference cases.

How is PPE labelled when equipment is hidden?

Each PPE item on a worker is labelled as worn, not worn, worn incorrectly or not visible. Not visible is kept separate from not worn, so the model is never taught that a hidden item is missing.

Can you label egocentric worker video for robotics?

Yes. We tag verb and object actions with written start and end rules, and can add hand, tool and part labels. This is the approach used on our Factory project of more than 1M frames.

Send us your hardest defect images.

Start with a Free Calibration Pilot