Automotive annotation

Automotive Data Annotation for ADAS and Autonomous Driving

Perception labels for driver assistance and self-driving stacks, written around the scenes that break models: darkness, weather, glare, crowded junctions and objects only half in view.

Start a Pilot Explore Services

Compatible workflows

Tools we can work with

Roboflow
SuperAnnotate
Darkhorse
Roboflow
SuperAnnotate
Darkhorse

TL;DR

Perception data for ADAS and autonomous driving is decided by its edge cases, so night, rain, glare and fog get their own written rules, not a footnote.
We label camera images, video sequences and LiDAR point clouds, and check that 2D and 3D labels agree when sensors are fused.
Lanes, road edges and freespace follow geometry rules for worn paint, merges, construction zones and wet reflections.
Occlusion and truncation are flagged per object, so your model knows the difference between a hidden pedestrian and an absent one.

Perception tasks

What ADAS and autonomous driving perception data has to cover

A driving model has to find road users, read the road and predict motion, often from several sensors at once. Each of those jobs needs its own label layer and its own acceptance rules.

Road users and traffic control

Vehicles, pedestrians, cyclists, motorcyclists and animals are labelled with boxes, polygons or cuboids depending on the model. Traffic lights, signs and cones get class and state attributes such as light colour, sign type or whether a sign faces the ego vehicle. The taxonomy is fixed before production, including how to treat rare classes like mobility scooters, trailers and emergency vehicles.

Attributes carry much of the safety signal. A parked car and a car pulling out look alike in a single frame, so motion and parking state are labelled from the sequence, not guessed from one image.

Tracking across time

In video and multi-frame LiDAR, each road user keeps one track ID from entry to exit. Reviewers watch tracks on playback to catch identity swaps at crossings, behind buses and in dense traffic, where single-frame checks miss them.

Box, polygon and segmentation rules are covered in depth on our image annotation page.

Edge cases

Night, rain, glare and fog: rules for the scenes that fail

Clear daytime scenes are rarely where a perception model fails. The guideline spends most of its pages on degraded conditions, because that is where annotators disagree most.

Low light and night driving

At night, many objects are visible only as headlights, tail lights or a partial silhouette. The guideline states when a light pair is enough to place a vehicle box, how far the box extends into darkness, and when an object must be marked as uncertain instead of drawn from imagination. Frames are reviewed at the brightness annotators actually worked with, so reviewers see the same evidence.

Rain, spray, glare and fog

Rain drops on the lens, road spray, low sun and fog each hide objects in a different way. We add scene-level condition tags, such as wet road, heavy rain, sun glare or fog, so you can measure performance per condition. Object labels in these scenes carry a visibility attribute, and reflections on wet tarmac are never labelled as objects.

When a frame is too degraded to label honestly, it is flagged rather than forced. A clear skip reason is more useful to a model team than a confident-looking box nobody could verify.

Multi-sensor fusion

Camera and LiDAR fusion labelling

Fused datasets are only useful when the 2D and 3D labels describe the same object in the same place. Most of the extra quality work in automotive projects sits here.

Calibration and projection checks

Using the calibration you supply, 3D cuboids are projected into each camera image and compared with the 2D labels. Mismatches point either to a labelling error or to calibration drift. Annotators report suspected drift on a sequence instead of nudging labels to hide it, and the issue goes back to your team.

Shared IDs across sensors

An object labelled in the point cloud and in three cameras keeps one identity across all of them. Consistency checks look for objects present in one sensor and missing in another without a written reason, such as being outside a camera's field of view.

Cuboid fields, ground filtering and sparse far-range objects are explained on the LiDAR and 3D point cloud annotation page.

Road geometry

Lane, road edge and freespace annotation

Road layout labels look simple until the paint is worn, the lane splits or a work zone replaces the markings. The guideline covers those cases one by one.

Lanes and road edges

Lane lines are drawn as polylines with type attributes: solid, dashed, double, temporary or virtual. Where markings fade, the rule says whether to continue the line through the gap or stop it. Curbs, barriers and grass edges are labelled as road boundaries separately from painted lines.

Drivable freespace

Freespace marks where the ego vehicle could physically drive, so it stops at curbs, parked cars and obstacles. It is labelled as a pixel or polygon region, and its edge is checked against the boundary and object layers so the three layers never contradict each other.

Occlusion

Occlusion and truncation rules for road scenes

Junctions, parked rows and large vehicles hide a lot of the scene. How hidden objects are labelled decides whether the model learns caution or false confidence.

Visible extent versus full extent

The project decides once whether boxes cover only visible pixels or the estimated full object, and the rule never changes mid-batch. Each object gets an occlusion level and a truncation flag when it is cut by the image border, so training code can filter or weight them.

A pedestrian between parked cars may be visible only as a head and shoulders. They are still labelled, with a high occlusion level, because that is exactly the case the model must learn.

See how these rules were applied to street scenes in the CityScope AI urban perception case study.

Quality

Calibration pilot and two-stage QA

Every automotive project starts with a calibration pilot on your hardest scenes. Disagreements become written rules before production begins.

How review works

In stage one, annotators check their own work against the guideline checklist. In stage two, an independent reviewer audits samples per batch, with extra weight on night, weather and dense-traffic scenes. Rejected items go back with the rule they broke, and new edge cases become guideline updates with a version number.

Track IDs, interpolation and correction passes are covered on our video annotation page.

Inspect Sample Deliverable & QA Rubric

Illustrative sample

// Illustrative sample, not client data
{
  "frame_id": "seq_014_f000120",
  "sensor": "lidar_top",
  "scene_tags": ["night", "wet_road"],
  "objects": [{
    "track_id": 37,
    "class": "pedestrian",
    "center": [12.4, -3.1, 0.9],
    "size_lwh": [0.6, 0.7, 1.7],
    "yaw": 1.57,
    "occlusion": "partial",
    "num_points": 41
  }]
}

Stage 1 — Annotator self-check

Every object in the label set is tagged
Boxes/segments are tight to visible edges
Class names match the guideline exactly
Occlusion and truncation flags set

Stage 2 — Independent reviewer audit

Independent reviewer re-checks a sampled batch
Errors logged against the rubric by type
Batch returned if it misses the accuracy threshold
Edge cases added back to the guideline

Automotive annotation FAQ

How do you label objects that are only visible as headlights at night?

The guideline defines the minimum evidence for a label. A clear light pair with a consistent position over several frames can support a vehicle box sized to the agreed rule. When the evidence is weaker, the object is marked uncertain or the frame is flagged, instead of drawing a confident box nobody could verify.

Do boxes cover the visible part of an occluded object or its full extent?

Either, depending on your model. We agree the convention in the calibration pilot and apply it to every batch. Each object also carries an occlusion level and a truncation flag, so you can filter or weight hidden and cut-off objects in training.

How do you keep camera and LiDAR labels consistent?

3D cuboids are projected into each camera with your calibration and compared with the 2D labels. Objects keep one ID across sensors. Mismatches are reviewed, and suspected calibration drift is reported back to you rather than hidden by adjusting labels.

How are worn or missing lane markings handled?

Lane polylines carry type attributes, and the guideline states when to continue a line through faded paint and when to stop it. Construction-zone markings and virtual lanes are labelled as their own types, and road edges are kept separate from painted lines.

Send us the scenes your model gets wrong.

Start with a Free Calibration Pilot