LiDAR & 3D annotation

LiDAR & 3D Point Cloud Annotation Services

LiDAR annotation labels 3D point clouds with cuboids, point classes and track IDs so perception models can locate objects in space. VIPRO also checks 2D-3D consistency by projecting cuboids onto calibrated camera images, under written rules and two-stage QA.

Point clouds and fused camera data labelled with cuboids, point classes and tracks, under written rules for orientation, sparse objects and sensor calibration.

Start a Pilot Explore Services

Compatible workflows

Tools we can work with

Roboflow
SuperAnnotate
Darkhorse
Roboflow
SuperAnnotate
Darkhorse

TL;DR

3D cuboids with centre, dimensions and yaw, drawn to written tight-fit and orientation rules your perception stack can rely on.
Camera–LiDAR fusion: cuboids are projected into each calibrated image and checked for consistency, so 2D and 3D labels agree.
Lane, freespace and ground segmentation, with elevation and intensity filtering to make sparse and far-range objects reviewable.
Multi-frame sequences with persistent track IDs, two-stage QA, and per-frame JSON following KITTI-style or nuScenes-style conventions.

3D geometry

3D point cloud cuboids: position, dimensions and yaw

A cuboid is a claim about where an object is in the world, how big it is and which way it faces. Each of those three needs a written rule, or two annotators will produce two different cars from the same points.

Position and dimensions

Each cuboid is stored as a centre point (x, y, z) in an agreed coordinate frame, usually the ego-vehicle or LiDAR sensor frame, plus length, width and height in metres. The guideline names the frame, the axis directions and the units before any labelling starts, because a frame mix-up silently corrupts every label in a batch.

How orientation is defined

Yaw is the rotation of the object around the vertical axis, measured in radians from the frame's forward axis. The guideline defines what the front of each class is: the bonnet of a car, the direction a pedestrian is facing or walking, the cab of a truck. For symmetric or ambiguous objects, such as a stationary bin, the rule is either a fixed convention or a direction-of-travel rule.

Pitch and roll are normally fixed at zero for road scenes. They are labelled only when your model needs them, for example on ramps or for tipped-over objects.

Tight-fit rules

The cuboid encloses all points belonging to the object and as little else as possible. Mirrors, antennas and loose cargo are included or excluded according to the class definition. When the object is only partly scanned, annotators estimate the full extent from class priors and the visible geometry, and flag the cuboid as estimated.

Sensor fusion

2D–3D sensor fusion: camera and LiDAR working together

Most perception stacks combine LiDAR with cameras. Fused labelling uses each sensor where it is strongest: LiDAR for range and shape, the camera for class, colour and fine detail.

Calibration

Fusion depends on the intrinsic camera parameters and the extrinsic transform between camera and LiDAR, plus synchronised timestamps. We work from the calibration you supply and check it on sample frames before production: projected points should land on the edges they belong to, such as poles, kerbs and vehicle outlines.

Projecting cuboids into the image

Each cuboid is projected into every camera that sees it, producing a 2D box or outline. Annotators use the image to confirm class and orientation, then correct the 3D label. The 2D image is never adjusted to match a wrong 3D label.

Consistency checks

Reviewers look for cuboids whose projection misses the object in the image, objects visible in the camera with no 3D label, and offsets that grow with distance or appear in one camera only. That last pattern usually points to calibration drift or a timing offset, and is reported back to you rather than patched label by label.

The 2D side of fused work follows our image annotation rules for boxes and masks.

Surfaces

Lane and freespace segmentation

Not everything in a point cloud is an object. Drivable surface, lane markings and boundaries are labelled as point classes or polylines.

Lanes

Lane markings are labelled as 3D polylines or as point classes, with attributes for marking type (solid, dashed, double) and colour when the camera shows it. Intensity often makes paint visible in the cloud even at night.

Freespace

Freespace is the surface the vehicle could physically drive on. The guideline settles the edge cases: whether kerbs, gravel shoulders and parking bays count, and how to handle areas the sensor can't see because something is blocking them.

Street-scene rules in practice: the CityScope AI urban perception case study.

Making points readable

Elevation and intensity filtering, and ground removal

A raw sweep is a dense, flat-looking cloud. Filtering is how annotators make the right points stand out, and its settings are part of the guideline, not left to personal preference.

Ground removal

Ground points are separated, or hidden for viewing, so low objects such as kerbs, debris and small animals stand out. On slopes and uneven surfaces a single height threshold fails, so annotators work with local ground estimates and confirm low objects in the camera view.

Elevation and intensity

Colouring by height separates overhanging structures, such as trees and signs, from objects on the road. Intensity, the strength of the returned signal, highlights retro-reflective surfaces like lane paint, number plates and signs. It helps with classes and markings, but it varies by sensor and surface, so it isn't used alone to decide a class.

Hard cases

Sparse far-range objects and occlusion

Point density falls off sharply with distance, and objects behind other objects may return only a handful of points. These are the labels models most need and reviewers most often disagree on.

Sparse far-range objects

The guideline sets a minimum evidence rule for each class: how many points, or how much camera support, is needed before an object is labelled. Below that threshold the object is either skipped or labelled with a low-confidence flag, depending on your training setup. Dimensions for sparse objects are informed by class priors and nearby frames in the sequence.

Occlusion

Each cuboid carries an occlusion level. Partly hidden objects are labelled at their full estimated extent when the rest of the object can be reasonably inferred, and the occlusion level tells your model how much to trust it.

Time

Multi-frame sequences and track IDs

Sequential sweeps let annotators use motion to resolve ambiguity, and let your model learn tracking and prediction.

Persistent IDs and consistent sizes

Each object keeps one track ID across the sequence. A rigid object's dimensions stay constant across frames, so once a car's size is settled from its densest frames, it is reused while position and yaw are updated. This removes the size jitter that per-frame labelling creates.

Objects leaving and re-entering the field of view are checked by a reviewer before an old ID is reused. Velocity or motion-state attributes can be added when your model needs them.

Tracking and re-identification principles are shared with video annotation.

Quality control

QA stages for point cloud annotation

Every project starts with a calibration pilot and runs on written rules, followed by two-stage review: the annotator's self-check, then an independent reviewer audit.

What reviewers check

Reviewers check class, centre, dimensions and yaw against the rubric in both the 3D view and the camera projection. They also check missed and extra objects, track ID continuity and attribute values. Errors are logged by type, failed batches are returned for rework, and recurring edge cases go back into the guideline.

Delivery

Delivery formats

We deliver in the convention your training pipeline already reads.

Per-frame JSON and common conventions

A typical delivery is one JSON file per frame listing each object's class, track ID, centre, dimensions, yaw and attributes, with projected 2D boxes in COCO JSON when cameras are in scope. We can follow KITTI-style conventions (camera-frame boxes in text files per frame) or nuScenes-style conventions (annotations linked to samples with quaternion rotation). The exact schema, coordinate frame and units are agreed in the pilot and tested against your loader.

Inspect Sample Deliverable & QA Rubric

Illustrative sample

{
  "info": "Illustrative sample — not client data",
  "frame_id": "seq01_000120",
  "coordinate_frame": "lidar_top",
  "units": "meters, radians",
  "objects": [{
    "track_id": "car_0007", "label": "car",
    "center": { "x": 18.42, "y": -3.15, "z": -0.62 },
    "dimensions": { "length": 4.52, "width": 1.86, "height": 1.55 },
    "yaw": 1.571,
    "attributes": { "occlusion": "partial", "estimated_extent": false }
  }]
}

Stage 1 — Annotator self-check

Every in-scope object labelled above the evidence threshold
Cuboids tight to points; yaw follows the class front rule
Projection checked in each camera that sees the object
Track IDs and fixed dimensions kept across frames

Stage 2 — Independent reviewer audit

Independent reviewer audits sampled frames in 3D and 2D
Errors logged: missed, extra, class, position, size, yaw
Calibration or timing offsets reported, not patched
Failed batches returned for rework before delivery

LiDAR annotation technical FAQ

How is cuboid yaw defined?

Yaw is the rotation around the vertical axis, in radians, measured from the forward axis of the agreed coordinate frame. The guideline defines the front of each class, such as the bonnet of a car or a pedestrian's facing direction, and gives a fixed convention for symmetric objects so orientation is never left to personal judgement.

How do you handle sparse objects at far range?

Each class has a written minimum-evidence rule based on point count and camera support. Objects below the threshold are skipped or flagged as low confidence, depending on your setup. Dimensions for sparse objects use class priors and denser frames from the same track.

What happens if camera–LiDAR calibration drifts?

Reviewers watch for projections that miss their objects, especially offsets that grow with distance or appear in one camera only. These point to calibration drift or timing offsets. We report them to you with example frames instead of quietly adjusting labels, because patched labels would hide a sensor problem.

How is ground filtering done on slopes and uneven roads?

A single height threshold fails on slopes, so annotators use local ground estimates and confirm low objects such as kerbs or debris in the camera view. Ground removal is a viewing aid. Ground and freespace are labelled as classes only when your model needs them.

How is LiDAR intensity used during annotation?

Intensity highlights retro-reflective surfaces, such as lane paint, signs and number plates, which helps with markings and some classes. Because it varies with sensor, range, angle and surface, it supports a decision alongside geometry and camera evidence but never decides a class on its own.

Test us on your hardest sweeps first.

Start with a Free Calibration Pilot