LiDAR Point Cloud Annotation: Cuboids, Fusion and QA

What a point cloud is

A LiDAR sensor sends out laser pulses and measures how long they take to return. Each return becomes a point with X, Y, Z coordinates, often with intensity. A single sweep can contain a very large number of points; close objects are dense, far objects are sparse.

That sparsity is the core challenge: a pedestrian at long range may be only a handful of points.

Main annotation types

3D cuboids

A 3D box with position, dimensions and yaw (heading). Used for vehicles, pedestrians, cyclists and other road users. Rules must define:

• whether the box fits the visible points or the estimated full object;
• how heading is set for objects that are stationary or seen from behind;
• minimum points before an object must be labeled.

Point cloud segmentation

Each point is assigned a class: road, sidewalk, vegetation, building, vehicle and so on. Used for drivable area, mapping and scene understanding.

Lanes, curbs and freespace

Polylines or regions marked on the ground plane for lane-level planning and free-space estimation.

Multi-frame tracking

The same object keeps one ID across a sequence. This lets models learn motion and helps reviewers spot boxes that jump or change size between frames.

Camera-LiDAR sensor fusion

Many vehicles carry several cameras and one or more LiDARs. Fusion annotation links them:

• a 3D cuboid is projected onto each camera image to check alignment;
• 2D boxes in images and 3D boxes in the cloud share the same object ID;
• calibration files (intrinsic and extrinsic) must be correct, or projections will not line up.

Good fusion work catches errors that are invisible in one view alone, such as a cuboid that looks fine in 3D but sits on the wrong car in the image.

Common edge cases

• Very distant or partly hidden objects with few points.
• Reflective surfaces and rain or fog creating noise.
• Groups of pedestrians close together.
• Trailers, articulated buses and unusual vehicles.
• Objects split across the edge of a sweep.

Each needs a written rule with an example.

QA checks that matter for 3D data

1. Dimension consistency: the same car should not change size between frames.
2. Heading checks: headings should follow the direction of travel for moving objects.
3. Track continuity: IDs should not switch between objects.
4. Projection checks: projected cuboids should align with objects in camera images.
5. Coverage: every object above the agreed point threshold is labeled.
6. Agreement metrics: 3D IoU between two annotators on a sample shows how consistent the team is.

Practical tips for buyers

• Share sensor specs and calibration files at the start.
• Agree on coordinate systems and export format (for example a common open dataset format or your own schema).
• Pilot on your hardest scenes, not your easiest.
• Decide whether labels are per frame or per tracked sequence before pricing.