Skip to content

Camera extrinsics

Cameras are calibrated against a Lidar or an IMU from ordinary motion, without a target. As in iKalibr, each camera serves as a gyroscope: features are tracked between images, the rotation between images is fitted, and it is aligned with the other sensor's rotation.

Intrinsics come from sensor_msgs/msg/CameraInfo (plumb_bob, rational_polynomial, equidistant) or from a camchain passed with --intrinsics, see camera intrinsics. Errors in the intrinsics carry into the rotation: a principal point a few pixels off tilts every ray.

Cameras are named by image topic. The camera info is matched by namespace, or given as IMAGE=CAMERA_INFO:

--camera /cam0/image_raw/compressed --camera /cam1/image_raw=/cam1/camera_info

Several cameras are tracked in one pass and also reported relative to the first.

Camera to IMU

dedrunk extrinsics camera-imu recording --camera /cam0/image_raw -O camchain-imucam.yaml

Rotation and clock offset against the gyroscope, written as a Kalibr camchain-imucam.yaml (T_cam_imu, T_cn_cnm1, timeshift_cam_imu). Camera and IMU do not determine the translation, so T_cam_imu has a zero translation.

Lidar to camera

dedrunk extrinsics lidar-camera lidar_traj --image-bag recording --camera /cam0/image_raw

The reference is a Lidar-only trajectory, as for Lidar to IMU. Two stages:

  1. Rotation and clock offset from the camera's rotation against the trajectory's.
  2. Translation and refined rotation by aligning Lidar depth edges with image edges (Levinson and Thrun, RSS 2013). Every few seconds the sharpest image becomes a keyframe. The raw scans around it are deskewed with the trajectory and accumulated in the Lidar frame. The near side of each range discontinuity along a ring is an edge point. The transform that best places these points on image edges is searched starting from stage 1. Only geometry is used, not intensity.

Stage 2 needs the raw PointCloud2 scans with a ring field in the image bag. Per-point times are read from timestamp, time, t or offset_time. Without them a scan is treated as instantaneous.

A result is flagged when

  • refits on random halves of the keyframes differ by more than 0.5° or 3 cm. The translation is then reported as not measured.
  • a camera's clock offset differs from the other cameras' by more than 10 ms. Cameras on one rig share a clock.
  • stage 1 has a large residual or frequent tracking restarts.

The last two widen the stage 2 search. A camera whose motion does not determine the rotation is left out, the others finish. Keyframes with fewer than 50 edge points in view are dropped.

--lever-arm TOPIC=x,y,z starts the search from a known lever arm. -o lidar_camera.fix_translation=true keeps it and refines only the rotation.

On GrandTour arc2 (ANYmal, Hesai XT32, Alphasense) three minutes of walking gave all five cameras within 0.4 to 1.0° of the dataset's calibration. The translations were within 2.6 to 8.3 cm. The right camera's translation was flagged as not measured.

--bundle calibration.dedrunk saves keyframes and scans for the GUI, which shows and adjusts the result.

Recording

  • Rotation about all three axes. A single axis leaves the rotation undetermined.
  • Sharp, textured images: short exposure, enough light.
  • For the translation, objects with clear outlines a few metres away: posts, door frames, building corners, vehicles.

The camera settings control tracking and the rotation fit, lidar_camera the edge alignment, see the annotated defaults.