Camera extrinsics¶
Cameras are calibrated against a Lidar or an IMU from ordinary motion, without a target. As in iKalibr, each camera serves as a gyroscope: features are tracked between images, the rotation between images is fitted, and it is aligned with the other sensor's rotation.
Intrinsics come from sensor_msgs/msg/CameraInfo (plumb_bob,
rational_polynomial, equidistant) or from a camchain passed with
--intrinsics, see camera intrinsics. Errors in the
intrinsics carry into the rotation: a principal point a few pixels off tilts
every ray.
Cameras are named by image topic. The camera info is matched by namespace, or
given as IMAGE=CAMERA_INFO:
Several cameras are tracked in one pass and also reported relative to the first.
Camera to IMU¶
Rotation and clock offset against the gyroscope, written as a Kalibr
camchain-imucam.yaml (T_cam_imu, T_cn_cnm1, timeshift_cam_imu). Camera
and IMU do not determine the translation, so T_cam_imu has a zero
translation.
Lidar to camera¶
The reference is a Lidar-only trajectory, as for Lidar to IMU. Two stages:
- Rotation and clock offset from the camera's rotation against the trajectory's.
- Translation and refined rotation by aligning Lidar depth edges with image edges (Levinson and Thrun, RSS 2013). Every few seconds the sharpest image becomes a keyframe. The raw scans around it are deskewed with the trajectory and accumulated in the Lidar frame. The near side of each range discontinuity along a ring is an edge point. The transform that best places these points on image edges is searched starting from stage 1. Only geometry is used, not intensity.
Stage 2 needs the raw PointCloud2 scans with a ring field in the image
bag. Per-point times are read from timestamp, time, t or offset_time.
Without them a scan is treated as instantaneous.
A result is flagged when
- refits on random halves of the keyframes differ by more than 0.5° or 3 cm. The translation is then reported as not measured.
- a camera's clock offset differs from the other cameras' by more than 10 ms. Cameras on one rig share a clock.
- stage 1 has a large residual or frequent tracking restarts.
The last two widen the stage 2 search. A camera whose motion does not determine the rotation is left out, the others finish. Keyframes with fewer than 50 edge points in view are dropped.
--lever-arm TOPIC=x,y,z starts the search from a known lever arm.
-o lidar_camera.fix_translation=true keeps it and refines only the rotation.
On GrandTour arc2 (ANYmal, Hesai XT32, Alphasense) three minutes of walking gave all five cameras within 0.4 to 1.0° of the dataset's calibration. The translations were within 2.6 to 8.3 cm. The right camera's translation was flagged as not measured.
--bundle calibration.dedrunk saves keyframes and scans for the
GUI, which shows and adjusts the result.
Recording¶
- Rotation about all three axes. A single axis leaves the rotation undetermined.
- Sharp, textured images: short exposure, enough light.
- For the translation, objects with clear outlines a few metres away: posts, door frames, building corners, vehicles.
The camera settings control tracking and the rotation fit, lidar_camera
the edge alignment, see the annotated defaults.