To address this, we present a unified Hand–Multiple–Eyes (HME) calibration framework that enforces closed-loop consistency across all sensor-to-robot and inter-sensor transformations. We define “closed-loop calibration” as the process of estimating each sensor’s pose while simultaneously ensuring that the entire set of transformations forms a coherent pose graph.
The core idea of this work is an equality-constrained optimization formulation that incorporates the natural loop structures arising in multi-sensor systems. By combining pairwise pose equations with loop-closure constraints, our method guarantees geometric consistency even under noise and limited motion excitation.
Three solution strategies are explored: an unconstrained baseline, a variable elimination scheme that embeds the constraints into a reduced parameter space, and a Lagrange-multiplier formulation that enforces them directly. The approach generalizes naturally to larger sensor networks using a minimal triangle loop basis that avoids redundant constraints.
Comprehensive simulation and real-world experiments on a UR10e robot equipped with multiple cameras show that the constrained methods achieve near machine-precision loop closure and high calibration accuracy, enabling robust multi-sensor perception pipelines.
