6DoF Object Pose Estimation from Calibrated Multi-View Silhouettes

Abstract: Accurate 6DoF object pose estimation is important for robotic perception, surgical assistance, and augmented reality, but remains challenging under occlusion, limited texture, reflective materials, and object symmetries. Many existing methods rely on RGB appearance, depth sensing, or real pose annotations, which can be difficult to obtain or unreliable in realistic settings. This work presents ECASP, a calibrated multi-view approach for 6DoF object pose estimation using only binary object masks and camera calibration. By discarding RGB appearance and depth, the method focuses on silhouette geometry, reducing sensitivity to texture, illumination, material properties, and rendering realism. The model is trained only from CAD-rendered synthetic masks, requiring no real images or real pose annotations. ECASP uses epipolar cross-view attention to fuse silhouette features across calibrated camera views while respecting multi-view geometry. The fused representation is then used by a hybrid pose head that combines learned rotation prediction with geometry-based translation estimation. Evaluation on the MVPSP surgical-object benchmark shows accurate pose estimation on real multi-view data, achieving a mean translation error of 1.58 mm despite using no appearance information. These results demonstrate that calibrated multi-view silhouettes can provide strong geometric constraints for accurate 6DoF pose estimation under occlusion.