Semi-direct Structure from Motion

Hailin Jin
Washington University
Paolo Favaro
Washington University
Stefano Soatto
example example
Project summary
The problem of structure from motion is often decomposed into two steps: feature correspondence and three-dimensional reconstruction. This separation often causes gross errors when establishing correspondence fails. Therefore, we advocate the necessity to integrate visual information not only in time (i.e. across different views), but also in space, by matching regions - rather than points - using explicit photometric deformation models. We present an algorithm that integrates image-feature tracking and three-dimensional motion estimation into a closed loop, while detecting and rejecting outlier regions that do not fit the model. Due to occlusions and the causal nature of our algorithm, a drift in the estimates accumulates over time. We describe a method to perform global registration of local estimates of motion and structure by matching the appearance of feature regions stored over long time periods. We use image intensities to construct a score function that takes into account changes in brightness and contrast. Our algorithm is recursive and suitable for real-time implementation.
Related publications
This research is supported in part by NSF grant IIS-9876145, ARO grant DAAD19-99-1-0139, ONR grant N00014-02- 1-0720, and Intel grant 8029.

2003 Hailin Jin, Paolo Favaro and Stefano Soatto.
Please send your comments to
Last updated on December 16, 2003.

BACK to Hailin Jin's homepage