FEATS: Synthetic Feature Tracks for Structure from Motion Evaluation

We present FEATS (Feature Extraction and Tracking Simulator), that synthesizes feature tracks using a camera trajectory and scene geometry (e.g. CAD, multi-view stereo). We introduce 2D feature and matching noise models that can be controlled using a few parameters. We also provide a new dataset of images and ground truth camera pose. We process this data (and a synthetic version) with several current SIM algorithms and show that the synthetic tracks are representative of the real tracks. We then show two practical uses of FEATS: (1) we generate hundreds of trajectories with varying noise and show that COLMAP is more robust to noise than OpenSfM and VisualSIM; and (2) we calculate 3D point error and show that accurate camera pose estimates do not guarantee accurate 3D maps.