Discovering Multiple Algorithm Configurations

Leonid Keselman and Martial Hebert
ICRA 2023

[Code]   [Paper]   [Video]


Finding multiple configurations by partitioning the dataset during optimization; here applied to stereo depth.


Abstract

Many practitioners in robotics regularly depend on classic, hand-designed algorithms. Often the performance of these algorithms is tuned across a dataset of annotated examples which represent typical deployment conditions. Automatic tuning of these settings is traditionally known as algorithm configuration. In this work, we extend algorithm configuration to automatically discover multiple modes in the tuning dataset. Unlike prior work, these configuration modes represent multiple dataset instances and are detected automatically during the course of optimization. We propose three methods for mode discovery, a post hoc method, a multi-stage method, and an online algorithm using a multi-armed bandit. Our results characterize these methods on synthetic test functions and in multiple robotics application domains: stereoscopic depth estimation, differentiable rendering, and motion planning. Across all methods, we find appreciable and clear benefits to finding an ensemble or portfolio of solutions for algorithms.


SGBM Test Set Performance


RealSense D435 Partitioning


Paper

Leonid Keselman and Martial Hebert Discovering Multiple Algorithm Configurations ICRA 2023.

[pdf] [bibtex] [video]