Iterative 3D Deformable Registration from Single-view RGB Images using Differentiable Rendering

Arul Selvam Periyasamy, Max Schwarz, Sven Behnke

2022

Abstract

For autonomous robotic systems, comprehensive 3D scene parsing is a prerequisite. Machine learning techniques used for 3D scene parsing that incorporate knowledge about the process of 2D image generation from 3D scenes have a big potential. This has sparked an interest in differentiable renderers that provide approximate gradients of the rendered image with respect to scene and object parameters. An efficient differentiable renderer facilitates approaching many 3D scene parsing problems using a render-and-compare framework, where the object and scene parameters are optimized by minimizing the difference between rendered and observed images. In this work, we introduce StilllebenDR, a light-weight scalable differentiable renderer built as an extension to the Stillleben library and use it for 3D deformable registration from single-view RGB images. Our end-to-end differentiable pipeline achieves results comparable to state-of-the-art methods without any training and outperforms the competing methods significantly in the presence of pose initialization errors.

Paper Citation

in Harvard Style

Periyasamy A., Schwarz M. and Behnke S. (2022). Iterative 3D Deformable Registration from Single-view RGB Images using Differentiable Rendering. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 107-116. DOI: 10.5220/0010817100003124

in Bibtex Style

@conference{visapp22,
author={Arul Selvam Periyasamy and Max Schwarz and Sven Behnke},
title={Iterative 3D Deformable Registration from Single-view RGB Images using Differentiable Rendering},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP},
year={2022},
pages={107-116},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010817100003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP
TI - Iterative 3D Deformable Registration from Single-view RGB Images using Differentiable Rendering
SN - 978-989-758-555-5
AU - Periyasamy A.
AU - Schwarz M.
AU - Behnke S.
PY - 2022
SP - 107
EP - 116
DO - 10.5220/0010817100003124
PB - SciTePress