Viewpoint-independent Single-view 3D Object Reconstruction using Reinforcement Learning

Seiya Ito, Byeongjun Ju, Naoshi Kaneko, Kazuhiko Sumi

2022

Abstract

This paper addresses the problem of reconstructing 3D object shapes from single-view images using reinforcement learning. Reinforcement learning allows us to interpret the reconstruction process of a 3D object by visualizing sequentially selected actions. However, the conventional method used a single fixed viewpoint and was not validated with an arbitrary viewpoint. To handle images from arbitrary viewpoints, we propose a reinforcement learning framework that introduces an encoder to extract viewpoint-independent image features. We train an encoder-decoder network to disentangle shape and viewpoint features from the image. The parameters of the encoder part of the network are fixed, and the encoder is incorporated into the reinforcement learning framework as an image feature extractor. Since the encoder learns to extract viewpoint-independent features from images of arbitrary viewpoints, only images of a single viewpoint are needed for reinforcement learning. The experimental results show that the proposed method can learn faster and achieves better accuracy than the conventional method.

Paper Citation

in Harvard Style

Ito S., Ju B., Kaneko N. and Sumi K. (2022). Viewpoint-independent Single-view 3D Object Reconstruction using Reinforcement Learning. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 811-819. DOI: 10.5220/0010825900003124

in Bibtex Style

@conference{visapp22,
author={Seiya Ito and Byeongjun Ju and Naoshi Kaneko and Kazuhiko Sumi},
title={Viewpoint-independent Single-view 3D Object Reconstruction using Reinforcement Learning},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP},
year={2022},
pages={811-819},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010825900003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP
TI - Viewpoint-independent Single-view 3D Object Reconstruction using Reinforcement Learning
SN - 978-989-758-555-5
AU - Ito S.
AU - Ju B.
AU - Kaneko N.
AU - Sumi K.
PY - 2022
SP - 811
EP - 819
DO - 10.5220/0010825900003124
PB - SciTePress