NeMF: Inverse Volume Rendering with Neural Microflake Field

  • Youjia Zhang1
  • Teng Xu1
  • Junqing Yu1
  • Yuteng Ye1

    1Huazhong University of Science and Technology    2Tencent    3Shanghai Tech University

Abstract

overview

Recovering the physical attributes of an object's appearance from its images captured under an unknown illumination is challenging yet essential for photo-realistic rendering. Recent approaches adopt the emerging implicit scene representations and have shown impressive results.However, they unanimously adopt a surface-based representation,and hence can not well handle scenes with very complex geometry, translucent object and etc.In this paper, we propose to conduct inverse volume rendering, in contrast to surface-based, by representing a scene using microflake volume, which assumes the space is filled with infinite small flakes and light reflects or scatters at each spatial location according to microflake distributions. We further adopt the coordinate networks to implicitly encode the microflake volume, and develop a differentiable microflake volume renderer to train the network in an end-to-end way in principle.Our NeMF enables effective recovery of appearance attributes for highly complex geometry and scattering object, enables high-quality relighting, material editing, and especially simulates volume rendering effects, such as scattering, which is infeasible for surface-based approaches.

overview

Video

Experiments

BibTeX

        
        @misc{zhang2023nemf,
            title={NeMF: Inverse Volume Rendering with Neural Microflake Field}, 
            author={Youjia Zhang and Teng Xu and Junqing Yu and Yuteng Ye and Junle Wang and Yanqing Jing and Jingyi Yu and Wei Yang},
            year={2023},
            eprint={2304.00782},
            archivePrefix={arXiv},
            primaryClass={cs.CV}
        }