.png){kind=icon}
Return
ReN Human: Learning Relightable Neural Implicit Surfaces for Animatable Human Rendering
.png){kind=icon}
PRE
AI
DOI：10.1145/3678002.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Recently, implicit neural representation has been widely used to learn the appearance of human bodies in the canonical space, which can be further animated using a parametric human model. However, how to decompose the material properties from the implicit representation for relighting has not yet been investigated thoroughly. We propose to address this problem with a novel framework, ReN Human, that takes sparse or even monocular input videos collected in unconstrained lighting to produce a 3D human representation that can be rendered with novel views, poses, and lighting. Our method represents humans as deformable implicit neural representation and decomposes the geometry, material of humans as well as environment illumination for capturing a relightable and animatable human model. Moreover, we introduce a volumetric lighting grid consisting of spherical Gaussian mixtures to learn the spatially varying illumination and animatable visibility probes to model the dynamic self-occlusion caused by human motion. Specifically, we learn the material property fields and illumination using a physically-based rendering layer that uses Monte Carlo importance sampling to facilitate differentiation of the complex rendering integral. We demonstrate that our approach outperforms recent novel views and poses synthesis methods in a challenging benchmark with sparse videos, enabling high-fidelity human relighting.
Keywords:
Relighting
sign distance function
animatable human
inverse rendering
volume rendering
Journal
IF:
9.5
Papers:
4.8K
Citations:
3.6W
Organization
View more organizations
Cited Papers
ShareSave
Significance of the Protein Interface Configuration for Allostery in Imidazole Glycerol Phosphate Synthase.png)
Biochemistry
BiochemistryIF0
PREAI
ShareSaveDoes the Complexity of the Task Influence Manual Laterality in De Brazza’s Monkeys (Cercopithecus neglectus)?Ethology
EthologyIF0
PREAI
ShareSaveThe reaction of 1,2,2-trimethylbicyclo[1.1.0]butane with excited state 1-cyanonaphthaleneTetrahedron
TetrahedronIF0
PREAI
ShareSaveShareSave
View more