FaceCLIP:CLIP-driven accurate and detailed 3D face reconstruction from a single imageOA
In recent years,3D face reconstruction has become a research hotspot in computer graphics and computer vision.Most current 3DMM-based methods focus on learning displacement maps to recover high-frequency facial details.However,they focus less on learning mid-frequency facial details and introduce displacement maps with noise,decreasing face reconstruction accuracy.Thus,this work presents a novel approach to regressing accurate and detailed 3D face shapes.First,we design a novel feature consistency loss to recover mid-frequency facial details.Specifically,we exploit the powerful CLIP as prior knowledge of faces to extract geometric and semantic features,which helps guide the reconstructed 3D geometric details to match local details in the input image.Furthermore,we propose a parameter refinement module to learn fine-grained features.It helps to obtain accurate model parameters and improve the accuracy of facial reconstruction.Extensive experiments on a FaceScape and a REALY benchmark demonstrate that our method outperforms several state-of-the-art methods in reconstruction accuracy.Furthermore,comprehensive qualitative results show that our approach achieves better visual performance than existing methods.
Yongtang Bao;Pengfei Zhou;Liang Qi;Yue Qi;Haojie Li
School of Computer Science and Engineering,Shandong University of Science and Technology,Qingdao 266590,China State Key Laboratory of Virtual Reality Technology and Systems,Beihang University,Beijing 100191,ChinaState Key Laboratory of Virtual Reality Technology and Systems,Beihang University,Beijing 100191,ChinaSchool of Computer Science and Engineering,Shandong University of Science and Technology,Qingdao 266590,ChinaState Key Laboratory of Virtual Reality Technology and Systems,Beihang University,Beijing 100191,China Virtual Reality Research Institute,Beihang University Qingdao Research Institute,Qingdao 266100,ChinaSchool of Computer Science and Engineering,Shandong University of Science and Technology,Qingdao 266590,China
信息技术与安全科学
3D face reconstructionCLIPsingle image
《Computational Visual Media》 2026 (1)
P.85-103,19
Funded by grants from the Qingdao Science and Technology for the People Demonstration Project,China(25-1-5-smjk-18-nsh)the National Natural Science Foundation of China(62072020)the Taishan Scholar Program of Shandong Province(tstp20221128)the Talented Young Teachers Training Program of Shandong University of Science and Technology(BJ20231201).
评论