3DCaricShop: A Dataset and A Baseline Method for Single-view 3D Caricature Face Reconstruction


Yuda Qiu1,2      Xiaojie Xu2,4      Lingteng Qiu1,2      Yan Pan1,2      Yushuang Wu1,2      Weikai Chen3      Xiaoguang Han#1,2*     

*Corresponding email: hanxiaoguang@cuhk.edu.cn

1The Chinese University of Hong Kong, Shenzhen       2Shenzhen Research Institute of Big Data       3Tencent America       4University of Science and Technology of China      



Introduction

Caricature is an artistic representation that deliberately exaggerates the distinctive features of a human face to convey humor or sarcasm. However, reconstructing a 3D caricature from a 2D caricature image remains a challenging task, mostly due to the lack of data. We propose to fill this gap by introducing 3DCaricShop, the first largescale 3D caricature dataset that contains 2000 high-quality diversified 3D caricatures manually crafted by professional artists. 3DCaricShop also provides rich annotations including a paired 2D caricature image, camera parameters and 3D facial landmarks. To demonstrate the advantage of 3DCaricShop, we present a novel baseline approach for single-view 3D caricature reconstruction. To ensure a faithful reconstruction with plausible face deformations, we propose to connect the good ends of the detailrich implicit functions and the parametric mesh representations. In particular, we first register a template mesh to the output of the implicit generator and iteratively project the registration result onto a pre-trained PCA space to resolve artifacts and self-intersections. To deal with the large deformation during non-rigid registration, we propose a novel view-collaborative graph convolution network (VCGCN) to extract key points from the implicit mesh for accurate alignment. Our method is able to generate high-fidelity 3D caricature in a pre-defined mesh topology that is animation-ready. Extensive experiments have been conducted on 3DCaricShop to verify the significance of the database and the effectiveness of the proposed method.

Video

Coming soon...

Publication

Paper - Coming soon | GitHub

If you find our work useful, please consider citing it: 

        
        Coming soon

Dataset


We contribute to 3DCaricShop dataset, a large collection to caricature face images and the corresponding 3D models manully crafted. The dataset proposed has serveral appealing features: