Hi! My name is Grace (a.k.a Gege). I am currently a research staff at National Laboratory of Pattern Recognition (NLPR) in the Institute of Automation (IA), Chinese Academy of Sciences (CAS), Beijing, China, working with Prof. Dr. Ran He. I'll be a Ph.D. student in the Autonomous Vision Group (AVG) starting Spring 2022, advised by Prof. Dr. Andreas Geiger.
My research interest is at the intersection of vision and learning. Currently, my research focuses on controllable image synthesis, as it provides a way to bridge the human will with artificial machines. I'm also working on causal representation learning to achieve better control and generalization ability in generative modeling.
Prior to NLPR, I studied in two Chinese "Excellent" departments of statistics in Beijing, i.e., School of Statistics, Renmin University of China (M.Sc. 2020) and School of Mathematics and Statistics, Central University of Finance and Economics (B.Sc. 2017), where I majored in Applied Statistics.
Causal Representation Learning for Context-Aware Face Transfer
Human face synthesis involves transferring knowledge about the identity and identity-dependent shape of a human face to target face images where the context (\eg, facial expressions, head poses, and other background factors) may change dramatically. Human faces are non-rigid, so facial expression leads to deformation of face shape, and head pose also affects the face observed in 2D images. A key challenge in face transfer is to match the face with unobserved new contexts, adapting the identity-dependent face shape (IDFS) to different poses and expressions accordingly. In this work, we find a way to provide prior knowledge for generative models to reason about the appropriate appearance of a human face in response to various expressions and poses. We propose a novel context-aware face transfer model, called CarTrans, that incorporates causal effects of contextual factors into face representation, and thus is able to be aware of the uncertainty of new contexts. We estimate the effect of facial expression and head pose in terms of counterfactuals by designing a controlled intervention trial, thus avoiding the need for dense multi-view observations to cover the pose-expression space well. Moreover, we propose a kernel regression-based encoder that eliminates the identity specificity of the target face when encoding contextual information from the target image. The resulting method shows impressive performance, allowing fine-grained control over face shape and appearance under various contextual conditions.
Information Bottleneck Disentanglement for Identity Swapping
Improving the performance of face forgery detectors often requires more identity-swapped images of higher-quality. One core objective of identity swapping is to generate identity-discriminative faces that are distinct from the target while identical to the source. To this end, properly disentangling identity and identity-irrelevant information is critical and remains a challenging endeavor. In this work, we propose a novel information disentangling and swapping network, called InfoSwap, to extract the most expressive information for identity representation from a pre-trained face recognition model. The key insight of our method is to formulate the learning of disentangled representations as optimizing an information bottleneck trade-off, in terms of finding an optimal compression of the pre-trained latent features. Moreover, a novel identity contrastive loss is proposed for further disentanglement by requiring a proper distance between the generated identity and the target. While the most prior works have focused on using various loss functions to implicitly guide the learning of representations, we demonstrate that our model can provide explicit supervision for learning disentangled representations, achieving impressive performance in generating more identity-discriminative swapped faces.
M.Sc. in Applied Statistics, 2020
- School of Statistics
- Renmin University of China
B.Sc. in Applied Statistics, 2017
- School of Mathematics and Statistics
- Central University of Finance and Economics
I was born in Beijing, China. My Chinese name is Gege Gao (高格格). Hešeri (赫舍里) is my ancestral surname.