TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing

Yanbo Xu*

Yueqin Yin*

Liming Jiang

Qianyi Wu

Chengyao Zheng

Chen Change Loy

Bo Dai

Wayne Wu⁺

Shanghai AI Laboratory

Nanyang Technological University

SenseTime Research

HKUST

Monash University

Code [GitHub]

CVPR 2022 [Paper]

Abstract

Recent advances like StyleGAN have promoted the growth of controllable facial editing. To address its core challenge of attribute decoupling in a single latent space, attempts have been made to adopt dual-space GAN for better disentanglement of style and content representations. Nonetheless, these methods are still incompetent to obtain plausible editing results with high controllability, especially for complicated attributes. In this study, we highlight the importance of interaction in a dual-space GAN for more controllable editing. We propose TransEditor, a novel Transformer-based framework to enhance such interaction. Besides, we develop a new dual-space editing and inversion strategy to provide additional editing flexibility. Extensive experiments demonstrate the superiority of the proposed framework in image quality and editing capability, suggesting the effectiveness of TransEditor for highly controllable facial editing.

Paper

Yanbo Xu*, Yueqin Yin*, Liming Jiang, Qianyi Wu, Chengyao Zheng, Chen Change Loy, Bo Dai, Wayne Wu.
TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing .
In CVPR, 2022. (Paper)

[Bibtex]

Method

Two latent spaces Z and P are used for generation. We correlate them via a cross-attention-based interaction module to facilitate editing.

Interpolation of two latent spaces. They are disentangled with different semantic meanings.

Interpolating Z space

Interpolating P space

Editing Results

Smile editing on Z space

Gender editing on Z and P space

Head pose editing on P space

Age editing on Z and P space

Comparison

Our method shows better editing ability compared with other SOTA methods.

Gender Editing Comparison

Pose Editing Comparison

Acknowledgements

This study is partly supported under the RIE2020 Industry Alignment Fund Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contribution from the industry partner(s).