All about Deepfakes
Catalogue
1 Deepfakes survey
- 2018 arxiv DeepFakes: a New Threat to Face Recognition? Assessment and Detection
- 2019 arXiv Deep Learning for Deepfakes Creation and Detection
- 2020 arXiv DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection
- 2020 arXiv Media Forensics and DeepFakes: an overview
- 2021 ACM Computing Surveys The Creation and Detection of Deepfakes: A Survey
- 荷兰初创公司Deeptrace发布两个年度（2018年和2019年）Deepfakes发展状况调研报告
2 Deepfakes Datasets
| Datasets | Year | Ratio tampered:original |
Total videos | Source | Participants Consent | Tools |
|---|---|---|---|---|---|---|
| UADFV | 2018 | 1 : 1.00 | 98 | YouTube | N | FakeAPP |
| FaceForensics | 2018 | 1 : 1.00 | 2008 | YouTube | N | Face2Face |
| Deepfake-TIMIT | 2019 | 1 : 1.00 | 620 | Vid-TIMIT | N | faceswap-GAN |
| FaceForensics++ | 2019 | 1 : 0.25 | 5000 | YouTube | N | faceswap DeepFake Face2Face NeuralTextures |
| DeepFakeDetection (part of FaceForensics++) |
2019 | 1 : 0.12 | 3363 | Actors | Y | |
| Celeb-DF | 2019 | 1 : 0.51 | 1203 | YouTube | N | a refined version of the DeepFake |
| DFDC Preview Dataset | 2019 | 1 : 0.28 | 5214 | Actors | Y | Unkonwn |
| KoDF | 2021 | |||||
| Face Forensics in the Wild | 2021 CVPR |
3 Deepfake Creation
3.1 Face-swapping methods
-
Open-source code
- Deepfakes [Star27.6k]
- Faceswap-GAN [Star2.5k]
- FaceSwap [Star300+]
- DeepFaceLab [Star11.1k]
- DFaker [Star300+]
- DeepFake-tf [Star<10]
-
Paper
| Year | Publications | Authors |
|---|---|---|
| 2017 ICCV | Fast Face-Swap Using Convolutional Neural Networks | Iryna Korshunova, Wenzhe Shi, Joni Dambre, Lucas Theis |
| 2018 IEEE FG | On Face Segmentation, Face Swapping, and Face Perception | Yuval Nirkin; Iacopo Masi; Anh Tran Tuan; Tal Hassner; Gerard Medioni |
| 2019 ICCV | FSGAN: Subject Agnostic Face Swapping and Reenactment | Yuval Nirkin, Yosi Keller, Tal Hassner |
| 2020 CVPR(Oral) | FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping | Lingzhi Li, Jianmin Bao, Hao Yang, Dong Chen, Fang Wen |
3.2 Face reenactment methods
4 Deepfake Detection
4.1 Fake image detection
| Year | Publications | Authors |
|---|---|---|
| 2017 ICSIP | Automated face swapping and its detection | Ying Zhang; Lilei Zheng; Vrizlynn L. L. Thing |
| 2018 IHMS Workshop | Fake Faces Identification via Convolutional Neural Network | Huaxiao Mo, Bolin Chen, Weiqi Luo |
| 2018 MIPR | Detection of GAN-Generated Fake Images over Social Networks | Francesco Marra; Diego Gragnaniello; Davide Cozzolino; Luisa Verdoliva |
| 2018 ISCCC | Learning to Detect Fake Face Images in the Wild | Chih-Chung Hsu; Chia-Yen Lee; Yi-Xiu Zhuang |
| 2019 CCBR | On the Generalization of GAN Image Forensics | |
| 2019 aXiv | Limits of Deepfake Detection: A Robust Estimation Viewpoint | Sakshi Agarwal, Lav R. Varshney |
| 2019 MDPI AG | Deep Fake Image Detection based on Pairwise Learning | Chih-Chung Hsu, Yi-Xiu Zhuang and Chia-Yen Lee |
| 2019 IEEE BTAS | Multi-task learning for detecting and segmenting manipulated facial images and videos | Huy H. Nguyen; Fuming Fang; Junichi Yamagishi; Isao Echizen |
| 2021 CVPR | Multi-attentional Deepfake Detection | Hanqing Zhao, Wenbo Zhou, Dongdong Chen, Tianyi Wei, Weiming Zhang, Nenghai Yu |
| 2021 CVPR | Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain | Honggu Liu, Xiaodan Li, Wenbo Zhou, Yuefeng Chen, Yuan He, Hui Xue, Weiming Zhang, Nenghai Yu |
| 2021 CVPR | Improving the Efficiency and Robustness of Deepfakes Detection through Precise Geometric Features | Zekun Sun, Yujie Han, Zeyu Hua, Na Ruan, Weijia Jia |
| 2021 CVPR | Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection | Jiaming Li, Hongtao Xie, Jiahong Li, Zhongyuan Wang, Yongdong Zhang |
| 2021 CVPR | MagDR: Mask-guided Detection and Reconstruction for Defending Deepfakes | Zhikai Chen, Lingxi Xie, Shanmin Pang, Yong He, Bo Zhang |
4.2 Fake video detection
4.2.1 Temporal features across frames based ones
| Year | Publications | Classifiers/ Techniques |
|---|---|---|
| 2018 AVSS | Intra-frame and temporal inconsistencies | CNN and LSTM |
| 2018 WIFS | In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking | LRCN |
| 2019 CVPR Workshop | Recurrent Convolutional Strategies for Face Manipulation Detection in Videos | RCN |
4.2.2 Visual artifacts within video frame based methods
| Year | Publications | Classifiers/ Techniques |
|---|---|---|
| 2017 CVPR Workshop | Two-Stream Neural Networks for Tampered Face Detection | Two-Stream |
| 2018 WIFS | MesoNet: a Compact Facial Video Forgery Detection Network | CNN |
| 2018 IMVIP | Detection of Deepfake Video Manipulation | PRNU |
| 2019 CVPR Workshop | Exposing DeepFake Videos By Detecting Face Warping Artifacts | VGG16 ResNet50, 101 or 152 |
| 2019 ICASSP | Capsule-forensics: Using Capsule Networks to Detect Forged Images and Videos | Capsule networks |
| 2019 ICASSP | Exposing Deep Fakes Using Inconsistent Head Poses | SVM |
| 2019 WACVW | Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations | Logistic regression and neural network |
| 2019 IEEE Access | Combating Deepfake Videos Using Blockchain and Smart Contracts | Blockchain |
| 2019 CVPR Workshops | Protecting World Leaders Against Deep Fakes | svm, OpenFace2 |