Awesome-Deepfakes-Materials

A curated, but probably biased and incomplete, list of awesome Deepfakes resources.

If you want to contribute to this list, feel free to pull a request. Also you can contact Mengnan Du from the Data Lab at Texas A&M University through email: [email protected], or Twitter @DuMNCH.

What is Deepfakes?

In December 2017, a user with name “DeepFakes” posted realistic looking videos of famous celebrities on Reddit. These fake videos are generated using deep learning, by swapping faces of original adult movies with celebrities’ faces. Since then, the topic of DeepFakes goes viral on internet.

Here, we denote DeepFakes as any fake contents generated by deep learning techniques. DeepFakes comes in different forms, perhaps the most typical ones are: 1) Videos and images, 2) Texts, and 3) Voices. Different deep learning techniques are used to generate DeepFakes. For instance, videos and images are usually created by Generative adversarial networks (GAN), while texts are mostly generated by deep language models based on Transformers.

Table of Contents

• Deepfake Videos and Images
  • Generation Papers
  • Generation Codes
  • Detection Papers
  • Detection Codes
  • Datasets and Challenges
  • General Online Articles of Deepfake Videos
• Deepfake Texts
  • Papers Mainly on Detection
  • Codes Mainly on Detection
  • General Online Articles of Neural Fake News
• Deepfake Voices
  • Papers Mainly on Generation
  • Codes Mainly on Generation
  • General Online Articles of Deepfake Voices

Deepfake Videos and Images

Generation Papers

• HeadOn: Real-time Reenactment of Human Portrait Videos
• Face2Face: Real-time Face Capture and Reenactment of RGB Videos
• Synthesizing Obama: Learning Lip Sync from Audio

Generation Codes

• FaceSwap
• Faceswap-GAN
• faceswap-pytorch
• faceswap-pytorch2
• DeepFaceLab
• face2face
• Synthesizing Obama

Detection Papers

• Review and General Papers

  • Deep Learning for Deepfakes Creation and Detection
  • DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection
  • Media Forensics and DeepFakes: an overview
  • Will Deepfakes Do Deep Damage?

• Relying on Artifacts

  • Exposing DeepFake Videos By Detecting Face Warping Artifacts
  • Exposing Deep Fakes Using Inconsistent Head Poses
  • In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking
  • Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations
  • Protecting World Leaders Against Deep Fakes

• Binary Classifiers

  • Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos
  • FaceForensics++: Learning to Detect Manipulated Facial Images
  • Deepfake Video Detection Using Recurrent Neural Networks

• Generalization performance

  • Towards Generalizable Forgery Detection with Locality-aware AutoEncoder
  • ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection
  • Face X-ray for More General Face Forgery Detection
  • On the generalization of GAN image forensics
  • Fake Face Detection Methods: Can They Be Generalized?

Detection Codes

• Deepfake detection for the masses
• Exposing DeepFake Videos By Detecting Face Warping Artifacts
• Vulnerability assessment and detection of Deepfake videos

Datasets and Challenges

• FaceForensics++
• Celeb-DF
• Deepfakes detection challenge by Facebook, Microsoft, etc.
• DeepfakeTIMIT
• Kaggle Deepfake Detection Challenge

General Online Articles of Deepfake Videos

• Deepfake: The Good, The Bad and the Ugly - Medium
• DeepFakes: AI-powered deception machines
• Exploring DeepFakes - KDnuggets
• Family fun with deepfakes. Or how I got my wife onto the Tonight Show - Towards Data Science
• Deepfake propaganda is not a real problem - The Verge

Deepfake Texts

Papers Mainly on Detection

• GLTR: Statistical Detection and Visualization of Generated Text
• Defending Against Neural Fake News
• CTRL: A Conditional Transformer Language Model for Controllable Generation

Codes Mainly on Detection

• GLTR: Giant Language Model Test Room
• Grover
• CTRL - A Conditional Transformer Language Model for Controllable Generation

General Online Articles of Neural Fake News

• OpenAI: Please Open Source Your Language Model
• OpenAI Shouldn’t Release Their Full Language Model
• Better fact-checking for fake news

Deepfake Voices

Papers Mainly on Generation

• Tacotron: Towards End-to-End Speech Synthesis
• Neural Voice Cloning with a Few Samples
• Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning
• Deep Voice 3: Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attention

Codes Mainly on Generation

• deep-voice-conversion
• Tacotron
• mimic2
• Neural Voice Cloning with Few Samples
• Deepvoice3_pytorch
• Real-Time Voice Cloning
• DeepFake Audio Detection

General Online Articles of Deepfake Voices

• You can now speak using someone else’s voice with Deep Learning
• A Voice Deepfake Was Used To Scam A CEO Out Of $243,000