This tutorial will cover the theory and practice of creating deepfakes: videos in which faces have been swapped using Machine Learning and Deep Neural Networks. If you are interested in learning more about this novel technique, this is the course for you.
After a theoretical introduction, this course will focus on how to make the most out of the popular applications FakeApp and faceswap; most of the deepfakes you will find online (such as the ones featuring Nicolas Cage) have been created using them.
You can read all the posts in this series here:
- Part 1. An Introduction to DeepFakes and Face-Swap Technology
- Part 2. The Ethics of Deepfakes
- Part 3. How To Install FakeApp
- Part 4. A Practical Tutorial for FakeApp
- Part 5. An Introduction to Neural Networks and Autoencoders
- Part 6. Understanding the Technology Behind DeepFakes
- Part 7. How To Create The Perfect DeepFakes
If you are interested in reading more about AI Art (Stable Diffusion, Midjourney, etc) you can check this article instead: The Rise of AI Art.
Introduction
Face detection has been a major research subject in the early 2000s. Almost twenty years later, this problem is basically solved and face detection is available as a library in most programming languages. Even face-swap technology is nothing new, and has been around for a few years.
In his article from 2016, Face Swap using OpenCV, author Satya Mallick showed how to swap faces programmatically, warping and colour correcting Ted Cruz’s face to fit Donald Trump (below).
When applied correctly, this technique is uncannily good at swapping faces. But it has a major disadvantage: it only works on pre-existing pictures. It cannot, for instance, morph Donald Trump’s face to match the expression of Ted Cruz.
That has changed in late 2017, when a new approach to face-swap has appeared on Reddit. Such a breakthrough relies on neural networks, computational models that are loosely inspired by the way real brains process information. This novel technique allows generating so-called deepfakes, which actually morph a person’s face to mimic someone else’s features, although preserving the original facial expression.
When used properly, this technique allows the creation of photorealistic videos at an incredibly low cost. The finale of Rogue One, for instance, featured a digital version of Princess Leia; a very expensive scene which required the expertise of many people. Below, you can see a comparison between the original scene and another one recreated using Deep Learning.
Creating Deepfakes
At the moment there are two main applications used to create deepfakes: FakeApp and faceswap. Regardless of which one you will use, the process is mostly the same, and requires three steps: extraction, training and creation.
Extraction
The deep- in deepfakes comes from the fact that this face-swap technology uses Deep Learning. If you are familiar with the concept, you should know that deep learning often requires large amounts of data. Without hundreds (if not thousands!) of face pictures, you will not be able to create a deepfake video.
A way to get around this is to collect a number of video clips which feature the people you want to face-swap. The extraction process refers to the process of extracting all frames from these video clips, identifying the faces and aligning them.
The alignment is critical, since the neural network that performs the face swap requires all faces to have the same size (usually 256×256 pixels) and features aligned. Detecting and aligning faces is a problem that is considered mostly solved, and is done by most applications very efficiently.
Training
Training is a technical term borrowed from Machine Learning. In this case, it refers to the process which allows a neural network to convert a face into another. Although it takes several hours, the training phase needs to be done only once. Once completed, it can convert a face from person A into person B.
This is the most obscure part of the entire process, and I have dedicated two posts to explain how it works from a technical point of view: An Introduction to Neural Networks and Autoencoders and Understanding the Technology Behind DeepFakes). If you really want to create photorealistic deepfakes, a basic understanding of the process that generates them is necessary.
Creation
Once the training is complete, it is finally time to create a deepfake. Starting from a video, all frames are extracted and all faces are aligned. Then, each one is converted using the trained neural network. The final step is to merge the converted face back into the original frame. While this sounds like an easy task, it is actually where most face-swap applications go wrong.
The creation process is the only one which does not use any Machine Learning. The algorithm to stitch a face back onto an image is hard-coded, and lacks the flexibility to detect mistakes.
Also, each frame is processed independently; there is no temporal correlation between them, meaning that the final video might have some flickering. This is the part where more research is needed. If you are using faceswap instead of FakeApp, have a look at df which tries to improve the creation process.
Conclusion
Deep Learning has made photorealistic face-swap not just possible, but also accessible. This technique is still in its infancy and many more improvements are expected to happen in the next few years.
In the meantime, places like the FakeApp forum or the fakeapp GitHub page are where most of the technical discussion around deepfakes is currently taking place. The community around deepfakes is constantly exploring new approaches, and developers are often very willing to share their creations. This is the case of user ZeroCool22 which created a deepfake video of Jimmy Fallon interviewing himself.
Another interesting reading on the subject is Exploring DeepFakes.
It cannot be denied that deepfakes have finally shown the world a practical application of Deep Learning. However, this very technique has often been used without the explicit consent of the people involved. While this is unlikely to be an issue with videos such as the ones shown in this article, the same cannot be said when it is used to create pornographic content. This is why, before showing how to create deepfakes, the next lecture in this online course will focus entirely on the legal and ethical issues of deepfakes.
You can read all the posts in this series here:
- Part 1. An Introduction to DeepFakes and Face-Swap Technology
- Part 2. The Ethics of Deepfakes
- Part 3. How To Install FakeApp
- Part 4. A Practical Tutorial for FakeApp
- Part 5. An Introduction to Neural Networks and Autoencoders
- Part 6. Understanding the Technology Behind DeepFakes
- Part 7. How To Create The Perfect DeepFakes
A special thanks goes to Christos Sfetsios and David King, who gave me access to the machine I have used to create the deepfakes used in this tutorial.
👾 This course was made possible thanks to the support of patrons on Patreon
🥇 $25+ |
🥈 $10+ |
🥉 $8+ |
🥉 $5+ |
💵 $1+ |
Dry Cactus Nicolas Gascon Aakash Sastry Binary Soul Antonio Cervantes Ariel Ahrens Sean Magnopus Skippy Takeshi |
Mike King Hữu Lộc John Faulkenbury Jashan Chittesh Piotr Podziemski Dexter André Osiander Perren Spence-Pearse Roberta Mota Arcadely Keith Miller Adam Boyne Craig Bowler Bryan Duggan Shawn Asmussen Dries De Smet Druelbozo TrojanFighter Shohei Takei 1011101 Glenn Jones Jean Georges Adam Frisby David Peterson Daniel Farrow Minseck Cho Zac Dixon Nathan Grotticelli Jessie Thomson Theo Lagendijk Ben Beagley Michael Cook Anthony Duquette Brad Hammond Danny Huynh zhouhua SaharaX TraceYang Brandon Tay Joakim Westgren Rohit Gupta David Erosa Jonathan Hopkins Michael Romaszewicz Omar Espinosa Kevin Comerford Shimon Vainer Fernando Monroy benfei Lawrence Hammad Bashir Al-Rashid Jamalul Kieran Belkus Yin Liu A. Moore fangeles Pixeye Juan Trelles Trabucco JoneyZ Kyle Phillips SUPER315 Abel Magnus Paulson Erga HyperDigital Joan Llobera Alb Pérez-Bermejo FruityFusion Mike Wuetherick Luke Kaalim Thomas Ingram unizen rizal ardianto William Wilson Stefano Gandolfo Stephen Lukasz Pasek fredqin Arielle Kim Jack Riddle Yangtze River 培基 朱 Josquin Z OccupiedVR shangmingyou Jordan Jackson Rick Palmer James Neal Yaser Saqib Apoc Shane Nilsson |
Jarrod Phillips Julian Heinken François Chéné |
Moritz Voss Jonathan Wright Jason Adams Jordan Carroll Juwenw Sun Aaron Eastburn Tomáš Koza Chris Sorrell rustinlee obscuresteel TxCx Ciro Elan Feingold Eric Cloutier Wilco Weggeman Mario Gutierrez FELIX Jaewon Jung Anthony Rosenbaum Justin Loudermilk Ryan Schake Mike Corsaro Sebastian KÄ™dra Tom Márton Helényi CatchCo Hans Palm mars Matei Giurgiu Zephyr Zhang Jules Stevenson Edouard Philippe Dave Cowling Andy De Meyer Phil Gosch Dave Chenell JeanD Kit Matthews Klemen BrajkoviÄ Liam Brady Andrew Johnson Gil Damoiseaux Jungchul Ha J Jeremy Zeler Elijah Cauley Jonadab Oomon Jake OBrien tmabs Gordon Chapman Kevin Lee Jr. Colin Schultz Brad Weiers Gabor Tornyos Marcel Medak Anders Ørum Pedersen Dirk Van Welden Thomas Key Yotam Harris John Bullock Justin Lindsey Nick Maurer Ray Markus Arning Bruno Di Pentima Tuboy Thers Daniel Saulnier Jacob Herold Ashley James Lukas Lundberg Jeremy Griffith Tom Campbell Luca Prasso Luis Jose Quintana Manjit Bedi Fernando Urquijo Sánchez starportx@sohu.co SpaceToad Jose Miguel Casas Pagan Edward del Villar Garrett Stevens Atle Mæland Leonardo Marques Maddox Max Gopeekrishnan Sobhana Wariar Devyn Cyphers Martin Hedlund Yaser S Fox Buchele Francis Ge mscottmcbee Gamoga Notyag Rolly Ted Milker Michal Szczepaniak Ian Thompson Kevin Doyon Ian Sterling Miikka Harjuntausta kc austin Martin Leissler Alex Chih Chung Lin Diego Sarmentero Andy Gittins Neil Blakey-Milner Victoria Mitchell Louis Hong mirko jugurdzija Derek Reynolds Mitu Khandaker nathan witkowski Daniel Plemmons Nicoll Hunt Kerry Joe Jonny Ree Kevin Webb Tim |
Vladislav Chetrusca Shaders Laboratory Peter Cardwell-Gardner Robin Baumgarten Andreas Wilcox Terkel Gjervig Nielsen Stella Cannefax Iruken Martin Eigel Ryan Chris Pierce McBride Shadow Darkwell Alo Mis Jim clownbaby Christopher Pereira Steve Demi Iain Wonky Kim Karl Goodloe Daniel Cassidy Alessandro Salani Doug Snook Marco Elizondo Edd Toomey Hotgates Brian Nicolucci Bryan Cinman Jan Ambroziak Alice Ruppert Josh Leong parag ponkshe Martin Pražák Clément Clement Gavin Rich Kevin Gregory Agwaze Håkon Nessjøen James Id Warren Moore devMidgard jp Jason Burton Ivan Herrera Maciel Noaksey Daniel Sanchez Jon Kimbel André Gröschel friuns Manish Gupta David Proffer Pavel Marek Majcher Arthur Cousseau Orlando Batista CptBonex Jakub Barszczewski Arlin Matheus Dom Camus Aubrey Hesselgren Kemp Roger Fretwell Cary Miller Matthew Senne Chris Butler C Paterson Nic DanielOthers: NachoB b Stewart Chris Yaz Khabiri Abisaid Fernandez de Lara valryon Thomas Schmall Patrick Peluse Goran Lalic |
💖 Support this blog
This websites exists thanks to the contribution of patrons on Patreon. If you think these posts have either helped or inspired you, please consider supporting this blog.
📧 Stay updated
You will be notified when a new tutorial is relesed!
📝 Licensing
You are free to use, adapt and build upon this tutorial for your own projects (even commercially) as long as you credit me.
You are not allowed to redistribute the content of this tutorial on other platforms. Especially the parts that are only available on Patreon.
If the knowledge you have gained had a significant impact on your project, a mention in the credit would be very appreciated. ❤️🧔🏻
Hello! Is there any way to contact you ? (i.e. email address)
Anna
Sure! You can find my contat details in the “About” section of the website!
Please tell me how to remove the message show on screen after installing the software.
When I installed CUDA, it prompted that visio studio must be installed first.Is that necessary?