Recently, researchers from the Institute of Automation of the Chinese Academy of Sciences published a paper on how to solve the problem of facial replacement in all postures, and proposed a real-time 3D solution.

Facial alignment has become an important topic in the field of computer vision, which adapts facial models to images and extracts the semantics of facial pixels. Previously, most of the algorithms were designed for small and medium-sized faces (the deflection angle is less than 45 degrees), lacking the ability to align faces in large gestures up to 90 degrees. This paper is a method proposed for face replacement in all pose ranges.

The improved version proposed by the doctoral student adds some extra work, including real-time training, training strategies, and so on, rather than just re-realizing "3D real-time face change". More detailed content will be published in related blogs in the future, including some important technical details. So far, this improved version has released the pytorch model for the first phase of pre-training, including the MobileNet-V1 structure, training data sets and code. On GeForce GTX TITAN X, the reasoning time of each image is about 0.27 milliseconds (with 128 input batches).

This improved version helps Pytorch improve the methods mentioned in the paper "Face Replacement in All Postures: A 3D Solution". One of the authors of this paper is Xiangyu Zhu from the Institute of Automation, Chinese Academy of Sciences. According to the information on his personal homepage, he and Dr. Cleadusk's tutors were both Professor Li Ziqing. They can be said to be brothers of the same professor.

Recently, cleardusk, an author of Github (currently a doctoral student at the Institute of Automation, Chinese Academy of Sciences), implemented an improved version of PyTorch called "3D Real-time Face Change". The reasoning time of each graph is only 0.27 milliseconds!

So, what applications can the improved version achieve?

First, it can achieve facial alignment.

Next is facial remodeling, to achieve "face change"!