# Animated Art with AI – face reeanactment in action

In this article I will show how to use artificial intelligence to add motion to the images and photos.

## Face reenactment

To bring photos to life we can use the face reenactment algorithm designed to transfer the facial movements in the video to another image.

In this project I have used github implementation: https://github.com/AliaksandrSiarohin/first-order-model. Where the extensive description of the neural network architecture can be found in this paper. The solution contains of two parts: motion module and generation module.
The motion module at the first stage extracts the key points from the source and target image. In fact in the solution we assume that reference image which we can to the source and target image exists and at the first stage the transformations from reference image to source ($T_{S \leftarrow R} (p_k)$) and target ($T_{T \leftarrow R} (p_k)$) image is calculated respectively. Then the first order Taylor expansions $\frac{d}{dp}T_{S \leftarrow R} (p)| {p=p_k}$ and $\frac{d}{dp}T_{T \leftarrow R} (p)| {p=p_k}$ is used to calculate dense motion field.
The generation module use calculated dense motion field and source image to generate new image that will resemble target image.

The whole solution is packed into docker image thus we can simply reproduce the results using command:

docker run -it --rm --gpus all -v $(pwd)/torch_models:/root/.torch/models -v$(pwd)/checkpoints:/ai/checkpoints -v $(pwd)/test:/ai/test qooba/aifacereeanactment python3 ./prepare.py --source_image /ai/test/test.jpg --driving_video /ai/test/test.mp4 --output /ai/test/test_generated.mp4  NOTE: additional volumes (torch_models and checkpoints) are mount because during first run the trained neural networks are downloaded. To reproduce the results we need to provide two files motion video and source image. In above example I put them into test directory and mount it into docker container (-v$(pwd)/test:/ai/test) to use them into it.

Below you have all command line options:

usage: prepare.py [-h] [--config CONFIG] [--checkpoint CHECKPOINT]
[--source_image SOURCE_IMAGE]
[--driving_video DRIVING_VIDEO] [--crop_image]
[--crop_video] [--output OUTPUT] [--relative]
[--find_best_frame] [--best_frame BEST_FRAME] [--cpu]

first-order-model

optional arguments:
-h, --help            show this help message and exit
--config CONFIG       path to config
--checkpoint CHECKPOINT
path to checkpoint to restore
--source_image SOURCE_IMAGE
source image
--driving_video DRIVING_VIDEO
driving video
--crop_image, -ci     autocrop image
autocrop image paddings left, upper, right, lower
--crop_video, -cv     autocrop video
--output OUTPUT       output video
--relative            use relative or absolute keypoint coordinates
--no-relative         don't use relative or absolute keypoint coordinates
keypoints
--find_best_frame     Generate from the frame that is the most alligned with
source. (Only for faces, requires face_aligment lib)
--best_frame BEST_FRAME
Set frame to start from.
--cpu                 cpu mode.


# New Face with AI

In this article I will show how to use artificial intelligence to generate human faces.

To generate realistic human faces, we can use neural networks with GAN (Generative adversarial network) architecture.

The GaN network consists of two parts of the Generator whose task is to generate the image from random input and a discriminator that checks if the generated image is realistic.

During training, the networks compete with each other, the generator tries to generate better and better images
and thereby deceive the Discriminator. On the other hand, the Discriminator learns to distinguish between real and generated photos.

To train the discriminator, we use both real photos and those generated by the generator.

Finally, we can achieve the following results using DCGAN network.
As you can see some faces look realistic while some are distorted, additionally the network can only generate low resolution images.

We can achieve much better results using the StyleGaN (arxiv article) network, which, among other things, differs in that the next layers of the network are progressively added during training.

I generated the images using pretrained networks and the effect is really amazing.
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 1, "mathjax_display_tex": 0, "mathjax_asciimath": 1, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 0, "wp-katex-eq": 4, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0, "math_score": 0.2601357400417328, "perplexity": 3407.308375348614}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "plain"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "warc_path": "s3://commoncrawl/crawl-data/CC-MAIN-2021-31/segments/1627046149929.88/warc/CC-MAIN-20210723143921-20210723173921-00254.warc.gz"}