# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

# + [markdown] id="view-in-github" colab_type="text"
# <a href="https://colab.research.google.com/github/Hirnkastl/machine-learning-for-artistic-style/blob/master/machine_learning_artistic_style_transfer_en.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>

# + [markdown] id="KxYzWnjJr86x"
# # Image Style Transfer Using Neural Networks
#
# Visual recognition of objects is a task that humans excel at. In recent days, by using neural networks computers have also become very reliable at detecting objects in specific contexts.
#
# Particularly good performance has been reported for the class of [Convolutional Neural Networks](https://en.wikipedia.org/wiki/Convolutional_Neural_Network) (ConvNets). These networks consists of a sequence of layers. The outputs of the first layers correspond to pixel-level patterns and the outputs further to the end of the sequences describe larger-scale patterns and objects.
#
# A research team from Tübingen [has described](https://arxiv.org/abs/1508.06576) how the style of paintings can be described via the activations of the earlier layers of a ConvNet. The have also shown a procedure, by which an image can be created such that it corresponds to one image in terms of content (the __content image__) and in style to another image (the  __style image__).
#
# <img src="https://github.com/shellerbrand/machine-learning-for-artistic-style/raw/master/resources/vgg-convnet-content-and-style.jpg" alt="Convolutional Neural Network" width="700"/>
#
# __With this notebook you can experiment with Neural Style Transfer.__
#
# The example was originally created for use in the course [Lernen, wie Maschinen lernen](https://www.mzl.uni-muenchen.de/lehramtpro/programm/Lernen_-wie-Maschinen-lernen/index.html) at LMU Munich. We are using some modified code from the [Google Magenta Repo](https://github.com/tensorflow/magenta) that implements the style transfer scheme described by [Ghiasi et al.](https://arxiv.org/abs/1705.06830).
#

# + [markdown] id="52Um4cugyInh"
# ---
#
# ## Preparation
#
# These steps only have to be run once.

# + id="5tbnXXX2sNYx" cellView="form"
#@title Load required models and copy style-transfer code to this runtime
# !curl https://raw.githubusercontent.com/Hirnkastl/machine-learning-for-artistic-style/master/dist/stylization-lib.tar.gz -o image-stylization-lib.tar.gz

import sys
IN_COLAB = 'google.colab' in sys.modules
# Ensure the required version of scipy is installed in colab
if IN_COLAB:
  # !pip install numpy==1.16.3
  # !pip install tensorflow==1.13.2
  # !pip install keras-applications==1.0.7
  # !pip install keras-preprocessing==1.0.9
  # !pip install scipy==1.2.1

# The Model can be downloaded as per Google Magenta project
# !curl https://storage.googleapis.com/download.magenta.tensorflow.org/models/arbitrary_style_transfer.tar.gz -o image-stylization-checkpoint.tar.gz

# Unpack
# !tar -zxvf image-stylization-lib.tar.gz
# !tar -zxvf image-stylization-checkpoint.tar.gz
print('\nDone.')

# + id="FDLtKpkuM_cM"
from google.colab import drive
drive.mount('/content/drive')

# + id="8HPnqEClr86y" cellView="form"
#@title Load helper functions (required for loading and display of images)

# 
# Funktionen um Bilder zu laden vom Image-Upload Tool
#

print('Loading functions for downloading style and content images.')

import requests
import os
import shutil

DIR = './tmp/'
EXP_CONTENT = '2_inhalt'
EXP_STYLE = '2_stil'
OUTDIR = './output/'


if not os.path.isdir(DIR):
    print('Making the directory: {}'.format(DIR))
    os.makedirs(DIR)

# OK

def clear_output():
    if os.path.isdir(OUTDIR):
        shutil.rmtree(OUTDIR)
        os.mkdir(OUTDIR)

# OK

def clear_experiment(experiment_id):
    expdir = os.path.join(DIR, experiment_id)
    if os.path.isdir(expdir):
        shutil.rmtree(expdir)

def download_experiment_images(source, experiment_id):
  expdir = os.path.join(DIR, experiment_id)
  source_dir = os.path.join("/content/drive/MyDrive", source)

  shutil.copytree(source_dir, expdir)

print('Loading functions for showing images.')
# Configuration of matplotlib
import matplotlib.pyplot as plt
import matplotlib as mpl
mpl.rcParams['figure.figsize'] = (10,10)
mpl.rcParams['axes.grid'] = False

# python image library, numpy etc.
import numpy as np
from PIL import Image
import time
import functools

from tensorflow.python.keras.preprocessing import image as kp_image

# Helper function for loading images from a path
def load_img(path_to_img):
  max_dim = 1024
  img = Image.open(path_to_img)
  long = max(img.size)
  scale = max_dim/long
  img = img.resize((round(img.size[0]*scale), round(img.size[1]*scale)), 
                   Image.ANTIALIAS)
  
  img = kp_image.img_to_array(img)
  
  # We need to broadcast the image array such that it has a batch dimension 
  img = np.expand_dims(img, axis=0)
  return img

# Helper function for showing an image object (as per PIL)
def imshow(img, title=None):
  # Remove the batch dimension
  out = np.squeeze(img, axis=0)
  # Normalize for display 
  out = out.astype('uint8')
  plt.imshow(out)
  if title is not None:
    plt.title(title)
  plt.axis('off')
  plt.imshow(out)
  

print('\nDone.')

# + [markdown] id="9K7v2hAAr862"
# ---
#
#
# ## Upload Images
#
# In the next step the images are imported, for which the style-transfer will be applied. You can adjust the images that server as model for the style.
#

# + id="_kyOe2cXr864" cellView="form"
#@title Upload images from Drive

style_folder = "hirnkastl/style_transfer_project/v1/img_style" #@param {type:"string"}
content_folder = "hirnkastl/style_transfer_project/v1/img_content" #@param {type:"string"}

clear_experiment(EXP_CONTENT)
clear_experiment(EXP_STYLE)
clear_output()
download_experiment_images(content_folder, EXP_CONTENT)
download_experiment_images(style_folder, EXP_STYLE)

print("Done!")

# + [markdown] id="5wptVGUt7Zze"
# ## Transfer Image-Styles
#
# Now the image style can be transferred.
#
# In order to do that, for every __style image__ a vector S is computed by the network for style-analysis. This vector described the style of the image and can be passed to the style-transfer network, which transfers this style onto the __content image__.
#
# <img src="https://github.com/shellerbrand/machine-learning-for-artistic-style/raw/master/resources/networks-for-analysis-and-transfer.jpg" alt="Convolutional Neural Network" width="700"/>
#
# The networks used in this example have already been trained by Google in their Project Magenta. In simple terms, training means that the network for style-analysis has already seen tens of thousands of images. Because of the experience it has gained from looking at all these pictures it can also describe the style for images it has not seen before. In a similar fashion the style-transfer network has been trained with many images on how to transfer a specific style onto a content-image.
#
# Both of these pre-trained networks are now applied for all combinations of __style images__ and __content images__ that have been uploaded in the previous step.
#

# + id="iXlSva7Hr867" cellView="form"
#@title Start style-transfer
from lib.arbitrary_image_stylization_with_weights_func import arbitrary_stylization_with_weights, StyleTransferConfig
# from lib.arbitrary_image_stylization_with_weights import main as dostyle
import tensorflow as tf

c = StyleTransferConfig()
c.content_images_paths = os.path.join(DIR,'2_inhalt/*')
c.output_dir = OUTDIR
c.style_images_paths = os.path.join(DIR,'2_stil/*')
c.checkpoint = './arbitrary_style_transfer/model.ckpt'
c.image_size = 512
c.style_image_size = 256
print(c)

with tf.device('/gpu:0'):
  arbitrary_stylization_with_weights(c)

print("\nDone.")

# + [markdown] id="PpHk--79r86_"
# ## View images
#
# If the style-transfer has finished without errors, then you can run the following steps to view the images.

# + id="dr12ukjYr87D" cellView="form"
#@title View content images
# Get all content images
cols = 3
basewidth=20
cfiles = os.listdir(path=os.path.join(DIR, EXP_CONTENT))
#print("{},{}".format(basewidth,len(files)/cols*basewidth))
plt.figure(num=1, figsize=(basewidth,len(cfiles)/(cols*cols)*basewidth))
pind = 1
for fi in cfiles:
    path = os.path.join(DIR, EXP_CONTENT, fi)
    # print(path)
    im = load_img(path_to_img=path).astype('uint8')    
    plt.subplot(np.ceil(len(cfiles)/cols),cols,pind)
    imshow(im,title=fi)
    pind = pind + 1
    
print("The images that styles will be applied to:")

# + id="B327ydjkr87G" cellView="form"
#@title View style images
# Get all style images
basewidth=20
sfiles = os.listdir(path=os.path.join(DIR, EXP_STYLE))
cols = len(sfiles)+1
#print("{},{}".format(basewidth,len(files)/cols*basewidth))
plt.figure(num=1, figsize=(basewidth,len(sfiles)/(cols*cols)*basewidth))
plt.axis('off')

pind = 1
sfiles.sort()
for fi in sfiles:
    path = os.path.join(DIR, EXP_STYLE, fi)
    # print(path)
    im = load_img(path_to_img=path).astype('uint8')    
    plt.subplot(np.ceil(len(sfiles)/cols),cols,pind)
    imshow(im,title=fi)
    pind = pind + 1
    
print("The style images:")

# + id="iHyKeDh2r87Q" cellView="form"
#@title View images with style-transfer applied
# Stylized Images
from re import match

cols = 3
basewidth=20
files = os.listdir(path=os.path.join(OUTDIR))
files = [x for x in files if match('^zzResult.*',x)]
#print("{},{}".format(basewidth,len(files)/cols*basewidth))
plt.figure(num=1, figsize=(basewidth,len(cfiles)/(len(sfiles)+1)*basewidth))
pind = 1
files.sort()
for fi in files:
    path = os.path.join(OUTDIR, fi)
    # print(path)
    im = load_img(path_to_img=path).astype('uint8')    
    plt.subplot(len(cfiles)+1,len(sfiles)+1,pind)
    imshow(im,title=fi)
    pind = pind + 1
    
print("The images that result from style-transfer:")

# + id="fyOsDJCxC2Op" cellView="form"
#@title Download all images in a .zip file

if IN_COLAB:
    # !zip output.zip output/*
    from google.colab import files
    files.download("output.zip")
else:
    print("Not in colab - skipping.")


# + [markdown] id="vB8asjdgwOxK"
# ---
#
# ## References
#
# * <NAME> al; A Neural Algorithm of Artistic Style, Sep 2015, [arxiv](https://arxiv.org/abs/1508.06576)
# * Google Magenta: Fast Style Transfer for Arbitrary Styles, [Github](https://github.com/tensorflow/magenta/blob/2c3ae9b0dd64b06295e48e2ee5654e3d207035fc/magenta/models/arbitrary_image_stylization/README.md)  
# * <NAME> al.; 
# Exploring the structure of a real-time, arbitrary neural artistic stylization network
# <NAME>, Aug 2017, [arxiv](https://arxiv.org/abs/1705.06830)
#
# The Source-Code for this notebook and the tool for uploading images is or will be published here:
# https://github.com/shellerbrand/machine-learning-for-artistic-style
#

StarcoderdataJupyterScriptsDedupFiltered