StyleRemix: An Interpretable Representation for Neural Image Style Transfer
Hongmin Xu Qiang Li Wenbo Zhang Wen Zheng




















StyleRemix: An Interpretable Representation for Neural Image Style Transfer 
    Hongmin Xu^*, 
Qiang Li^*, 
Wenbo Zhang, 
Wen Zheng

Y-tech, Kwai



















    Received 18 December 2018 / Accepted 26 February 2019
========================================================================================






Multi-Style Transfer (MST) intents to capture the high-level visual vocabulary of different styles and expresses these vocabularies in a joint model to transfer each specific style. Recently, Style Embedding Learning (SEL) based methods represent each style with an explicit set of parameters to perform MST task. 
However, most existing SEL methods either learn explicit style representation with numerous independent parameters or learn a relatively black-box style representation, which makes them difficult to control the stylized results. In this paper, we outline a novel MST model, StyleRemix, to compactly and explicitly integrate multiple styles into one network. 
 By decomposing diverse styles into the same basis, StyleRemix represents a specific style in a continuous vector space with 1-dimensional coefficients.
With the interpretable style representation, StyleRemix not only enables the style visualization task but also allows several ways of remixing styles in the smooth style embedding space. Extensive experiments demonstrate the effectiveness of StyleRemix on various MST tasks compared to state-of-the-art SEL approaches.
[*The first two authors contributed equally to this work.]









§ INTRODUCTION





Style transfer aims to convincingly confer artist's painting style, such as the shapes, lines, colors, tones, and textures created by the unique techniques of artist, to arbitrary images. With style transfer, people can produce Van Gogh's The Starry Night style in their photographs. In the past, it may take much time of a well-trained artist to create such paintings. The critical issue of style transfer and its previous closely related work texture synthesis is how to model visual texture of target style image and generate a stylized image, which has similar visual fashion with the target style image while retaining the shape of input content image. Traditional methods <cit.> usually focused on modeling texture with low-level features to handle this problem.






The seminal work of Gatys <cit.>, Neural Style (NS) transfer, first modeled texture and style by the summary statistics in the domain of Convolutional Neural Network (CNN). Specifically, style is defined by the covariance matrix of feature response computed by multi-layers of VGG-19 <cit.>. 
Based on online powerful iterative optimization, NS outputs a stylized image, whose style is close to the target image while retaining the shape of input image. Although NS produces better visually satisfied image than traditional methods, NS is time-consuming since its expensive computation cost. 
Therefore, Fast Neural Style (FNS) <cit.> transfer approaches trained offline feed-forward style-specific model to produce stylized images, which is hundreds of times faster than the NS method. 
However, FNS loses the flexibility of NS: different generative networks have to be trained for each specific style image. 

In order to improve the flexibility of FNS, Multi-Style Transfer (MST) methods aim to incorporate multiple styles into one single model. There are two paths to achieve this goal: 1) parametric Arbitrary Style Transfer (AST) approaches <cit.> and 2) non-parametric Style Embedding Learning (SEL) based style transfer approaches, such as <cit.>. 


AST methods <cit.> often learn a parametric network to describe a large set of styles and manually design <cit.> or learn a network as a universal style transfer function <cit.>. Therefore, the scalability of AST is larger than SEL to some extend. However, AST suffers from two shortcomings: 1) They always produce compromised visual quality stylized images, as some new styles may be poorly represented by the parametric style network or the designed/learned style transfer function may not be suitable for the new styles. 2) The style representation learned by a black-box network is relatively hard to explain and difficult to control the stylization.









In parallel, SEL methods <cit.> focused on learning a small set of parameters to explicitly define each style, which enable them to generate better visually impressive stylized results than AST. 
Dumoulin <cit.> utilized the parameters of instance normalization layers to represent one specific style. Li <cit.> defined a noise map to indicate one particular style. Yanai <cit.> used a simple one-hot map to denote different styles. However, those style representations are relatively hard to interpret.
As for Chen <cit.>, they proposed an explicit style representation method, StyleBank, which is more explainable 
 whereas StyleBank need a layer of convolution filter banks to represent each style.  Therefore, for all the state of the art SEL methods, either they <cit.> are hardly interpretable or  <cit.> inefficient to represent styles.


In this paper, we propose a SEL method StyleRemix that brings interpretability and efficiency to the style representation in MST task. Specifically, StyleRemix separates content (of input image) and style (of target image) representation. 
Content is transformed into multi-layers activations of an autoencoder. Style is described as weighted convolution filters, which is convolved with the intermediate feature embedding (content) to get the final stylized image. Style representation consists of two parts: style basis and style weights.
	
	
	Style basis correspond to interpretability and their channels describe the fundamental style elements. Style weights correspond to efficiency and a specific style is modeled as a 1-dimensional continuous vector, which is more efficient than the block-wise representation method in <cit.>. 










Our method learns the style embedding space for a set of styles. Thus, MST task can be performed by providing the inference network with the style weights of a specific style. 
With the learned style representations,

StyleRemix makes it possible for the task of style visualization.
Besides, thanks to the interpretability and efficiency of the style representation, our method offers several ways to manipulate styles, such as convex combination, noise perturbation, and even Collection Style Transfer (CST), as shown in Fig. <ref>. Furthermore, StyleRemix can handle a certain amount of diverse styles with consistent visual quality according to a scalability analysis on the number of styles.







































































	
	
	
	
	
	

	
	
	
	

	
	
	
	
	
	
	
	
	






§ RELATED WORKS






  
Paired Image-to-Image Translation. Image-to-Image Translation can be dated back to image analogy <cit.>. The goal of image analogy is modeling the transfer for a single paired input and output image. A recent work <cit.> investigated the image analogy problem in deep feature level. 

On the other hand,  the goal of image-to-image translation is learning the transfer function between the input image set and output image set.
For paired image-to-image translation, recent approaches used a paired dataset of input and output images to learn a parametric transfer function by using CNNs <cit.>. Isola <cit.> used conditional generative adversarial network (GAN) <cit.> to learn the transfer function between two sets.  With the similar idea, Sangkloy <cit.> generated images conditioned on both sketched boundaries and
sparse color strokes. 

Karacan <cit.> synthesized images from the semantic layout.

However, how to get the paired dataset is a tough problem in practice. 





  
Unpaired Image-to-Image Translation. 

To handle the issue of obtaining paired dataset, unpaired image-to-image translation <cit.> methods have been proposed. The goal is tackling the unpaired data setting and learn the transfer function between two domains. To achieve this goal, Liu <cit.> shared the weights of the generators of different GANs to learn the joint representation in cross domains. Later, Liu <cit.> extended the framework of <cit.> by combining the variational autoencoders <cit.> with generative adversarial network.  
Another line of works, Zhu <cit.> and Kim <cit.>, applied a cycle consistency loss to preserve the key attributes between two domains.








Concurrently, 

Collection Style Transfer (CST) is another approach to perform unpaired image-to-image translation task. The goal of CST is to compose the stylized image by taking advantage of multiple style images.

Sanakoyeu <cit.> handled the CST task with the help of a style-aware loss and an adversarial loss to improve the stylization results.
The difference between MST and CST is that MST can transfer any element in the content image set into a specific style. However, the style of CST is defined by all style images in the style image set, and CST usually doesn't offer a method that can transfer the input content image into a particular style.






§ STYLEREMIX NETWORK


The goal of style transfer can be defined as finding a stylized image o, which contains the visual fashion of target image s while retaining the content of the input image c. For MST task, recent methods often use two sets: content image set C={c_1, c_2, …, c_N}  and style image set S={s_1, s_2, …, s_M}, to perform style transfer task. More specifically, MST methods aim to offer a model, which can transfer any element of the content set C to a specific style s_j in the style set S.






 §.§ Overview

StyleRemix aims to learn a compact and interpretable continuous vector representation for each style. The framework of StyleRemix is shown in Fig. <ref>, which consists of three parts: an autoencoder, (encoder ℰ and decoder 𝒟), a style basis layer ℬ, and style weights layers 𝒲. 
In StyleRemix, all styles in style set are decomposed and stored in a shared convolution layer and each particular style  is constructed by a weighted style basis.

As a result, we can simply represent each specific style with its style weights given the shared style basis. 
Finally, the weighted style basis is convolved with the intermediate feature maps <cit.> of input content image to obtain the stylized image o.
	


 §.§ Encoder and Decoder
  
We do not set any limitation on the architecture of encoder and decoder as shown in Fig. <ref>. Since the stylized process in StyleRemix is the weighted style basis directly operating on the intermediate feature maps of the input content image given by autoencoder. Any symmetry or asymmetry autoencoder architecture can be used to produce the intermediate feature maps F = ℰ(c). For simplicity, we use a 3 layers symmetry encoder and decoder, which is similar to <cit.>.



 §.§ Style Basis and Style Weights
  

	
	

  
Style Basis. By sharing the redundant convolution operators, which may be used to produce similar texture patterns, coarsening or softening strokes among different styles, style basis B is introduced to describe an embedding space of the style set. By doing so, a specific style s_j in StyleRemix is described as a layer of weighted convolution filters, which map the high-level features of target style s_j to input image c. The transferred feature maps F_j given by style s_j can be achieved by the convolution operation of the weighted style basis B_j over the intermediate feature maps F:
	
    F_j = B_j ⊗F,

	where F∈ℝ^c_in× h × w, B_j ∈ℝ^c_out× c_in× k_h × k_w, F_j ∈ℝ^c_out× h × w, c_in and c_out are numbers of feature channels for F and F_j respectively, (h,w) is feature map size, and (k_h, k_w) is the kernel size. The weighted style basis B_j is a point in learned style embedding space. In another word, B_j is determined by its corresponding coordinates, style weights w_j. Therefore, B_j is defined as:
	
    B_j = (1w_j^T) ⊙B,

	where 1∈ℝ^c_out· k_h · k_w × 1, w_j ∈ℝ^c_in× 1, and B∈ℝ^c_out· k_h · k_w × c_in, and ⊙ is element-wise product. B_j in equation <ref> and <ref> are equivalent by reshaping operation. 
	
	In StyleRemix, style basis B is implemented as a single convolution layer ℬ, which locates at the end of encoder ℰ and the beginning of decoder 𝒟 as shown in Fig. <ref>.

	

  
Style Weights. As we mentioned above, one purpose of style basis is compressing the redundancy of different styles. On the other hand, we hope that style basis itself should be interpretable. 
	
In another word, if the style base has no contribution to a specific style, its coefficient should be close to zero. To deliver this intuition, all weighted style basis are forced to be a convex combination of style basis B. That is to say, style weights w_j lies in the simplex:
	
	
    Δ_c_in = {w_j ∈ℝ^c_in× 1}, 
        s.t.  w_j[i] ≥ 0  and ∑_i w_j[i] = 1.

	

	
	Style basis and style weights are jointly trained during one optimization iteration step. There are many ways to learn style weights while meeting the constraint in Eq. <ref>. For simplicity, we implement style weights as a learnable layer 𝒲, which consists of two sublayers: a linear layer without bias ℒ and a softmax layer 𝒮. The function of linear layer is defined as:
	
    y = ax,

	where a is a scalar, x, y∈ℝ^c_in× 1, 
	The function of softmax layer is defined as:
	
    y_i = exp(y_i)/∑_j exp(y_j),
	
	where y_i is ith element in y. By combining Eq. <ref> and <ref> and setting a=1, style weights can be achieve by: w = 1 →ℒ→𝒮. It is worth noticing that  one of the state of the art MST framework, StyleBank<cit.>, can be regarded as a special case of StyleRemix. To be specific, in StyleBank, one can combine all style basis together, and set the style weights to a block-wise one-hot vector which is not learnable.
	
		
	Since the style s_j is modeled by the coordinates in the learned style embedding space, style manipulation can be achieved by simply operating style weights. For instance, the convex combination among styles can be performed by the convex combination of the style weights. Besides, it is possible to produce various stylized results for a particular style s_j by adding noise to its corresponding style weights. Furthermore, since style basis are learned to represent the styles of the style set, StyleRemix even allows to achieve CST task by, such as, simply averaging the style weights vectors or offering the same value to all style weights. In that way, the final stylized results benefit from all style basis, which is the goal of CST.
	
	





 §.§ Loss Function.
 StyleRemix includes two training branches: autoencoder branch (ℰ→𝒟) and stylizing branch (ℰ→ (𝒲→ℬ) →𝒟, where 𝒲: 1 →ℒ→𝒮). In order to clearly understand the forward method of the stylizing branch, we show the example of how it works in Fig. <ref>. 


	
	For the autoencoder training branch, the Euclidean Distance between the input content image c and the reconstructed content image ĉ is adopted to measure the reconstruction error, which can be achieved by:
	
    ℒ_r(ĉ, c) = ||ĉ - c ||^2_2.

	
	For stylizing training branch, the perceptual loss defined in <cit.> is used to learn style representation. The perceptual loss can be achieved by:
	
    ℒ_p(ĉ, c, s_j) = αℒ_c(ĉ, c)  + βℒ_s(ĉ, s_j),

	where s_j is a specific style belonging to the style set S. 
	
	Content loss ℒ_c and style loss ℒ_s are respectively defined to measure the difference of feature maps and gram matrix <cit.> between different images. Both feature maps and gram matrix are computed from VGG-16 network <cit.>. Content loss ℒ_c and style loss ℒ_s can be respectively achieved by:
	
    ℒ_c(ĉ, c)  = ∑_l ∈{l_c}||F^l(ĉ) - F^l(c) ||^2_2, 
       ℒ_s(ĉ, s_j)  = ∑_l ∈{ l_s}1/n_l||G^l(ĉ) - G^l(s_j) ||^2_2,

	where F^l and G^l are feature maps and gram matrix calculated from lth layer of VGG-16 network <cit.>, n_l is the total number of units of layer l, {l_c} and {l_s} are layers which deliberately chosen to obtain F^l and G^l.

	




 §.§ Training Strategy.



StyleRemix is difficult to train because the model has to convincingly express all the styles in the style set while comprehensively describing the content information.
Inspired by <cit.> and <cit.>, our training strategy includes two phases: a warming-up stage and a finetuning stage. 


The detailed training steps of the warming-up and the finetuning stages are shown in Fig. <ref>. 
The warming-up stage is trying to make the network get a general picture of the content and style information to be learned gradually.
For this purpose, in the warming-up stage, each new style will be incrementally added to the current style set, from S_1 = {s_1}, S_2 = {s_1, s_2} to S_m = {s_1, s_2, …, s_m}. 
In the first K iterations, only the autoencoder is trained. 
During the following each K iterations, a new style set S_j will be trained until no more new style is available. 


After the warming-up stage, we conduct the finetuning phase to enforce the network to strengthen the learned content and style characteristics. In particular, we employ the T+1 training strategy, namely 1 round training on content images and T rounds training on the style set S_m to capture the content and style characteristics simultaneously.







§ EXPERIMENT








  
Implementation details. 
We employ a simple symmetry autoencoder same to <cit.> but with 256 channels style basis. 
The detailed architecture of network is shown in Table <ref>. The linear layer has 1 input channel and 256 output channels and the softmax layer has 256 input channels and 256 output channels.

We employed VGG-16 <cit.>, pre-trained on the ImageNet dataset <cit.>, as perceptual feature extractor. We respectively chose layer Conv2_2 and layers Conv1_2, Conv2_2, Conv3_2, and Conv4_2 for content and style losses. The parameters α and β in Eq. <ref> are respectively set to: α = 1 and β = 3e4 for all results of StyleRemix  in MST task except for specific statements. We randomly sample 1000 images from Microsoft COCO dataset <cit.> as content dataset. For style dataset, S_50 contains 50 style images which are widely used in other MST papers. 

 Note that another style dataset, which contains 50 paintings of Van Gogh, is employed for CST task.

During the training process, content images are randomly cropped to 512 × 512, style images are scaled to 600 along large side. StyleRemix is trained with batch size  4, style weights initialized with the same value, parameters T in style learning stage is set to 2, learning rate 0.001 with decayed 0.8 at every 30K iterations, and Adam optimizer <cit.> for 300K iterations. Our implementation is based on PyTorch <cit.>. 



  
Baselines
To measure the performance of StyleRemix, we compare StyleRemix against several baselines which aim to MST task to better understand the differences between MST methods, such as <cit.>, <cit.>, and <cit.>. Also, we compare the results with SST tasks to better comprehend the differences between MST and SST. The details of baselines are described as follows:

NS Neural style <cit.> is based on slow optimization method to stylized a content image, that makes NS always be the gold-standard method in all style transfer community. We train NS with 4000 iterations for several styles by using all default parameters from its open source project. 

FNS Fast Neural Style (FNS) train a single network for each style. We train FNS for several styles with randomly corp content images into 512 × 512 and scaling style image to 600 with long side as the same we set to StyleRemix. The parameters α and β in Eq. <ref> are respectively set to: α = 1e5 and β = 1e10.

StyleBank As we mentioned above, StyleBank <cit.> is a special case of StyleRemix with one-style-one-style-basis setting. For fair comparison, we implement StyleBank <ref> in the same settings as we mentioned above for StyleRemix. Therefore, except for the extra ISB training stage for StyleRemix, all other settings are the same except for specific statements. 

CIN Conditional Instance normalization (CIN) <cit.> learn the scaling and shifting parameters in instance normalization layers to  identify a specific style. All parameters are set to the default value from their open source project. We trained CIN with style set S_50 for MST task. 

MSG-Net Multi-Style Generative Network (MST-Net) <cit.> forward both style and content image through pre-trained VGG network to obtain content and style representation. By learning some trainable weights which minimize the objective function, MST can be achieved by MSG-Net. We used S_50 to train MSG-Net 5 epochs with default parameters from its open source project for MST task.










 §.§ Multi-Style Transfer



We compare StyleRemix against several baselines to measure the performance of MST task. Baselines include: Neural style (NS) <cit.>, Fast Neural Style (FNS) <cit.>, StyleBank <cit.>, Multi-Style Generative Network (MST-Net) <cit.>, and Conditional Instance normalization (CIN) <cit.>. StyleBank is trained using the same network and parameters as StyleRemix. Other baselines are trained with default parameters from their open source projects, except that NS is optimized for 4000 iterations, MSG-Net is trained for 5 epochs. Besides, all MST baselines are trained with the same style set S_50 as StyleRemix.





  
Qualitative Result. 
The stylized images (512 × 512) are shown in Fig. <ref>.

In general, our method has achieved competitive stylization results compared to other baselines.





More specifically, for Single-Style Transfer (SST) methods, since plenty of parameters can be used to describe a single style, SST methods often yield better results than other baselines. However, NS produces many artifacts result in the details of stylized results looking unnatural, rows 1, 2, and 4. 
It may be caused by that the optimization process is based on the representation of the whole image.  FNS yields larger texture pattern, which even larger than textures in target style image, rows 2, 3, and 4. 
This issue is possibly due to the reason that FNS uses 9×9 kernels and deep architecture which has a larger receptive field.


For MST methods, since the diversity of style set has a significant influence on the stylized results, they have to compromise with different styles. For CIN, it can be observed the color histogram shift only, and some weird textures scattered inconsistently throughout, rows 1 and 3. Besides, CIN fails to reproduce texture patterns in style images. 
 One possible reason is CIN's style representation come from all the instance normalization layers, which do not explicitly separate content and style. For the same reason, CIN may be incapable of explicitly describing the textures of style.
For MSG-Net, it seems to transfer the same structure but different color textures, rows 1, 2, and 3, and contain some large artifacts, rows 1, 3, and 5. The root cause may be that MST-Net explicitly model the universal transfer function by a set of learnable weights. Likewise, MSG-Net suffers from some interferences among different styles.

Comparing with SST methods, StyleRemix produces appealing stylized results in a different way. To be specific, StyleRemix creates the fundamental elements of style, such as texture patterns, softening strokes, instead of the abstract color area produced by NS or FNS, rows 2 and 5.
Besides, thanks to the separated representations of content and style, StyleRemix yields more visually pleasing results

than CIN and MSG-Net. For example, StyleRemix has captured textures in style images, rows 1, 3, and 6. Since StyleBank is a special case of StyleRemix, their stylized results are visually similar.    




 



















  
Quantitative Results. 
We compare the quantitative results of StyleRemix with other baselines in three aspects: Deception Rate <cit.> (DR), the number of float-point operations (FLOPs), and the cost of adding a new style (Cost/Style). 
The DR is calculated as the fraction of generated images which were classified by a network as the artworks of an artist for which the stylization was produced. The network is pre-trained on Wikiart to classify 624 artists.
For DR, we use the available 14 artists from the open source project of  Sanakoyeu <cit.>. Since DR test mainly focus on paintings, we set the β = 6e4 for paintings stylization. And we generate 4200 stylized images (14 styles, 300 per style) for every method. 
FLOPs is calculated by counting the number of all convolution operations for single style. Since MSG-Net is based on the architecture of FNS <cit.> and need an extra parametric network to obtain style representation, we didn't list its FLOPs. 

Cost/Style is defined by the parameters cost of adding a new particular style.
Results in Tab. <ref> show that our method has competitive performance compared to other baselines.
Worth mentioning that, StyleRemix requires the lightest cost when adding a new style.






















 §.§ Style Visualization
 

According to our best knowledge, only CIN <cit.> claims that they learn the embedding of style and more experiment results are presented in <cit.> to support this opinion. We compare the style embedding learned by StyleRemix and CIN. The style representation of StyleRemix is S_sr∈ℝ^256 × 1, and the style representation of CIN is S_cin = {γ_s, β_s},  S_cin∈ℝ^3206 × 1, where γ_s and β_s are respectively the learned mean and standard deviation of instance normalization layers. 


We examine the low dimensional style representations of both CIN and StyleRemix. 
The style parameter of StyleRemix, in Eq. <ref>, is set to β = 6e4,  to capture more information about style. Also, in order to achieve low dimensional display, we first employ Principal Component Analysis to reduce both style representation to 10 dimensions and then t-SNE <cit.> dimensional reduction technique to further reduce the 10 dimensions style representation to 2 dimensions. Notice t-SNE will necessarily distort the representation significantly in order to compress the representation to small dimensionality. Therefore, we restrict our analysis to the qualitative description. The t-SNE results are shown in Fig. <ref>. 

Comparing the t-SNE results of  CIN in Fig. <ref> (left) and StyleRemix in Fig. <ref> (right), one may notice StyleRemix has better locality than CIN. As shown in Fig. <ref>, we use different color boxes to identify different class styles. For instance, for the t-SNE results of StyleRemix, in the yellow box, the styles contain only black, white color and both of them look like Sketch. 

We also show the correlation matrix of 10 selected styles, which is directly calculated by original high dimensional style weights, in Fig.<ref>. The correlation matrix provides strong evidence that the style embedding learned by StyleRemix can distinguish styles in semantic similarity. The visually similar styles have high correlation coefficients, and the visually different styles have low correlation coefficients. Therefore, the correlation matrix of StyleRemix (Fig.<ref> right) shows Blocking Effect, which is not observed in the correlation matrix of CIN (Fig. <ref> left).

	






 §.§ Style Manipulation
 
We conduct the Style Remixing manipulation by decomposing all styles into the same style embedding space and recombining them to create a large number of new style effects. 


We show some heuristic examples to show how to remix styles via simply operating the style weights.







  
Convex Combination. Convex combination offers a smooth transition from one style to the other, which can be achieved by simply apply the convex combination in style weights of different styles. Supposing there are two different styles s_l and s_k with style weights w_l and w_k, the convex combination style s_new can be achieved by:
	
    w_new = αw_l + (1-α) w_k,

	where 0 ≤α≤ 1, w_new is the style weights of  s_new. Fig. <ref> presents the convex combination results, of which one can observe the smooth transition between the two styles.
	
	




  
Perturbation Study. 
We show an example how to remix styles by adding Gaussian noise to a particular style. The results are shown in Fig. <ref>. We add Gaussian noise ε∼ N(μ, σ^2), where μ=1256, σ = 0.005, to the style weights w_j. Then normalized the vector v = w_j + ε by v = v /  ∑^256_i = 1 v_i.
	
	
	
	
	One may notice that parts of the style weights of perturbed style are negative. Interestingly, it is difficult to differentiate the perturbed stylized results from the original results. These results may suggest that StyleRemix has captured a smooth embedding space, which is robust to noise to some extend.
	



  
Collection Style Transfer. StyleRemix may offer a way to perform Collection Style Transfer (CST) task. As we mentioned before, the style of CST is defined by all styles in the style set. As the style elements of the style set are fully represented by the style basis in StyleRemix, it is possible to define the style of CST by manipulating the style weights. We train StyleRemix with 50 random paintings of Vincent Van Gogh from Wikiart <cit.>. The parameters α and β in Eq. <ref> are respectively set to: α = 1 and β = 6e4. We show CST results in Fig. <ref> with two simple manipulating strategies. One is to use the average vector of all 50 styles (row 1) as style weights, another is to set the same value 1256 (row 2) for all weights. They yield similar results as both are capturing all the style elements of Van Gogh paintings. On the other hand, they have subtle difference on chrominance, luminance and local texture as the style weights varied.













 §.§ Style Scalability

 
Style scalability is critical to SEL methods. However, it is difficult to measure since the maximum capabilities of a single model is highly related to the particular set of styles.

For example, a SEL method is likely to have good style scalability if all styles come from one domain instead of different sources.

To evaluate the capacity of StyleRemix, we experiment with the same diverse style set S_50 as Section <ref>
and study how varying the number of styles affects the stylization.

We train StyleRemix with subsets of S_50, S_5⊂ S_10⊂ S_15, with the same parameters.


According to Fig. <ref>, as the number of styles increases, the result shows visually consistent stylization effect though some details of style may fade progressively. This phenomena indicates that StyleRemix can handle a certain number of diverse styles without significant visual degradation. 























§ CONCLUSION


In this paper, we have proposed a style embedding learning approach, StyleRemix, for multi-style transfer task. 

Combining interpretability with efficiency, our method effectively perform multi-style transfer and produce  visually appealing stylization. In addition, the proposed method can be extended style visualization task. 
More importantly, StyleRemix provides several ways to remix styles in the style embedding space. Experimental results demonstrate the effectiveness of StyleRemix on many multi-style transfer tasks. As future direction, one may extend StyleRemix to handle arbitrary style transfer.










ieee



