End-to-end learning for arbitrary image style transfer

Real-time arbitrary style transfer is based on a feed-forward network, which consists of a pre-trained encoder, a feature transformer, and a trainable decoder. However, the previous approach has some degrade in style quality of output image because the pre-trained encoder is not optimised for image style transfer but originally for image classification task. An end-to-end learning scheme is introduced that optimises the encoder as well as the decoder for the task of arbitrary image style transfer. Experiments conducted with a public database proves that the style transfer network trained with the end-to-end learning scheme outperforms the network with a fixed encoder in terms of minimising both content and style losses and quality of the stylised images.