In diversified image style transfer tasks, the goal is to translate an everyday photograph to diverse stylized images conditioned on the style of an artwork. Although several style transfer methods have achieved certain diversity through noise injection, they still have two unsolved problems: (1) relatively limited diversity and (2) significant degradation of quality. In this work, we introduce an effective Gating PatternPyramid block (GPP) to resolve these issues by employing a well-designed multibranch architecture. The main idea is underpinned by a finding that different network architectures capture different style patterns from the same artwork. In addition, the GPP block is compatible with many feed-forward style transfer models and empowers them with the ability of generating diverse stylization results. To the best of our knowledge, this is the first style transfer method that achieves significant diversity without injecting random noise, which provides an inspiring perspective for diverse translation research. We perform qualitative and quantitative experiments, showing the effectiveness and superiority of our method against the state-of-the-art diversified style transfer methods.
Arbitrary image style transfer aims to mimic the artistic features of a randomly given style image while maintaining the semantic characters of a reference content image. Existing algorithms have achieved astonishing style transfer results. However, they are insufficient to capture the global information of the content image due to the locality in convolutional neural networks. As a result, the content structures of the stylized images are disrupted by scattered textures. To address this issue, we propose to embed additional global spatial information of the reference content image into image style transfer algorithms with the adaptive instance normalization trick. We train a light-weighted network to distill the global structural information provided by the depth map of the reference content image. We then utilize the refined results to enhance the transformations between deep features of the inputs. Experimental results show that our proposed method can generate stylized outputs with more consistent structures and less unwanted textures, achieving impressive visual effects.
This work proposes an improved ranking model of learning local feature descriptors for matching image patches by introducing a variance shrinkage constraint. Previous ranking losses, such as triplet ranking loss and quadruplet ranking loss, have proven powerful in separating corresponding patch pairs from noncorresponding ones. However, they are unable to restrict the intraclass variation since they are only designed to keep noncorresponding pairs away from corresponding ones. Consequently, those scattered pairs get mixed up near the separating hyperplane, which are difficult to discriminate and may disrupt the performance. To resolve this problem, we introduce a variance shrinkage constraint that aims to reduce the variance of patch pairs in the same class and force them to be close to each other. The combination of ranking losses and the variance shrinkage constraint can efficiently reduce overlaps between patch pairs of different classes, which is confirmed by our experiments. Experiments also show that our model achieves a significant improvement in performance compared with original ranking models and other latest methods.
In recent years, gradient-domain methods have been widely discussed in the image processing field, including seamless cloning and image stitching. These algorithms are commonly carried out by solving a large sparse linear system: the Poisson equation. However, solving the Poisson equation is a computational and memory intensive task which makes it not suitable for real-time image editing. A new matrix decomposition graphics processing unit (GPU) solver (MDGS) is proposed to settle the problem. A matrix decomposition method is used to distribute the work among GPU threads, so that MDGS will take full advantage of the computing power of current GPUs. Additionally, MDGS is a hybrid solver (combines both the direct and iterative techniques) and has two-level architecture. These enable MDGS to generate identical solutions with those of the common Poisson methods and achieve high convergence rate in most cases. This approach is advantageous in terms of parallelizability, enabling real-time image processing, low memory-taken and extensive applications.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.