Abstract

Skies are common backgrounds in photos but are often lessinteresting. Professional photographers correct this by using sophisticatedtools that are beyond the ability of ordinary users. In this work, we propose an automaticbackground replacement algorithm that can generate realistic, artifact-free imageswith diverse styles of skies. The key idea of our algorithm is to utilizevisual semantics to guide the entire process including sky segmentation, searchand replacement. First, we train a deep convolutional neural network for semanticscene parsing, which is used as visual prior to segment sky regions in acoarse-to-fine manner. Second, in order to find proper skies for replacement,we propose a data-driven scheme based on semantic layout of the input image. Finally,to re-compose the stylized sky with the original foreground naturally, anappearance transfer method is developed to match statistics locally and semantically.We show that the proposed algorithm can automatically generate a set ofvisually pleasing results. In the end of the talk, a brief summarization of recent work on visualobject recognition will also be discussed.

Bio

Yi-HsuanTsai received the B.S. degree in Electronics Engineering from NationalChiao-Tung University, Hsinchu, Taiwan and the M.S. degree in Electrical Engineeringand Computer Science from University of Michigan, Ann Arbor. He is currentlyworking toward the PhD degree in the department of Electrical Engineering andComputer Science at University of California, Merced. His research interestsinclude computer vision, computational photography and machine learning with thefocus on visual object recognition and exemplar-based image editing. He is therecipient of the CVPR Doctoral Consortium in 2016 for the work on video objectsegmentation. He also did research internships at Qualcomm Research, Max PlanckInstitute and Adobe Research.