Ms. Jiaxin Wang Profile

Jiaxin Wang

at Northwest Univ

SPIE Involvement:

Author

Publications (1)

Proceedings Article | 8 July 2021 Presentation + Paper

Deep learning for the extraction of sketches from spectral images of historical paintings

Qunxi Zhang, Shan Cui, Lu Liu, Jiaxin Wang, Jun Wang, Erlei Zhang, Jinye Peng, Sotiria Kogou, Florence Liggins, Haida Liang

Proceedings Volume 11784, 1178407 (2021) https://doi.org/10.1117/12.2593680

KEYWORDS: Edge detection, Convolution, Imaging spectroscopy, Image segmentation, Data modeling, Near infrared, Principal component analysis, Independent component analysis, Computer programming, Reflectivity

Read Abstract +

The sketches of painted cultural objects can be the most indicative of the style of paintings. Extraction of the sketches is an integral process used by conservators and art historians for documentation and for artists to learn historical painting styles through copying and painting. However, at present, sketch extraction is mainly manually drawn, which is not only time-consuming, but also subjective and dependent on experience. Therefore, both accuracy and efficiency need to be improved. In recent years, with the development of machine learning, a series of extraction methods based on edge detection have emerged. However, most of the existing methods can only perform successful extraction if the sketches are well preserved , but for the data with faded sketches or severe conservation issues, the extraction methods need to be improved. It is beneficial to extract the bands that accentuate the sketches while suppressing the effects of the degraded areas and the overlapping paints. We propose a sketch extraction method based on hyperspectral image and deep learning. Firstly, the hyperspectral image data is collected and the bands sensitive to the sketches are extracted by a prior knowledge of the sketches (e.g. near infrared bands will be chosen if the sketches are made of carbon ink), and a dataset including a large number of existing natural images is used to pre-train the bi-directional cascade network (BDCN). The network parameters in the model are then fine-tuned by using the images of painted cultural objects drawn by experts, so as to solve the problem of insufficient sketch dataset of painted cultural objects and enhance the generalization ability of the model. Finally, the U-net network is used to further suppress the noise, i.e. unwanted information, and make the sketch clearer. The experimental results show that the proposed method can not only effectively extract sketch from ideal data, but also extract clear sketches from data with faded sketches and even with noise interference. It is superior to the other six advanced based on edge detection methods in visual and objective comparison, and has a good application prospect. The proposed deep learning method is also compared with an unsupervised clustering method using Self-Organising Map (SOM) which is a ‘shallow learning’ method where pixels of similar spectra are grouped into clusters without the need for data labeling by experts.

View contact details

UPDATE YOUR PROFILE

Is this your profile? Update it now.

Sign into your SPIE.org account

Don’t have a profile and want one?

Create an account on SPIE.org

Keywords/Phrases

Search In:

Publication Years