In Late-Medieval panel paintings from the Tuscan area, mechanical tools called punches were used to impress repeated motifs on gold foils to create decorative patterns. Such patterns can be used as clues to objectively support the attribution of the paintings, as proposed by art historian Erling S. Skaug in his decades-long study on punches. We investigate the feasibility of employing automatic pattern recognition techniques for accelerating the process of classification of punches by experts working in the field. We propose a system composed of (a) a Convolutional Neural Network for categorizing a punch contained in a frame, and (b) an additional component for uncertainty estimation, aimed at recognizing possible Out-of-Distribution (OOD) samples. After collecting a set of 3815 punches from four 14th century panel paintings from Tuscany, we train a Convolutional Neural Network which achieves very high test-set accuracy. As far as the uncertainty estimation is concerned, we experiment with two techniques, OpenGAN and II-loss, both exhibiting very positive results. The former seems to work better on specific data extracted from images of panel paintings, while the latter showcases a more consistent behavior when considering additional OOD data obtained randomly. These outcomes indicate that an application of our system in support of experts is feasible, although we subsequently show that additional experiments on larger datasets might be required

An Artificial Intelligence System for Automatic Recognition of Punches in Fourteenth-Century Panel Painting

Zullich Marco
;
Macovaz Vanja;Pinna Giovanni;Pellegrino Felice Andrea
2023-01-01

Abstract

In Late-Medieval panel paintings from the Tuscan area, mechanical tools called punches were used to impress repeated motifs on gold foils to create decorative patterns. Such patterns can be used as clues to objectively support the attribution of the paintings, as proposed by art historian Erling S. Skaug in his decades-long study on punches. We investigate the feasibility of employing automatic pattern recognition techniques for accelerating the process of classification of punches by experts working in the field. We propose a system composed of (a) a Convolutional Neural Network for categorizing a punch contained in a frame, and (b) an additional component for uncertainty estimation, aimed at recognizing possible Out-of-Distribution (OOD) samples. After collecting a set of 3815 punches from four 14th century panel paintings from Tuscany, we train a Convolutional Neural Network which achieves very high test-set accuracy. As far as the uncertainty estimation is concerned, we experiment with two techniques, OpenGAN and II-loss, both exhibiting very positive results. The former seems to work better on specific data extracted from images of panel paintings, while the latter showcases a more consistent behavior when considering additional OOD data obtained randomly. These outcomes indicate that an application of our system in support of experts is feasible, although we subsequently show that additional experiments on larger datasets might be required
File in questo prodotto:
File Dimensione Formato  
10016708.pdf

accesso aperto

Tipologia: Documento in Versione Editoriale
Licenza: Creative commons
Dimensione 176 B
Formato Adobe PDF
176 B Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11368/3038944
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 2
  • ???jsp.display-item.citation.isi??? 1
social impact