Explainable artificial intelligence (xAI) applied to deep computer vision of microscopy imaging and spectroscopy for assessment of oleogel stability over storage

During storage, changes in crystal lattice conformation of wax-based oleogels can cause oil to separate from the matrix, affecting appearance, texture and, consequently, perceived quality. In this sense, our study investigated conformational changes through microscopic images of oleogels stored during to nine months and explored the relation between these changes and oil holding capacity or oil loss. A comparative analysis between these results and non-invasive techniques via spectroscopic methods was performed, with the aim of obtaining complementary interpretation about the structural and chemical transformations of oleogels over storage. We employed a convolutional neural network (CNN) coupled with explainable artificial intelligence (XAI) to analyse the microscopic images, allowing us to identify the most influential crystalline regions for prediction. Classification model for oleogel storage period achieved accuracy of 87.53%. The results demonstrate that the use of deep computer vision systems (DCVS) combined with XAI provides an effective approach to monitor the storage stability of different oleogels, based on the detailed analysis of crystalline networks depicted in microscopic images. Near-infrared (NIR) and Raman spectroscopy were applied to identify oleogel modifications during storage. The VIP scores from NIR and Raman models indicated changes in bands associated with oxidation process, allowing to associate them with physical changes in the crystal conformation and the loss of oil holding capacity.