Automatic date fruit sorting system based on machine learning and visual features

Abstract

The Algerian date market holds signi cant economic potential, ranking third in global date production. However, a substantial gap exists between its production capacity and date fruit exports. This limitation stems from slow, error-prone, and labour-intensive traditional manual sorting methods that rely on visual inspection of various quality factors. This thesis addresses these limitations by leveraging Convolutional Neural Networks (CNNs) to automate date fruit sorting. CNNs incorporate information beyond single-view visual data, creating a more e cient solution. Our novel approach utilizes a multimodal dataset that combines features from multiple fruit faces alongside thermal imaging data and weight measurements, providing a richer and more comprehensive representation of each date fruit. The thesis delves into three key contributions that explore and demonstrate the e ectiveness of these CNN-based approaches. The rst contribution demonstrates the e ectiveness of a multi-modal approach with CNNs, achieving 94% testing accuracy using a VGG16 model by combining all information into one visual data input. The second contribution investigates multi-modal data fusion with late fusion techniques. In Scenario I, fruits are classi ed based on four-view images. Scenario II extends scenario I by incorporating thermal images and weight measurements. The results highlight the signi cant accuracy improvement observed when incorporating additional features in Scenario II. The nal contribution addresses the limitations of single-face analysis and small datasets. It proposes a method to combine information from multiple fruit faces and utilizes permutation functions to increase dataset size. This approach signi - cantly enhances classi cation accuracy, with a ne-tuned VGG16 model achieving perfect accuracy (100%) with merged four faces, highlighting the potential of data augmentation techniques to address limitations associated with limited datasets. In conclusion, this thesis demonstrates the potential of Convolutional Neural Networks (CNNs) combined with multi-modal data fusion. By leveraging information from four visual images capturing di erent faces of the date fruit, the proposed approach enhances the accuracy and richness of information about the entire fruit. This paves the way for revolutionizing automated Algerian date fruit sorting, ultimately leading to a more e cient and accurate future for the Algerian date fruit market