Yoruba Sign Language Digit Recognition System using Deep Convolution Neural Network and Machine Learning

Tools to support communication in the Yor̀ubá language between the hearing-impaired and unimpaired individuals are limited. Thus, this study implements and evaluates a real-time sign language recognition system for Yor̀ubá sign language and determines specific features responsible for recognizing the numeral gestures. A sample of 1000 hand gestures containing ten gestures (each for numeral 1–10) was collected from a deaf school in Nigeria and pre-processed using the augmented techniques to generate 11000 variations of the sign digit images. The processed digits were recognized using Convolution Neural Networks (CNN). The developed model was compared with pre-trained models and three other prevalent machine learning techniques: Artificial Neural Networks, Support Vector Machines, and K-nearest neighbor, using precision, recall, F1-scores, and accuracy as performance metrics. The performance results showed that the CNN model outperformed other models with an average precision (99.52%), recall rate (99.50%), F1-score (99.49%), and accuracy (99.50%). Thus, the developed CNN model successfully recognized Yor̀ubá hand gestures and would thus assist in bridging the gap between hearing-impaired and unimpaired individuals.