Sign to speech convolutional neural network-based Filipino sign language hand gesture recognition system
Loading...
Date
2021
Journal Title
Journal ISSN
Volume Title
Publisher
2021 International Symposium on Computer Science and Intelligent Controls (ISCSIC)
Abstract
Sign Language Recognition is a breakthrough for helping deaf-mute people and has been studied for many years. Unfortunately, every research has its own limitation and are still unable to be used commercially. In this study, we developed a real-time Filipino sign language hand gesture recognition system based on Convolutional Neural Network. A manually gathered dataset consists of 237 video clips with 20 different gestures. This dataset underwent data cleaning and augmentation using image pre-processing techniques. The Inflated 3D convolutional neural network was used to train the Filipino sign language recognition model. The experiments considered retraining the pretrained model with top layers and all layers. As a result, the model retrained with all layers using imbalanced dataset was shown to be more effective and achieving accuracy up to 95% over the model retrained with top layers to classify different signs or hand gestures. Using the Rapid Application Development model, the Filipino sign language recognition application was developed and assessed its usability by the target users. With different parameters used in the evaluation, the application found to be effective and efficient.
Description
Keywords
Filipino Sign Language, Sign-to-speech system, Convolutional neural network, Hand gesture recognition, Artificial intelligence, Assistive technology, Computer vision, Sign Language, Hand Gestures, Hand Gesture Recognition, Convolutional Neural Network, Developed Model, Clean Data, Video Clips, Sign Language Recognition, Model Performance, Training Set, Deep Learning, Image Processing, Validation Set, Recognizable, Computer Vision, Model Building, Transfer Learning, Series Of Images, Processing Software, Optical Flow, American Sign Language, Deaf People, Lower Learning Rate, Applying Transfer Learning, Deaf Community, Validation Accuracy, CNN Model, Video Editing
Citation
Jarabese, M. B. D., Marzan, C. S., Boado, J. Q., Lopez, R. R. M. F., Ofiana, L. G. B., & Pilarca, K. J. P. (2021). Sign to speech convolutional neural network-based Filipino sign language hand gesture recognition system. 2021 International Symposium on Computer Science and Intelligent Controls (ISCSIC). 147–153. DOI 10.1109/ISCSIC54682.2021.00036