- Title
- Improving signer-independence using pose estimation and transfer learning for sign language recognition
- Creator
- Marais, Marc, Brown, Dane L, Connan, James, Boby, Alden
- Subject
- To be catalogued
- Date
- 2022
- Type
- text
- Type
- article
- Identifier
- http://hdl.handle.net/10962/463406
- Identifier
- vital:76406
- Identifier
- xlink:href="https://doi.org/10.1007/978-3-031-35644-5"
- Description
- Automated Sign Language Recognition (SLR) aims to bridge the com-munication gap between the hearing and the hearing disabled. Com-puter vision and deep learning lie at the forefront in working toward these systems. Most SLR research focuses on signer-dependent SLR and fails to account for variations in varying signers who gesticulate naturally. This paper investigates signer-independent SLR on the LSA64 dataset, focusing on different feature extraction approaches. Two approaches are proposed an InceptionV3-GRU architecture, which uses raw images as input, and a pose estimation LSTM architecture. MediaPipe Holistic is implemented to extract pose estimation landmark coordinates. A final third model applies augmentation and transfer learning using the pose estimation LSTM model. The research found that the pose estimation LSTM approach achieved the best perfor-mance with an accuracy of 80.22%. MediaPipe Holistic struggled with the augmentations introduced in the final experiment. Thus, looking into introducing more subtle augmentations may improve the model. Over-all, the system shows significant promise toward addressing the real-world signer-independence issue in SLR.
- Format
- computer, online resource, application/pdf, 1 online resource (13 pages), pdf
- Publisher
- SpringerLink
- Language
- English
- Relation
- International Advanced Computing Conference, Marais, M., Brown, D., Connan, J. and Boby, A., 2022, December. Improving signer-independence using pose estimation and transfer learning for sign language recognition. In International Advanced Computing Conference (pp. 415-428). Cham: Springer Nature Switzerland, International Advanced Computing Conference p. 118 2022 1865-0937
- Rights
- Publisher
- Rights
- Use of this resource is governed by the terms and conditions of the SpringerLink Terms of Use Statement ( https://link.springer.com/termsandconditions)
- Rights
- Closed Access
- Hits: 23
- Visitors: 24
- Downloads: 1
Collections
Brown, Dane A (Dr)
Connan, James (Mr)
RU Department of Computer Science
SDG 10. Reduced Inequalities
SDG 09. Industry, Innovation and Infrastructure
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details | SOURCE1 | Improving signer-independence using pose estimation and transfer learning for sign language recognition.pdf | 772 KB | Adobe Acrobat PDF | View Details |