Image based recognition of Pakistan sign language
Abstract
Sign language is the language of gestures used for non-verbal communication. Thispaper deals with alphabets and digit signs recognition from Pakistan Sign Language(PSL). The deep pixels-based analysis is pursued for the recognition of fingers (fromindex to small finger) while thumb position is determined through Template Matching.After fingers identification, the isolated signs are recognized based on finger states ofbeing raised or lowered besides thumb’s position in 2D. For a quick recognition, signsare categorized into seven groups. The algorithm identifies these groups following amodel of seven phases. The system’s accuracy achieved a satisfactory level of 84.2%when evaluated with signs comprising 180 digits and 240 alphabets.References
Al-Jarrah, O. & Alaa, H. 2001. Recognition of gestures in Arabic sign language using neuro-fuzzy
systems. Artificial Intelligence, 133(1-2): 117-138.
Alia, O. M., Mandava R., Ramachandram D. & Aziz M. E., 2009. A novel image segmentation
algorithm based on harmony fuzzy search algorithm. In international conference of soft computing
and pattern recognition, pp 335–340.
Alvi, A. K., Azhar, M. Y. B., Usman, M., Mumtaz, S., Rafiq, S., Rehman, R. U. & Ahmed, I. 2005.
Pakistan sign language recognition using statistical template matching. International Conference on
Information Technology, (ICIT 2004), Istanbul, Turkey pp. 108-111.
Assaleh K. & Al-Rousan, M. 2005. Recognition of Arabic sign language alphabet using polynomial
classifiers. EURASIP Journal on Applied Signal Processing, 13, pp. 2136-2146.
Chalechale A., & Safae, F. 2008. Visual-based interface using hand gesture recognition and object
tracking. Iranian Journal of Science and Technology; Transaction B, Engineering Shiraz University,
: 279–293.
Doliotis, P., Mcmurrough, C., Eckhard, D. & Athitsos, V. 2011. Comparing gesture recognition accuracy
using colours and depth information. Conference on Pervasive Technologies Related to Assistive
Environments, Heraklion, Crete, Greece.
Erdem, U. M. & Sclaroff, S. 2002. Automatic detection of relevant head gestures in American Sign
Language communication. International Conference on Pattern Recognition,Vol.1, BRAC
University Dhaka, Bangladesh.
Fan, Z. & Ling S. 2014. Weakly-supervised cross-domain dictionary learning for visual recognition,
International Journal of Computer Vision, 109(1-2): 42-59.
Fang, G., Wen, G. & Zhao, D. 2007. Large-vocabulary continuous sign language recognition based on
transition-movement models. IEEE Transactions on Systems, Man & Cybernetics, Part A: Systems
and Humans, 37(1): 305-314.
Holden, E. & Owens., R. 2001. Visual sign language recognition. Multi-Image Analysis, Lecture Notes
in Computer Science, Springer, pp. 270–287.
Jerde, T. E., Soechting, J. E. & Flanders, M. 2003. Biological constraints simplify the recognition of
hand shapes. Transactions on Biomedical Engineering, 50(2): 265–269.
Kadous M. W. 1996. Machine recognition of Auslan signs using power gloves towards large-lexicon
recognition of sign language. Proceedings of the Workshop on the Integration of Gestures in
Language & Speech, pp. 165-174, Delaware and Wilmington, Delaware .
Kuch, J. J. & Huang, T. S. 1995. Vision-based hand modelling and tracking for virtual teleconferencing
and tele-collaboration. IEEE International Conference on Computer Vision, Cambridge, MA,
USA.
Ling, S., Li, L., & Xuelong, L. 2014. Feature Learning for Image Classification via Multi objective
Genetic Programming, IEEE Transactions on Neural Networks and Learning Systems, 25(7): 1359-
Liu, X. & Fujimura, K. 2004. Hand gesture recognition using depth data. In: Proceedings of FGR, Seoul,
Korea.
Muir, L. & Leaper, S. 2003. Gaze tracking and its application to video coding for sign language. Picture
Coding Symposium, the Robert Gordon University, School hill, Aberdeen, UK.
Nagarajan, S., Subashini, T. & Ramalingam, V. 2012. Vision Based Real Time Finger Counter for Hand
Gesture Recognition. International Journal of Technology, 2(2): 1-5, CPMR-IJT.
Ong, E. J., Cooper, H., Pugeault, N. & Bowden, R. 2012. Sign language recognition using sequential
pattern trees. In International Conference on Computer Vision and Pattern Recognition (CVPR),
pp. 2200–2207.
Raees, M., Sehat U. 2014. Alphabet signs recognition using pixels-based analysis. 5th Conference on
Language and Technology (CLT14), DHA Suffa University (DSU), Karachi.
Rajeshree, R. & Manesh, K. 2009. Gesture recognition by thinning method. International Conference on
Digital Image Processing, IEEE Computer Society, Bangkok, pp. 284-287.
Ravikiran, J., Kavi, M., Suhas, M., Dheeraj, R., Sudheender, S. & Nitin, V. P. 2009. Finger detection
for sign language recognition. Proceedings of the International Multi Conference of Engineers and
Computer Scientists, Hong Kong.
Saad, A., Jonas, B. & Hedvig, K. 2012. Visual recognition of isolated Swedish sign language signs. PhD
Thesis Cornell University Ithaca, New York, United States.
Shanableh, T. 2007. Arabic sign language recognition in user independent mode. IEEE International
Conference on Intelligent and Advanced Systems, Kuala Lumpur, Malaysia.
Simon, L., Marco, D. & Block, B. 2011. Sign language recognition with kinect. MS Thesis, Freie
Universität Berlin, Germany.
Starner, T., Weaver, J. & Pent I. A. 1998. Real-time American sign language recognition using desk and
wearable computer based video. IEEE Transactions on Pattern Analysis & Machine Intelligence,
(2): 1371-1375.
Subha, R. & Balakrishnan, G. 2011. Real-time Indian sign language recognition system to aid deafdumb
people. IEEE 13th International Conference on Communication Technology. Indian Institute
of Technology Kharagpur, Kharagpur, India.
Sumera, K., Younus, M. & Shaleeza, S. 2008. Recognition of gestures in Pakistani sign language using
fuzzy classifier. 8th International Conference on Signal Processing, Computational Geometry and
Artificial Vision, (ISCGAV’08), Rhodes, Greece.
Vezhnevets, S. V. & Reeva, A. 2003. A Survey on pixel-based skin colour detection techniques.
Proceedings of Graphicon, Moscow, Russia.
Von, A., Knorr, M., Kraiss, K. & Jonghwa, K. 2008. The significance of facial features for automatic sign
language recognition. 8th IEEE International Conference on Automatic Face Gesture Recognition,
Amsterdam, Netherlands.
Wen, G., Jiyong, M., Jangqin, W. & Chunli, W. 2000. Sign language recognition based on HMM/ANN/
DP. International Journal of Pattern Recognition and Artificial Intelligence, 14(5): 587-602.
Yang H. D. & Lee S. W. 2010. Robust sign language recognition with hierarchical conditional random
fields. In 20th International Conference on Pattern Recognition, Istanbul Turkey.
Zabulis, X., Baltzakisy, H. & Argyroszy, A. 2007. Vision-based hand & gesture recognition for humancomputer
interaction. Institute of Computer Science Foundation for Research and Technology,
University of Crete Heraklion, Crete, Hellas, Heraklion.
Zhang, L., Zhen, X., & Shao, L. 2014. Learning object-to-class kernels for scene classification. IEEE
Transactions on Image Processing, 23(8): 3241-3253.
