Hand Gesture Recognition By Jonathan Pritchard Outline Motivation - - PowerPoint PPT Presentation

Hand Gesture Recognition

By Jonathan Pritchard

Outline

• Motivation
• Methods
• Kinematic Models
• Feature Extraction
• Implemented Algorithm
• Results

Motivation

• Virtual Reality – Manipulation of virtual objects with
• ne’s hands.
• Robotics/Telepresence – Precise control of

machinery from remote locations.

• Sign Language – Help the disabled interact with
• computers. ASL can be used as test bed for

different algorithms.

Murthy, G. R. S., & Jadon, R. S. (2009). A review of vision based hand gestures recognition. International Journal of Information Technology and Knowledge Management, 2(2), 405-410.

Kinematic Models

• Simplifying assumptions about hand motion used to

limit the degrees of freedom in the model

• Many model based approaches use a form of

causal tracking to ease computation.

• Filtering used to estimate state (pose, gesture covariance

matrix) based on previous state(s)

• Wire Frame and Silhouette models
• J. M. Rehg and T. Kanade. “Visual tracking of high DOF articulated structures: an application to human hand

tracking”. In J.-O. Eklundh, editor, Proc. 3rd European Conf. on Computer Vision, volume II of Lecture Notes in Computer Science 801, pages 35–46. Springer-Verlag, May 1994.

Kinematic Models: Wire Frame

• J. M. Rehg and T. Kanade. “Visual tracking of high DOF articulated structures: an application to human hand

tracking”. In J.-O. Eklundh, editor, Proc. 3rd European Conf. on Computer Vision, volume II of Lecture Notes in Computer Science 801, pages 35–46. Springer-Verlag, May 1994.

Stereo Vision - Hand Features Identified Pose Estimation 3D Model Filtering

Kinematic Models: Silhouette

Stenger, B., Mendonca, P. & Cipolla, R. “Model-Based 3D Tracking of an Articulated Hand”. In IEEE Conference

• n Computer Vision and Pattern Recognition, (2001) 310–315.

Silhouette matched to gesture

• utline with error minimizing

Kalman Filtering Silhouettes From 3D Model 3D Model (Truncated Quadratics)

Feature Extraction

• “Getting your man without finding his body parts”
• Low level image features used to extract

information without estimating pose

• Not nearly as robust as model based approaches,

but far simpler and faster to compute.

• R. Polana and R. Nelson, “Low level recognition of human motion”, in Proc. of IEEE Workshop on Motion of Non-Rigid

and Articulated Objects, Austin, 1994, pp. 77–82.

Feature Extraction: Number of Fingers

New, J. R., Hasanbelliu, E. and Aguilar, M. “Facilitating User Interaction with Complex Systems via Hand Gesture Recognition.” In Proc. of Southeastern ACM Conf., Savannah, (2003).

Threshold applied to saturation space, only largest connected contour kept Image separated into HSL color spaces Wrist removed, centroid calculated Circle centered at centroid used to calculate number of fingers

Feature Extraction: Fingertips

• J. Raheja, K. Das & A. Chaudhary “Fingertip Detection: A Fast Method with Natural Hand“. International

Journal of Embedded Systems and Computer Engineering , Vol. 3, No. 2, July-December 2011, pp 85-88

Orientation found by comparing oriented histograms Fingertips detected through algorithm looking at top edge of hand, and it’s derivative HSV color space used to obtain binary image

Implemented Algorithm

• Fingertip Detection using MATLAB image processing

toolbox.

• Combination of previous feature extraction

algorithms

• HSV color space used to threshold binary image
• Detect orientation of hand, find outline of top
• Use outline values, derivative filter, and knowledge of hand
• rientation to locate fingertips.

Preliminary Results

Binary Image Result Top Outline Fingertips

Preliminary Results

Binary Image Top Outline Fingertips Result

Preliminary Results

Binary Image Top Outline Fingertips Result

Preliminary Results

Binary Image Top Outline Fingertips Result

Continued Work

• Orientation invariance through wrist detection
• Calculate centroid of binary image
• Reject detected fingertips that are too close to centroid
• Subtract wrist for more accurate centroid

calculation.

Questions?

• M. Randall “Questions”, XKCD, no. 1256 Available: http://imgs.xkcd.com/comics/questions.png