not logged in. (login)


Gesture Analysis and Recognition


The main goal of this project is to develop computer vision techniques for automatic recognition of gestural communication. The complexity of simultaneous expression of linguistic information on the hands, the face, and the upper body creates special challenges for computer-based recognition. Results of this effort include algorithms for localizing and tracking human hands, estimating hand pose and upper body pose, tracking and classifying head motions, and analysis of eye and facial gestures. Algorithms are also being developed for efficiently spotting, comparing, and recognizing specific gestures of interest in video streams. In addition, pattern analysis algorithms are being investigated that can discover the co-occurrence, overlap, relative timing, frequency, and magnitude of linguistically significant movements of the hands, face, and upper body.

This work is supported by the National Science Foundation under Grant Nos. 0308213, 0329009, and 0202067. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Personnel:
  • Jonathan Alon
  • Vassilis Athitsos
  • Margrit Betke
  • George Kollios
  • Carol Neidle
  • Stan Sclaroff
  • Jared Wickman
  • Hee Deok Yang
  • Quan Yuan


    • Related Publications

    title year

    Quan Yuan , "Learning A Family Of Detectors," Doctoral Dissertation, 2009.
    		 2009

    Hee-Deok Yang, Stan Sclaroff and Seong-Whan Lee , "Sign Language Spotting with a Threshold Model Based on Conditional Random Fields," IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 31, No. 7, pp 1264-1277, 2009.
    		 2009

    Jonathan Alon, Vassilis Athitsos, Quan Yuan and Stan Sclaroff , "A Unified Framework for Gesture Recognition and Spatiotemporal Gesture Segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 31, No. 9, pp 1685-1699, 2009.
    		 2009

    Vassilis Athitsos, Carol Neidle, Stan Sclaroff, Joan Nash, Alexandra Stefan, Quan Yuan and Ashwin Thangali , "The American Sign Language Lexicon Video Dataset," Proc. IEEE Workshop on Computer Vision and Pattern Recognition for Human Communicative Behavior Analysis, 2008.
    		 2008

    Alexandra Stefan,Vassilis Athitsos,Jonathan Alon and Stan Sclaroff , "Translation and Scale Invariant Gesture Recognition in Complex Scenes," Proc. Intl. Conf. on Pervasive Technologies Related to Assistive Environments (PETRA), 2008.
    		 2008

    Philippe Dreuw, Carol Neidle, Vassilis Athitsos, Stan Sclaroff and Hermann Ney , "Benchmark Databases for Video-Based Automatic Sign Language Recognition," Proc. 6th International Conf. on Language Resources and Evaluation (LREC), 2008.
    		 2008

    Mikhail Gorman, Amir Lahav, Elliot Saltzman and Margrit Betke , "A Camera-Based Music-Making Tool for Physical Rehabilitation," Computer Music Journal, Vol. 31, No. 2, pp 39-53, 2007.
    		 2007

    Jonathan Alon , "Spatiotemporal Gesture Segmentation," BU CS Technical Report, No. 2006-024, 2006.
    		 2006

    Margrit Betke, Oleg Gusyatin and Mikhail Urinson , "Symbol design: a user-centered method to design pen-based interfaces and extend the functionality of pointer input devices," Universal Access in the Information Society, Vol. 4, No. 3, pp 223-236, 2006.
    		 2006

    Jonathan Alon, Vassilis Athitsos and Stan Sclaroff , "Online and Offine Character Recognition Using Alignment to Prototypes," Proc. International Conf. on Document Analysis and Retrieval (ICDAR), Vol. 2, pp 839-843, 2005.
    		 2005

    Jonathan Alon, Vassilis Athitsos and Stan Sclaroff , "Fast and Accurate Gesture Spotting via Pruning and Subgesture Reasoning," Proc. IEEE Human Computer Interface Workshop, pp 189-198, 2005.
    		 2005

    Tai-Peng Tian, Rui Li and Stan Sclaroff , "Articulated Pose Estimation in a Learned Smooth Space of Feasible Solutions," Proceedings of IEEE Workshop on Learning in Computer Vision and Pattern Recognition, 2005.
    		 2005

    Panagiotis Papapetrou, George Kollios, Stan Sclaroff and Dimitrios Gunopulos , "Discovering Frequent Arrangements of Temporal Intervals," Proc. IEEE Conf. on Data Mining (ICDM), pp 354-361, 2005.
    		 2005

    M. Gorman, M. Betke, E. Saltzman and A. Lahav , "MusicMaker -- A Camera-based Music Making Tool for Physical Rehabilitation," BU CS Technical Report, No. BUCS-2005-032, 2005.
    		 2005

    Stan Sclaroff, Margrit Betke, George Kollios, Jonathan Alon, Vassilis Athitsos, Rui Li, John Magee and Tai-peng Tian , "Tracking, Analysis, and Recognition of Human Gestures in Video," Proc. International Conf. on Document Analysis and Retrieval (ICDAR), pp 806-810, 2005.
    		 2005

    Jonathan Alon, Vassilis Athitsos, Quan Yuan, and Stan Sclaroff , "Simultaneous Localization and Recognition of Dynamic Hand Gestures," Proc. IEEE Workshop on Motion and Video Computing , pp 254-260, 2005.
    		 2005

    Quan Yuan,Stan Sclaroff and Vassilis Athitsos , "Automatic 2D Hand Tracking in Video Sequences," Proc. IEEE Workshop on Applications of Computer Vision, 2005.
    		 2005

    Tai-Peng Tian and Stan Sclaroff , "Handsignals Recognition From Video Using 3D Motion Capture Data," Proc. IEEE Workshop on Motion and Video Computing , pp 189-194, 2005.
    		 2005

    Ashwin Thangali and Stan Sclaroff , "Periodic Motion Detection and Estimation via Space-Time Sampling," Proc. IEEE Workshop on Motion and Video Computing , pp 176-182, 2005.
    		 2005

    Vassilis Athitsos, Jonathan Alon, Stan Sclaroff and George Kollios , "BoostMap: A Method for Efficient Approximate Similarity Rankings," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vol. II, pp 268-275, 2004.
    		 2004

    Oleg Gusyatin, Mikhail Urinson and Margrit Betke , "A Method to Extend Functionality of Pointer Input Devices," User-Centered Interaction Paradigms for Universal Access in the Information Society, pp 426-439, 2004.
    		 2004

    M. La Cascia, L. Valenti and S. Sclaroff , " Fully automatic, real-time detection of facial gestures from generic video," Proc. Multimedia Signal Processing Workshop, pp 175-178, 2004.
    		 2004

    Stephen Crampton and Margrit Betke , "Counting Fingers in Real Time: A Webcam-Based Human-Computer Interface with Game Applications," Proceedings of the Universal Access in Human-Computer Interaction Conference, pp 1357-1361, 2003.
    		 2003

    Vassilis Athitsos and Stan Sclaroff , "Database Indexing Methods for 3D Hand Pose Estimation," Proc. of the Gesture Workshop, 2003.
    		 2003

    Vassilis Athitsos and Stan Sclaroff , "Estimating 3D Hand Pose From a Cluttered Image ," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 2, pp 432-439, 2003.
    		 2003

    Stephen Crampton and Margrit Betke , "Finger Counter: A Human-Computer Interface," Proceedings of the 7th ERCIM Workshop, pp 195-96, 2002.
    		 2002

    Vassilis Athitsos and Stan Sclaroff , "An Appearance-Based Framework for 3D Hand Shape Classification and Camera Viewpoint Estimation ," Proc. IEEE International Conf. on Automatic Face and Gesture Recogntion (FG), pp 45-50, 2002.
    		 2002

    Ugur Murat Erdem and Stan Sclaroff , "Automatic Detection of Relevant Head Gestures in American Sign Language Communication," Proc. International Conf. on Pattern Recognition (ICPR), Vol. 1, pp 460-463, 2002.
    		 2002

    C. Neidle, S. Sclaroff, and V. Athitsos , "SignStream: A tool for linguistic and computer vision research on visual gestural language data," Behavior Research Methods, Instruments and Computers, Vol. 33, No. 3, pp 311-320, 2001.
    		 2001

    Vassilis Athitsos and Stan Sclaroff , "3D Hand Pose Estimation by Finding Appearance-Based Matches in a Large Database of Training Views," Proc. IEEE Workshop on Cues in Communication, 2001.
    		 2001

    M. La Cascia, S. Sclaroff and V. Athitsos , "Fast, reliable head tracking under varying illumination: An approach based on registration of texture-mapped 3D models," IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 22, No. 4, pp 322-336, 2000.
    		 2000

    Marco La Cascia and Stan Sclaroff , "Fast, reliable head tracking under varying illumination," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vol. I, pp 604-610, 1999.
    		 1999

    Marco La Cascia, John Isidoro and Stan Sclaroff , "Head tracking via robust registration in texture map images," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 508-514, 1998.
    		 1998