not logged in. (login)

Document Information

Sign Language Spotting with a Threshold Model Based on Conditional Random Fields (2009) Hee-Deok Yang, Stan Sclaroff and Seong-Whan Lee	Download html
Abstract: Sign language spotting is the task of detecting and recognizing signs in a signed utterance, in a set of vocabulary. The difficulty of sign language spotting is that instances of signs vary in both motion and appearance. Moreover, signs appear within a continuous gesture stream, interspersed with transitional movements between signs in a vocabulary and non-sign patterns (which include out-of-vocabulary signs, epentheses, and other movements that do not correspond to signs). In this paper, a novel method for designing threshold models in a conditional random field (CRF) model is proposed, which performs an adaptive threshold for distinguishing between signs in a vocabulary and non-sign patterns. A short-sign detector, a hand appearance-based sign verification method, and a subsign reasoning method are included to further improve sign language spotting accuracy. Experiments demonstrate that our system can spot signs from continuous data with an 87.5% spotting rate and can recognize signs from isolated data with a 94.2% recognition rate, versus 74.5% and 85.4% respectively for CRFs without a threshold model, short-sign detection, subsign reasoning, and hand appearance-based sign verification. Our system can also achieve a 14.7% sign error rate (SER) from continuous data and a 5.8% SER from isolated data, versus 76.2% and 14.6% respectively for conventional CRFs.
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 31, No. 7, pp 1264-1277, 2009.