Faculty of Science

Application Deadline 21^st June 2013. This studentship is available for a 1^st October 2013 start.

Supervisory Team Primary: Professor Stephen Cox

The Project

Audio-visual speech recognition (AVSR) has the potential to improve the quality of speech recognition when the speech is uttered in a noisy environment, and is timely now that cameras are ubiquitous on mobile devices.  Our Lab has been researching lip-reading for several years, and we have a good understanding of techniques that are useful for extracting speech information from visual signals: so far, these have not been integrated into an audio-visual speech recognition system. 

Research in AVSR has tended to concentrate on how to combine the audio and video feature streams or how to use the outputs from separate audio and video recognisers.  Little attention has been paid to the fact that much of the information in the audio signal is actually missing in the visual signal, and so it makes little sense to apply traditional audio recognition techniques to a visual signal.  Recent work from our Lab suggests that a better strategy for using the visual signal in speech recognition would be to make use of “islands of certainty” or “landmarks” in the signal where the lips can provide useful complementary information to the audio signal.  It appears that lip readers make use of such speech cues e.g. lip closures, lip-rounding, as well as visual patterns for high frequency words such as “yes”, “okay”), etc. This approach has similarities to recent work in speech recognition which is also concerned itself with the detection of reliable landmarks in the speech signal and utilizing these with advanced machine-learning techniques to decode the signal.

References

Landmark-based speech recognition: report of the 2004 Johns Hopkins Summer Workshop.  Mark Haswgawa-Johnson et al. 

www.ncbi.nlm.nih.gov/pmc/articles/PMC2638080/ Accessed October 31^st 2012.

Entry Requirements

A first or upper second class degree (or international equivalent) in Computing Science, Mathematics, or any Science subject that has included study of programming.

Funding

This studentship will cover the cost of fees at the UK/EU rate and provide a stipend at the standard RCUK rate (currently £13,726). Non-EU students are welcome to apply but due to funding restrictions will be required to pay the difference between UK/EU and International fees.

Making your Application:

Please apply via the University’s online application system.

To discuss the application process please contact the Admissions Office, email pgr.enquiries.admiss@uea.ac.uk or telephone +44 (0)1603 591709.