Homepage » Research » Speech, Language and Virtual Humans » Virtual Humans » TESSA and VANESSA » TESSA
The Tessa Project

TESSA, the TExt and Sign Support Assistant, has been developed at UEA to meet the needs of people who are deaf or hard of hearing.

Developed in collaboration with the UK Post Office, the system combines speech recognition technology and state of the art virtual human animation to enable Post Office workers to communicate with deaf customers.

A Post Office Clerk speaks into a microphone which is then recognised by a computer speech recognition system. The speech is converted to British Sign Language and signed by the virtual human for the customer. English text is also displayed for those who do not use sign language.

The movements of the virtual human are copies of those of a native sign language user. Software specially developed for the project captures the signer's hand, mouth and body movements using a variety of electronic sensors. These movements are then stored and used to animate the avatar when required.

TESSA was the result of work at the School of Computing Sciences at UEA, Televirtual, the UK Post Office and the Royal National Institute for Deaf People (RNID). UEA carried out system design, speech, linguistic and interface development, while Televirtual developed the Avatar and animation. The Post Office and the RNID were involved in advisory capacities, and with the assessment of the system.

The RNID, the UK's largest charity representing deaf people, were closely involved in the project. Their advice concerning the quality of the avatar and animation as well as information concerning the needs of deaf people helped to shape the system's development. During the summer of 2001, the system was on display at the Science Museum in London and was simultaneously on trial at the Post Office there.The system was also tested at selected Post Offices in April 2001.

Phrase Recognition and Translation
The project uses commercially available speech-to-text software, which has been optimised to work in a noisy Post Office environment with several different speakers. TESSA enables over 450 of the most frequent Post Office transactions including those involving variable quantities, such as money and days of the week, to be completed more effectively. A Post Office Clerk does not have to use any exact form of words. The system “understands” what he or she is likely to have meant.

New signing lexicons can be produced so that the system may be easily adapted to other areas. Furthermore, TESSA has been developed around the principle that British Sign Language is simply another language. So the technology can also be used to translate into other spoken languages for tourists or ethnic minorities.

Developments
While the system is currently targeted at transactions in the UK Post Office, the ease with with new signing lexicons can be produced means that the system may be easily adapted to other areas.

The second phase of the project moved from simple speech recognition to speech understanding so that the PO clerk was not so constrained in what he or she said to the system. Investigations were also undertaken into the possibility of the system recognising a limited number of signs made by a deaf customer.

TESSA was just part of the EU funded ViSiCAST project aimed at benefiting deaf citizens by allowing them access to information and services in sign language. As part of ViSiCAST, UEA researchers also developed the technology to facilitate the use of sign language on television and the Internet, and this work was continued in the eSIGN Project. The TESSA system has also been enhanced to create VANESSA, a two-way system allowing deaf people to communicate back with clerks or assistants.

TESSA won both a gold medal and the overall IT award at the prestigious British Computer Society's Information Technology Awards.

More information about TESSA is available, including videos and articles in the media. There are also publications and presentations, and images of TESSA and PDF Handouts

Faculty Member: 
Professor S J Cox
Researchers: 
Dr M Lincoln (Senior Research Associate)
Ms Judy Tryggvason (Research Associate)
School of Computing Sciences, University of East Anglia, Norwich, NR4 7TJ, UK.
Tel: +44 (0)1603 592847  Fax: +44 (0)1603 593345  E-mail: www-admin@cmp.uea.ac.uk		 University of East Anglia (UEA)