TESSA, the TExt and Sign Support Assistant, has been developed at UEA to meet the needs of people who are deaf or hard of hearing.
Developed in collaboration with the UK Post Office, the system combines speech recognition technology and state of the art virtual human animation to enable Post Office workers to communicate with deaf customers.
A Post Office Clerk speaks into a microphone which is then recognised by a computer speech recognition system. The speech is converted to British Sign Language and signed by the virtual human for the customer. English text is also displayed for those who do not use sign language.
The movements of the virtual human are copies of those of a native sign language user. Software specially developed for the project captures the signer's hand, mouth and body movements using a variety of electronic sensors. These movements are then stored and used to animate the avatar when required.
TESSA was the result of work at the School of Computing Sciences at UEA, Televirtual, the UK Post Office and the Royal National Institute for Deaf People (RNID). UEA carried out system design, speech, linguistic and interface development, while Televirtual developed the Avatar and animation. The Post Office and the RNID were involved in advisory capacities, and with the assessment of the system.