Input from the clerk can be via a microphone or it can be typed in as text using a clerk screen interface. Both inputs are processed by comparing them with a statistical language model which is able to suggest the nearest matching phrases the system has available for signing. These are displayed for the clerk. The preferred phrase can then be selected and signed for the client. If none of the phrases is appropriate, text alone may be sent.

The clerk’s screen also allows the clerk to receive communication back from the client either in the form of typed text, or via a selection of

responses presented to the client for them to choose from.

Two records of any dialogues are possible. Firstly, in the form of a print out of the dialogue which the client can take away with them, and secondly a file which logs all events that have taken place in the course of an interview. The latter has the advantage that it allows use of the system to be reviewed and improved. In particular, it can highlight frequently-used phrases for which no signed version is available. New signs for such phrases could then be created and added to the system.

The client is provided with an interface which contains an avatar, or virtual human, who can sign what the CIC assistant has said or typed. The client can communicate with the assistant either by typing, or when available, by selecting from a number of labelled clickable button responses.

Signs are created synthetically for the system. The signing has to be accurately described in HamNoSys notation, and this is then transcribed into a form of XML called SiGML which can drive the avatar. Phrases vary from those used in general conversation, such as “Hello,

how can I help you?”, through to specific sentences relating to form filling, for example “You will need a passport sized photo of yourself to send with the form”.

Additional software has been included as part of the Deliverable to demonstrate that VANESSA can be easily modified to translate into other spoken languages as well as sign language. There is also an application that allows new content to be added. If required, such additions may be made in multiple languages.

VANESSA has been developed as a partnership effort between UEA, the RNID and Norfolk County Council. It was part of the EU-funded eSIGN Project aimed at benefiting deaf citizens by giving them access to online information and services in sign language. A working version of VANESSA was installed in the CIC in the Forum in Norwich in August 2004.

VANESSA is a greatly enhanced version of TESSA, a system to help Post Office clerks communicate with their deaf customers. Detailed information about VANESSA is available as a PDF (816K)