Structured Content Generation
There are many web pages where, while the content changes frequently, the format of the page remains the same. An example is the Weather Forecast and a Weather Forecast Creator, developed as part of the ViSiCAST Project. This facilitated the daily generation of weather forecast web pages that included a signed version of the information for deaf people. The signing was done by an Avatar, or virtual human.
This model has been extended as part of the eSIGN Project to produce a generic Structured Content Generation Tool, which is easily modified to update the content of any specific web page. Both Tools are capable of creating output in three different sign languages and text versions, simultaneously.

Weather Forecast Creator
The signed weather forecast web page was based on the regular weather forecasts produced by the Royal Dutch Meteorological Institute (KNMI). While maintaining the content, the original free-form sentences of these weather forecasts were standardised, to make the input suitable for (semi) automatic conversion to sign language. For this, a model consisting of 20 sentence patterns was used, and all possible words and phrases that could fill each slot in the pattern were added. An example of a sentence pattern is “The weather forecast of the KNMI, drawn up on [WEEKDAY][NUMBER][MONTH], and valid till midnight.” This pattern has three slots, for the day of the week, for the month and for the day of the month respectively. In a pattern, the slots are indicated with the category of the word that is to be filled in. Many patterns consist entirely of slots, for example “[TIME] [WEATHER CONDITION]”. The category TIME consists of phrases such as “in the morning”, “in the afternoon”, “during the whole day”, “now and then”, etc., and the category WEATHER CONDITION of phrases as “rain”, “snow”, “sunny”, “fog”, “clouded”, “heavily clouded”, “veil clouds” etc. The model was based on 100 real life weather forecasts from the Netherlands, representing most types of Dutch weather.

With the structure in place, the written language version could be mapped to previously recorded sign language files capable of driving the Avatar. The recorded sign language files were obtained by tracking the motions of an experienced signer using an optical system for tracking facial expression, a magnetic body suit for posture, and data gloves for hand and finger shapes. The signer’s movements were captured at a very high resolution (far higher than is usually used for animation or video games). This is important for the generation of animated sign language, where even the slightest difference in speed, direction or gesture has a bearing upon meaning. Signs of three different sign languages were captured: Sign Language of the Netherlands, German Sign Language and British Sign Language, each with native signers. The signs were post-edited to correct shapes or gestures that may have been performed or recorded inaccurately. Real-time visualisation software was used blend one captured sequence into another, on the resultant web pages creating a smooth signed performance – something that would be unachievable with joined-up clips of video.

                                     

Developments
The Weather Forecast Creator was an obvious starting point for the development a more generic Structured Content Generation Tool. However, it was not realistic, nor practical in the long term, to develop a similar, but nonetheless new, tool for every web page, or even group of web pages, the project or future applications might require. A more general table-driven system was needed.

From this, the idea of a generic program for such content creation evolved. All aspects of the domain would be represented as an XML description file and passed to the core program as input. This would include not only the entire list of contents which could be selected, but also the GUI used to enable the selection. The XML would require some degree of expertise to set up, but considerably less than that needed to write an entirely new program. The core program itself remains the same.

It would also be possible to modify output from the Tool if the need arose, for example, if versions in more than one language were required, or to restructure any text output from the Tool. This could be achieved by the use of XSLT stylesheets.

The fact that the ViSiCAST Weather Forecast Creator was written in C++, restricted its use to Windows and PC machines. For applications to be available to a wider number of users, not least members of the eSIGN project itself, it should be written in a language that is less platform-dependent. Java was chosen to facilitate this.

Further advancements in the Avatar technology meant that motion capture was no longer needed in order to describe the movements the Avatar made. Now, the description of movement was entirely synthetically generated. Further details can be found on the eSIGN project pages

As part of eSIGN, the Tool was used to develop Job Vacancy web pages which included signed versions of the text. As with all web pages which use the Avatar signing, it is necessary to install the Avatar software on the user's computer to view the pages.

 
Faculty Member: 
Professor J R W Glauert
Researchers: 
Ms Judy Tryggvason (Research Associate)
School of Computing Sciences, University of East Anglia, Norwich, NR4 7TJ, UK.
Tel: +44 (0)1603 592847  Fax: +44 (0)1603 593345  E-mail: www-admin@cmp.uea.ac.uk
University of East Anglia (UEA)