SHACHI - Language Resource Metadata Database

Broadcast Resources
The ESTER Corpus is a subset of the ESTER Evaluation Package (catalogue ref. ELRA-E0021), which was produced within the French national project ESTER (Evaluation of Broadcast News enriched transcription systems), as part of the Technolangue programme funded by the French Ministry of Research and New Technologies (MRNT). The ESTER project enabled to carry out a campaign for the evaluation of Broadcast News enriched transcription systems using French data.

This corpus includes the material that was used for the ESTER evaluation campaign, excluding the textual data (available in this catalogue and referenced ELRA-W0015 and ELRA-W0023):

1) About 100 hours of orthographically transcribed news broadcast, including annotations of named entities.
2) The evaluation tools allow to evaluation each task defined above.
3) Two guides and manuals were produced and are provided in the package distributed by ELDA :
o Guide for the annotation of named entities
o Specifications and evaluation protocol

An extra corpus of 1,700 hours of non-transcribed radio broadcast news recordings can also be provided upon request, on hard disk, as an adding to this package at a cost of 100 Euro (plus shipment fee).

A description of the project is available at the following address:
http://www.technolangue.net/article.php3?id_article=60 (in French language)

Written Corpora
Electronic archiving of "Le Monde" articles started on 1 January 1987. Some 200 articles are added every day, and as of October 1997 the database contains more than 500,000 articles, making it the biggest of its kind for all French daily newspapers.

Years 1987 to 2002 are available in an ASCII text format. Years 2003 to 2007 are available in .XML format. Each month consists of some 10 MB of data (circa 120 MB per year).

The number of words available since 2005 is given below:
- 2005: 19 million words
- 2006: 17 million words
- 2007: 21 million words

Years 2008 to 2012 are also available, in an ASCII text format, with no markup.

Data ranging from 1987 until 2012 are available through ELRA.

The Taiyō Corpus is a text database of the periodical Taiyō, which was read by a wide range of readers from the end of 19th to the beginning of the 20th centuries. The individual articles exhibit a variety of writing styles and orthographic characteristics, providing an excellent resource for the study of the development of modern Japanese. The Taiyō Corpus provides texts of 3,409 articles in 60 issues published over the period of 1895-1925, amounting to approximately 15 million characters. All texts are XML formatted, allowing for efficient text retrieval and data handling. Shown below are samples of the original text of Taiyō (top left), the XML document, the XML document formatted with an XSL style sheet (bottom right), and the dialogue of the information retrieval tool provided with the corpus .

Sample data of travel phrases in Japanese, Chinese, Korean, Mongolian, Thai, Vietnamese, Hindi, Persian, Arabic, Turkish, Spanish, English.

Telephone
The OrienTel United Arab Emirates MCA (Modern Colloquial Arabic) database comprises 880 speakers (432 males, 448 females) recorded over the local fixed and mobile telephone network. This database is partitioned into 1 DVD. The speech databases made within the OrienTel project were validated by SPEX, the Netherlands, to assess their compliance with the OrienTel format and content specifications.

Speech samples are stored as sequences of 8-bit 8 kHz A-law. Each prompted utterance is stored in a separate file. Each signal file is accompanied by an ASCII SAM label file which contains the relevant descriptive information.

Each speaker uttered the following items:
• 1 isolated single digit
• 1 sequence of 10 isolated digits
• 4 connected digits : 1 prompt sheet number (6 digits), 1 telephone number (6-15 digits), 1 credit card number (14/16 digits), 1 PIN
• code (6 digits), 1 spontaneous phone number
• 2 currency money amounts
• 1 natural number
• 4 dates : 1 spontaneous (date or year of birth), 1 prompted date, 1 relative or general date expression, 1 prompted date
• phrase (Islamic calendar)
• 2 time phrases : 1 time of day (spontaneous), 1 time phrase (word style)
• 3 spelled words : 1 spontaneous (own forename), 1 city name, 1 real/artificial word for coverage
• 5 directory assistance utterances : 1 spontaneous forename, 1 city of childhood (spontaneous), 1 frequent city name, 1 frequent company name, 1 common forename and surname
• 2 yes/no questions : 1 predominantly yes question, 1 predominantly no question
• 6 application keywords/keyphrases
• 1 word spotting phrase using embedded application words
• 4 phonetically rich words
• 9 phonetically rich sentences
• 3 spontaneous items (for control)

The following age distribution has been obtained: 488 speakers are between 16 and 30, 309 speakers are between 31 and 45, 83 speakers are over 46.

A pronunciation lexicon with a phonemic transcription in SAMPA is also included.

Telephone
The OrienTel United Arab Emirates MSA (Modern Standard Arabic) database comprises 500 speakers (254 males, 246 females) recorded over the local fixed and mobile telephone network. This database is partitioned into 2 DVDs. The speech databases made within the OrienTel project were validated by SPEX, the Netherlands, to assess their compliance with the OrienTel format and content specifications.

Speech samples are stored as sequences of 8-bit 8 kHz A-law. Each prompted utterance is stored in a separate file. Each signal file is accompanied by an ASCII SAM label file which contains the relevant descriptive information.

Each speaker uttered the following items:
• 1 isolated single digit
• 2 sequence of 10 isolated digits
• 7 connected digits : 1 prompt sheet number (6 digits), 6 strings of 4 digits in written format
• 2 currency money amounts
• 2 natural numbers
• 3 dates : 1 prompted date, 1 relative or general date expression, 1 prompted date phrase (Islamic calendar)
• 1 time phrase (word style)
• 2 spelled items : string of 4-letter sequences written out as Aleph, Bae, Jim, etc
• 3 directory assistance utterances : 1 frequent city name, 1 frequent company name, 1 common forename and surname
• 2 yes/no questions : 1 predominantly yes question, 1 predominantly no question
• 6 application keywords/keyphrases
• 1 word spotting phrase using embedded application words
• 4 phonetically rich words
• 9 phonetically rich sentences
• 4 spontaneous items (for control)

The following age distribution has been obtained: 318 speakers are between 15 and 30, 129 speakers are between 31 and 45, 53 speakers are over 46.

A pronunciation lexicon with a phonemic transcription in SAMPA is also included.

Telephone
The Orientel English as spoken in the United Arab Emirates database comprises 535 speakers (266 males, 269 females) recorded over the local fixed and mobile telephone network. This database is partitioned into 2 DVDs. The speech databases made within the OrienTel project were validated by SPEX, the Netherlands, to assess their compliance with the OrienTel format and content specifications.

Speech samples are stored as sequences of 8-bit 8 kHz A-law. Each prompted utterance is stored in a separate file. Each signal file is accompanied by an ASCII SAM label file which contains the relevant descriptive information.

Each speaker uttered the following items:
• 1 isolated single digit
• 1 sequence of 10 isolated digits
• 5 connected digits : 1 prompt sheet number (6 digits), 1 telephone number (6-15 digits), 1 credit card number (14/16 digits), 1 PIN code (6 digits), 1 spontaneous phone number
• 1 currency money amounts
• 2 natural number
• 3 dates : 1 spontaneous (date or year of birth), 1 prompted date, 1 relative or general date expression
• 2 time phrases : 1 time of day (spontaneous), 1 time phrase (word style)
• 3 spelled words : 1 first name, 1 city name, 1 real/artificial word for coverage
• 5 directory assistance utterances : 1 spontaneous forename, 1 city of childhood (spontaneous), 1 frequent city name, 1 frequent company name, 1 common forename and surname
• 4 yes/no questions : 1 predominantly yes question, 1 predominantly no question, 2 fuzzy spontaneous, 1 fuzzy prompted
• 6 application keywords/keyphrases
• 1 word spotting phrase using embedded application words
• 4 phonetically rich words
• 9 phonetically rich sentences
• 4 spontaneous items (for control)

The following age distribution has been obtained: 319 speakers are between 15 and 30, 162 speakers are between 31 and 45, 54 speakers are over 46.

A pronunciation lexicon with a phonemic transcription in SAMPA is also included.

Telephone
The MEDIA speech database for French was produced by ELDA within the French national project MEDIA (Automatic evaluation of man-machine dialogue systems), as part of the Technolangue programme funded by the French Ministry of Research and New Technologies (MRNT).

It contains 1,258 transcribed dialogues from 250 adult speakers. The method chosen for the corpus construction process is that of a ‘Wizard of Oz’ (WoZ) system. This consists of simulating a natural language man-machine dialogue. The scenario was built in the domain of tourism and hotel reservation.

The database is formatted following the SpeechDat conventions and it includes the following items:
• 1,258 recorded sessions for a total of 70 hours of speech. The signals are stored in a stereo wave file format. Each of the two speech channels is recorded at 8 kHz with 16 bit quantization with the least significant byte first (“lohi” or Intel format) as signed integers.
• Manual transcription of each session in XML format. Label files were created with the free transcription tool Transcriber (TRS files).
• Phonetic lexicon containing all the words spoken in the database. Column 1 contains the orthography of the French word. Column 2 shows the frequency of the word. Column 3 contains the pronunciation in SAMPA format. Here is a sample entry of the lexicon:
1) agitée 3 A/ Z i t e
• Documentation and statistics are also provided with the database.

The semantic annotation of the corpus is available in this catalogue and referenced ELRA-E0024 (MEDIA Evaluation Package).

This video corpus consists of nine task-oriented dialogues between two persons. Such annotations as syntactic analysis with anaphoral relations, dialogue acts and prosody have been attached. We have also developed a viewing/annotating tool of the corpus, which is written in Java and platform independent, Since the video data is linked to the annotation data, the user can search a particular part of the corpus by specifying a phrase and/or annotations. Various playback modes are equipped for precise annotating.

Desktop/Microphone
The Japanese Speecon database is divided into 2 sets:
1) The first set comprises the recordings of 556 adult Japanese speakers (268 males, 288 females), recorded over 4 microphone channels in 4 recording environments (office, entertainment, car, public place).
2) The second set comprises the recordings of 51 child Japanese speakers (25 boys, 26 girls), recorded over 4 microphone channels in 1 recording environment (children room).

This database is partitioned into 29 DVDs (first set) and 4 DVDs (second set).
The speech databases made within the Speecon project were validated by SPEX, the Netherlands, to assess their compliance with the Speecon format and content specifications. Each of the four speech channels is recorded at 16 kHz, 16 bit, uncompressed unsigned integers in Intel format (lo-hi byte order). To each signal file corresponds an ASCII SAM label file which contains the relevant descriptive information.

Each speaker uttered the following items (over 290 items for adults and over 210 items for children):
Calibration data:
6 noise recordings
The “silence word” recording
Free spontaneous items (adults only):
5 minutes (session time) of free spontaneous, rich context items (story telling) (an open number of spontaneous topics out of a set of 30 topics)
17 Elicited spontaneous items (adults only):
3 dates, 2 times, 3 proper names, 2 city names, 1 letter sequence, 2 answers to questions, 3 telephone numbers, 1 language
Read speech:
30 phonetically rich sentences uttered by adults and 60 uttered by children
5 phonetically rich words (adults only)
4 isolated digits
1 isolated digit sequence
4 connected digit sequences
1 telephone number
3 natural numbers
1 money amount
2 time phrases (T1 : analogue, T2 : digital)
3 dates (D1 : analogue, D2 : relative and general date, D3 : digital)
3 letter sequences
1 proper name
2 city or street names
2 questions
2 special keyboard characters
1 Web address
1 email address
218 application specific words and phrases per session (adults)
74 toy commands, 14 phone commands and 34 general commands (children)

The following age distribution has been obtained:
Adults: 236 speakers are between 15 and 30, 235 speakers are between 31 and 45, 85 speakers are over 46.
Children: 18 speakers are between 8 and 10, and 33 speakers are between above 11 (up to voice break).

A pronunciation lexicon with a phonemic transcription in SAMPA is also included.

Prices available upon request. Please contact us.