SHACHI - Language Resource Metadata Database

This corpus is currently the most widely used test collection for text categorization research. The data was originally collected and labeled by Carnegie Group, Inc. and Reuters, Ltd. in the course of developing the CONSTRUE text categorization system.

*Introduction*

Santa Barbara Corpus of Spoken American English Part IV was produced by Linguistic Data Consortium (LDC) catalog number LDC2005S25 and ISBN 158563-348-8.

Santa Barbara Corpus of Spoken American English Part IV is based on hundreds of recordings of natural speech from all over the United States, representing a wide variety of people of different regional origins, ages, occupations, and ethnic and social backgrounds. It reflects many ways that people use language in their lives: conversation, gossip, arguments, on-the-job talk, card games, city council meetings, sales pitches, classroom lectures, political speeches, bedtime stories, sermons, weddings, and more.

The corpus was collected by: University of California, Santa Barbara Center for the Study of Discourse (Director: John W. Du Bois (UCSB), Authors: John W. Du Bois and Robert Englebretson. Associate Editors: Wallace L. Chafe (UCSB), Charles Meyer (UMass, Boston), and Sandra A. Thompson (UCSB)).

For software and additional data resources, please refer to the following sites: TalkBank, International Corpus of English.

Part I of the Santa Barbara Corpus of Spoken American English is available as LDC2000S85.

Part II of the Santa Barbara Corpus of Spoken American English is available as LDC2003S06.

Part III of the Santa Barbara Corpus of Spoken American English is available as LDC2003S10.

*Data*

The audio data consists of 14 wave format speech files, recorded in two-channel pcm, at 22050Hz. The speech files total 5.75 hours of audio (1.5 GB), representing over 58,000 words and over 6,000 unique words in the transcribed text.

*Samples*

For an example of this corpus, please examine this audio sample and its transcript.

*Note*

The cost of the first 100 copies of this publication (not counting the copies distributed to LDC members) is covered by NSF Grant Number BCS-998009, and therefore free of charge to qualified researchers; a $30 shipping and handling fee applies. After these first 100 copies are distributed, additional copies will be available for the production cost of $200 per DVD-ROM.

*Acknowledgements*

The completion and release of this corpus was facilitated by funding extended by the TalkBank Project. TalkBank is an interdisciplinary research project funded by a five-year grant (BCS-998009, KDI, SBE) from the National Science Foundation to Carnegie Mellon University and the University of Pennsylvania.

*Introduction*

This file contains documentation on the Spanish Gigaword First Edition, Linguistic Data Consortium (LDC) catalog number LDC2006T12 and ISBN 1-58563-393-3.

The Spanish Gigaword Corpus is a comprehensive archive of newswire text data that has been acquired over several years by the Linguistic Data Consortium (LDC) at the University of Pennsylvania. This is the first edition of the Spanish Gigaword Corpus, though some of the data included here has been released previously in other LDC corpora.

The three distinct international sources of Spanish newswire in this edition, and the time spans of collection covered for each, are as follows:

* Agence France-Presse, Spanish Service (afp_spa) May 1994 - Dec 2005
* Associated Press Worldstream, Spanish (apw_spa) Nov 1993 - Dec 2005
* Xinhua News Agency, Spanish Service (xin_spa) Sep 2001 - Dec 2005
The seven-letter codes in the parentheses above include the three-character source name abbreviations and the three-character language code ("spa") separated by an underscore ("_") character. The three-letter language code conforms to LDC's new internal convention based on the new ISO 639-3 standard.

The seven-letter codes are used in both the directory names where the data files are found, and in the prefix that appears at the beginning of every data file name. It is also used (in all UPPER CASE) as the initial portion of the DOC "id" strings that uniquely identify each news story.

*Data*

The overall totals for each source are summarized below. Note that the "Totl-MB" numbers show the amount of data you get when the files are uncompressed (i.e. approximately 5 gigabytes, total); the "Gzip-MB" column shows totals for compressed file sizes as stored on the DVD-ROM; the "K-wrds" numbers are simply the number of whitespace-separated tokens (of all types) after all SGML tags are eliminated.

Source #Files Gzip-MB Totl-MB K-wrds #DOCs AFP_SPA 139 926 2731 393354 1382679 APW_SPA 144 600 1806 263225 886998 XIN_SPA 52 212 648 94459 388561 TOTAL 335 1738 5185 751038 2658238 The following tables present "Text-MB", "K-wrds" and "#DOCS" broken down by source and DOC type; "Text-MB" represents the total number of characters (including whitespace) after SGML tags are eliminated.

Text-MB K-wrds #DOCs type="advis": AFP_SPA 40 15505 40580 APW_SPA 11 6173 11112 XIN_SPA 0 0 0 TOTAL 51 21678 51692 type="multi": AFP_SPA 12 10282 12514 APW_SPA 30 12519 30892 XIN_SPA 32 17773 32463 TOTAL 74 40574 75869 type="other": AFP_SPA 126 28305 126530 APW_SPA 153 39038 153932 XIN_SPA 26 3325 26828 TOTAL 305 70668 307290 AFP_SPA 2166 339271 1202785 APW_SPA 1287 205501 691062 XIN_SPA 463 73360 329270 TOTAL 3916 618132 2223117

*Samples*

For an example of the data in this publicaiton, please examine this sample file.

*Introduction*

This release of Spanish newswire contains data from the following sources:

* Agence France Presse (January 13, 1996--December 13,1998)
* Associated Press Worldstream (December 1, 1995--August 31, 1998)
* El Norte (January 1, 1997--December 31, 1998)
*Data*

The consistent format chosen for release consists of SGML tagging and the ISO-8859-1 (Latin1) 8-bit character set. Our general strategy for SGML tagging is as follows:

All document units (articles) are bounded by the tags DOC and /DOC, and within these units, the text content of each article is bounded by TEXT and /TEXT. Following each DOC tag is a DOCID tag that provides a unique identifying string for that article. Other tags within the DOC unit (but external to TEXT) provide additional information that was receieved with the article (e.g. headline, dateline, byline, keywords, etc), but the inventory and nature of additional information varies from one source to the next (and in some cases, from one article to the next), and this variability is reflected in the SGML tags that are used to preserve the information. Within the TEXT units, tagging is kept to a minimum, typically consisting only of paragraph tags.

*Updates*

There are no updates at this time.

The Spanish News Corpus consists of journalistic text data from one newspaper (El Norte, Mexico) and from the Spanish-language services of three newswire sources: Agence France Presse, Associated Press Worldstream, and Reuters. (The Reuters collection comprises two distinct services: Reuters Spanish Language News Service and Reuters Latin American Business Report).

All text data are stored in a standard compressed form. The fours sets of newswire data (AFP, APWS and two Reuters services) are each organized as one data file per day of collection. The period covered by these collections runs from December 1993 (for APWS and Reuters) or May 1994 (APWS) through December 1995. (The El Norte data, provided to us by INFOSEL Mexico, are arbitrarily grouped into files of about 1 megabyte in size when uncompressed; date information is not available for individual articles, but the general period of the collection is 1993).

The approximate amounts of data per source (when uncompressed) is indicated below (in total megabytes and millions of words of text):

Source
MB
MW

AFP
345
44

APWS
253
33

REUSL
333
41

REULA
233
23

INFOSEL
209
31

The presentation of text data in these collections is modeled on the TIPSTER corpus. Within each data file, SGML tagging is used (1) to mark article boundaries, (2) to delimit the text portion within each article and (3) to label various pieces of information about the article that are external to the text content (e.g. headlines, bylines and so on).

The copyright holders of this text have requested that it be made available to LDC members only. Due to the release date this corpus is available to 1995 and 1996 members. In order to obtain this corpus, current LDC members must submit a signed User Agreement Form.

*Introduction*

This file contains documentation on Speech Controlled Computing, Linguistic Data Consortium (LDC) catalog number LDC2006S30 and ISBN 1-58563-380-1.

The Speech Controlled Computing corpus was designed to support the development of small footprint, embedded ASR applications in the domain of voice control for the home. It consists of the recordings of 125 speakers of American English from four dialect regions, three age groups and two gender groups, pronouncing isolated words. The four primary dialect regions covered by the corpus are North, South, West and Midland as defined by Williams Labov's Atlas of North American English. The three primary age groups covered by the corpus are 18-29, 30-49 and 50+.

The recordings were conducted in a sound-attenuated room at LDC with the AKG C4000B studio condenser microphone. The omni-directional mode of the C4000B was used. Each speaker read a randomized word list consisting of 2,100 words (100 distinct words appearing 21 times each). Speech utterances were digitized and recorded to a DAT, as well as to a hard disk drive via the Townshend DATLINK+ digital audio interface.

Speech utterances were audited as they were recorded, and any utterances detected by the recorder that were not spoken clearly or correctly were re-recorded. This included extraneous clicks, coughs, sighs and breathing that may have corrupted the recorded words. Utterances that were spoken too soft or too loud were also re-recorded.

The digitized utterances were automatically segmented and aligned to the word list. Then each utterance was audited and the segmentation was checked, and corrected if necessary, by an annotator using an auditing and segmenting tool developed by LDC.

Finally, sound files containing individual utterances were generated using the alignment and segmentation information. The sound files for this corpus were created with 100 msec of silent time before and after each utterance. Any files that contained noticeable clipping were automatically removed.

*Samples*

For an example of this corpus, please listen to this audio sample

*Introduction*

This publication contains the Speech in Noisy Environments (SPINE) Evaluation Audio Corpus created for the Department of Defense (DoD) Digital Voice Processing Consortium (DDVPC) by Arcon Corp., and produced by the Linguistic Data Consortium (LDC) catalog number LDC2000S96, ISBN 1-58563-188-4. A companion corpus, Speech in Noisy Environments (SPINE) Audio Transcripts, was also produced by the Linguistic Data Consortium (LDC) catalog number LDC2000T54 and ISBN 1-58563-189-2. These corpora support the 2000 Speech in Noisy Environments (SPINE1) evaluation. There are a total of 120 files, one conversation each, for a rough total of nine hours and 22 minutes (2.2 Gigabytes) of audio data. For an example of a corresponding transcript from the Speech in Noisy Environments (SPINE) Evaluation Transcripts Corpus, please click here. Due to size and format considerations, no example of a speech file is provided.

The 2000 Speech in Noisy Environments Evaluation (SPINE1) is a first attempt to assess the state of the art and practice in speech recognition technology in noisy military environments and to exchange information on innovative speech recognition technology in the context of fully implemented systems that perform realistic tasks. It is intended to be of interest to all university, industrial and commercial speech system developers working on the problem of robust speech recognition. The evaluation gives participants the opportunity to participate in a flexible evaluation, suited to development needs and abilities.

The SPINE1 evaluation focuses on the task of transcribing speech produced in noisy environments with emphasis on noisy military environments. The evaluation is designed to promote research progress in this area, to provide the opportunity for participants to try out new ideas for developing robust speech recognition systems that are of both scientific and practical interest, and to measure the performance of this technology. More information on this evaluation is available at SPINE1.

This work was sponsored in part by National Science Foundation Grant No. IIS-9982201.

*Data*

The evaluation task is to transcribe speech produced in noisy environments. The training and test speech data to be used for this evaluation were generated by ARCON Corp. for the DoD Digital Voice Processing Consortium (DDVPC) under controlled conditions. The speech data consists of conversations between two communicators working on a collaborative, Battleship-like task in which they seek and shoot at targets (ARCON Communicability Exercise, ACE). Participants may talk freely, but the total vocabulary used is fairly limited. Each person is seated in a sound chamber in which a previously recorded military background noise environment is accurately reproduced. The participants use handsets and transmission channels that are resident to the particular environment. The evaluation data includes 20 talker-pairs, with six five-minutes conversations per talker-pair (about 600 minutes total), from a set of four scenarios.

*Samples*

For an example of the speech data in this corpus, please examine this audio sample.

*Updates*

There are no updates at this time.

*Introduction*

This publication contains the Speech in Noisy Environments (SPINE) Evaluation Transcripts, created for the Department of Defense (DoD) Digital Voice Processing Consortium (DDVPC) by Arcon Corp., and produced by the Linguistic Data Consortium (LDC) catalog number LDC2000T54 and ISBN 1-58563-189-2. A companion corpus, Speech in Noisy Environments (SPINE) Evaluation Audio, was also produced by the Linguistic Data Consortium (LDC); catalog number LDC2000S96, ISBN 1-58563-188-4. These corpora support the 2000 Speech in Noisy Environments evaluation. For an example transcript, please click here.

The 2000 Speech in Noisy Environments Evaluation (SPINE1) is a first attempt to assess the state of the art and practice in speech recognition technology in noisy military environments and to exchange information on innovative speech recognition technology in the context of fully implemented systems that perform realistic tasks. It is intended to be of interest to all university, industrial and commercial speech system developers working on the problem of robust speech recognition. The evaluation gives participants the opportunity to participate in a flexible evaluation, suited to development needs and abilities.

This work was sponsored in part by National Science Foundation Grant No. IIS-9982201.

*Data*

The SPINE1 evaluation focuses on the task of transcribing speech produced in noisy environments with the emphasis on speech produced in noisy military environments. The evaluation is designed to promote research progress in this area, to provide the opportunity for participants to try out new ideas for developing robust speech recognition systems that are of both scientific and practical interest, and to measure the performance of this technology. More information on this evaluation is available at SPINE1.

The evaluation task is to transcribe speech produced in noisy environments. The training and test speech data to be used for this evaluation were generated by ARCON Corp. for the DoD Digital Voice Processing Consortium (DDVPC) under controlled conditions. The speech data consists of conversations between two communicators working on a collaborative, Battleship-like task in which they seek and shoot at targets (ARCON Communicability Exercise, ACE). Participants may talk freely, but the total vocabulary used is fairly limited. Each person is seated in a sound chamber in which a previously recorded military background noise environment is accurately reproduced. The participants use handsets and transmission channels that are resident to the particular environment. The evaluation data includes 20 talker-pairs, with six five-minutes conversations per talker-pair (about 600 minutes total), from a set of four scenarios

*Updates*

August 13, 2001: A tagging error was discovered in which several files containing occurrences of the incorrect tag "[{noise}]," were converted to the correct tag, "[/noise]." There were 433 occurrences of this error across all files. Also, a single occurrence of two instances of "[noise/]" on the same line was corrected to "[/noise]" in the second instance. If you previously purchased this corpus and would like to download a corrected copy please contact ldc@ldc.upenn.edu.

*Introduction*

This publication contains the Speech in Noisy Environments (SPINE) Training Audio Corpus created for the Department of Defense (DoD) Digital Voice Processing Consortium (DDVPC) by Arcon Corp., and produced by the Linguistic Data Consortium (LDC) catalog number LDC2000S87, ISBN 1-58563-173-6. A companion corpus, Speech in Noisy Environments (SPINE) Training Transcripts, was also produced by the Linguistic Data Consortium (LDC) catalog number LDC2000T49 and ISBN 1-58563-174-4. These corpora support the 2000 Speech in Noisy Environments (SPINE1) evaluation.

The 2000 Speech in Noisy Environments Evaluation (SPINE1) is a first attempt to assess the state of the art and practice in speech recognition technology in noisy military environments and to exchange information on innovative speech recognition technology in the context of fully implemented systems that perform realistic tasks. It is intended to be of interest to all university, industrial and commercial speech system developers working on the problem of robust speech recognition. The evaluation gives participants the opportunity to participate in a flexible evaluation, suited to development needs and abilities.

The SPINE1 evaluation focuses on the task of transcribing speech produced in noisy environments with emphasis on noisy military environments. The evaluation is designed to promote research progress in this area, to provide the opportunity for participants to try out new ideas for developing robust speech recognition systems that are of both scientific and practical interest, and to measure the performance of this technology. More information on this evaluation is available at SPINE1.

This work was sponsored in part by National Science Foundation Grant No. IIS-9982201.

The transcripts for this release are available as Speech in Noisy Environments (SPINE) Training Transcripts (LDC2000T49)

*Data*

The evaluation task is to transcribe speech produced in noisy environments. The training and test speech data to be used for this evaluation were generated by ARCON Corp. for the DoD Digital Voice Processing Consortium (DDVPC) under controlled conditions. The speech data consists of conversations between two communicators working on a collaborative, Battleship-like task in which they seek and shoot at targets (ARCON Communicability Exercise, ACE). Participants may talk freely, but the total vocabulary used is fairly limited. Each person is seated in a sound chamber in which a previously recorded military background noise environment is accurately reproduced. The participants use handsets and transmission channels that are resident to the particular environment. The training data includes 10 of twenty available talker pairs with 14 five-minute conversations per talker pair (about 720 minutes total), which include four noise scenarios.

*Samples*

For an example of the data contained in this corpus, please listen to this audio sample.

*Updates*

There are no updates at this time.

*Introduction*

This publication contains the Speech in Noisy Environments (SPINE) Training Transcripts, created for the Department of Defense (DoD) Digital Voice Processing Consortium (DDVPC) by Arcon Corp., and produced by the Linguistic Data Consortium (LDC) catalog number LDC2000T49 and ISBN 1-58563-174-4. A companion corpus, Speech in Noisy Environments (SPINE) Training Audio, was also produced by the Linguistic Data Consortium (LDC) catalog number LDC2000S87, ISBN 1-58563-173-6. These corpora support the 2000 Speech in Noisy Environments evaluation. For an example transcript, please click here.

The 2000 Speech in Noisy Environments Evaluation (SPINE1) is a first attempt to assess the state of the art and practice in speech recognition technology in noisy military environments and to exchange information on innovative speech recognition technology in the context of fully implemented systems that perform realistic tasks. It is intended to be of interest to all university, industrial and commercial speech system developers working on the problem of robust speech recognition. The evaluation gives participants the opportunity to participate in a flexible evaluation, suited to development needs and abilities.

The SPINE1 evaluation focuses on the task of transcribing speech produced in noisy environments with the emphasis on speech produced in noisy military environments. The evaluation is designed to promote research progress in this area, to provide the opportunity for participants to try out new ideas for developing robust speech recognition systems that are of both scientific and practical interest, and to measure the performance of this technology. More information on this evaluation is available at SPINE1.

This work was sponsored in part by National Science Foundation Grant No. IIS-9982201.

Corresponding Audio is available as Speech in Noisy Environments (SPINE) Training Audio (LDC2000S87)

*Data*

The evaluation task is to transcribe speech produced in noisy environments. The training and test speech data to be used for this evaluation were generated by ARCON Corp. for the DoD Digital Voice Processing Consortium (DDVPC) under controlled conditions. The speech data consists of conversations between two communicators working on a collaborative, Battleship-like task in which they seek and shoot at targets (ARCON Communicability Exercise, ACE). Participants may talk freely, but the total vocabulary used is fairly limited. Each person is seated in a sound chamber in which a previously recorded military background noise environment is accurately reproduced. The participants use handsets and transmission channels that are resident to the particular environment. The training data includes 10 of 20 available talker pairs with 14 five-minute conversations per talker pair (about 720 minutes total) available, which include four noise scenarios.

*Updates*

There are no updates at this time.