SHACHI - Language Resource Metadata Database

The corpus contains Chinese speech data from 1000 all male speakers. Each speaker uttered two segments of natural speech without disguising their voice. Each segment was recorded twice; on a GSM mobile phone and on a landline telephone.

The corpus contains Chinese speech data from 1000 all male speakers. Each speaker uttered two segments of speech; 1) a brief personal introduction and 2) 40 Chinese sentences. Each segment was recorded twice; on a GSM mobile phone and on a landline telephone. Only the first 70 seconds of each utterance were recorded.

The corpus contains Chinese speech data from 3000 all male speakers. Each speaker uttered two segments of speech; 1) a brief personal introduction and 2) 40 Chinese sentences. Each segment was recorded twice; on a GSM mobile phone and on a landline telephone. Only the first 70 seconds of each utterance were recorded.

The corpus contains Chinese speech data from 6000 all male speakers. Each speaker uttered two segments of speech; 1) a brief personal introduction and 2) 40 Chinese sentences. Each segment was recorded twice; on a GSM mobile phone and on a landline telephone. Only the first 70 seconds of each utterance were recorded.

The corpus contains Chinese speech data from 10,000 all male speakers. Each speaker uttered two segments of speech; 1) a brief personal introduction and 2) 40 Chinese sentences. Each segment was recorded twice; on a GSM mobile phone and on a landline telephone. Only the first 70 seconds of each utterance were recorded.

The corpus contains Chinese speech data from 50 speakers. Each speaker utters a total of 27 segments of speech, with each segment recorded on one of 27 handsets (21 mobile and 6 landline phones). The reading texts for these 27 segments for each speaker are different.

The corpus contains Chinese speech data from 100 speakers. Each speaker utters a total of 36 segments of speech, with each segment recorded on one of 36 handsets (27 mobile and 9 landline phones). The reading texts for these 36 segments for each speaker are different.

The corpus contains Chinese speech data from 50 speakers. The data consist of affective reading of paragraphs and commands, where affects are listed as neutral, happiness, fear, anger and sadness.

*Introduction*

The Penn Discourse Treebank (PDTB) is an NSF funded project at the University of Pennsylvania. The goal of the project is to annotate the 1 million word Wall Street Journal corpus in Treebank-2 (LDC95T7) with discourse relations holding between the eventualities and propositions mentioned in text, which serve as the arguments to the relation. Discourse relations are assumed to have exactly two arguments. PDTB, version 2.0. is a continuation of PDTB, version 1.0. (made available freely in 2006 but no longer available). Following a lexically grounded approach to annotation, the PDTB annotates relations realized explicitly by Explicit connectives drawn from syntactically well-defined classes, as well as relations between adjacent sentences when no Explicit connective appears to relate the two. Arguments of relations are annotated in each case. For Explicit connectives, arguments are unconstrained in terms of their distance from the connective and can be found anywhere in the text. Between adjacent sentences where no Explicit connective appears, four scenarios hold: (a) the sentences may be related by a discourse relation that has no realization in the second sentence, in which case a connective (called an Implicit connective) is provided to express the inferred relation (b) the sentences may be related by a discourse relation that is realized by some alternative non-connective expression, in which case these alternative lexicalizations are annotated as the carriers of the relation (labelled as AltLex) (c) the sentences may be related not by a discourse relation, but merely by an entity-based coherence relation, in which case the presence of such a relation is labelled (as EntRel) and (d) the sentences may not be related at all, in which case they are labelled as such (NoRel).

In addition to the argument structure of relations, the PDTB provides (a) sense annotations for each discourse relation while also capturing the polysemy of connectives, and (b) attribution annotations of relations and each of their arguments, with each instance of attribution providing the corresponding text span along with four features to capture the semantic contribution of the attribution. Both sense and attribution annotations are provided for Explicit, Implicit, and AltLex relations, but not for EntRel and NoRel.

The lexically grounded approach in the PDTB exposes a clearly defined level of discourse structure which will support the extraction of a range of inferences associated with discourse connectives.

To date, the PDTB group has carried out various experiments on the corpus, particularly examining the following issues:

* alignment between syntax and discourse, particularly with regards to attribution
* sense disambiguation of discourse connectives
* complexity of dependencies in discourse

The annotations in Penn Discourse Treebank Version 2.0 are linked to the Penn Treebank.

The PDTB group will continue to explore these issues and to focus on more extended projects such as discourse parsing, automatic summarization, and natural language generation. Further work will also explore foundational issues in discourse.

PDTB, version 2.0. annotates 40600 discourse relations, distributed into the following five types:

* 18459 Explicit Relations
* 16053 Implicit Relations
* 624 Alternative Lexicalizations
* 5210 Entity Relations
* 254 No Relations

*Samples*

For an example of the data in this corpus, please review the sample below:

________________________________________________________ ____Explicit____ 544..551 4,2 #### Text #### however ############## #### Features #### Wr, Comm, Null, Null however, Comparison.Contrast ____Sup1____ 374..515 23 #### Text #### Its index inched up to 47.6% in October from 46% in September. Any reading below 50% suggests the manufacturing sector is generally declining ############## ____Arg1____ 288..372 1,3,1,1,1,1 #### Text #### that the manufacturing economy contracted in October for the sixth consecutive month ############## #### Features #### Ot, Comm, Null, Null 260..287 1,3,1,01,3,1,1,01,3,1,1,1,0 #### Text #### its latest survey indicated ############## ____Arg2____ 563..624 4,5,1 #### Text #### that orders turned up in October after four months of decline ############## #### Features #### Ot, Comm, Null, Null 519..542553..562 4,04,14,34,44,5,04,6 #### Text #### The purchasing managers also said ############## ________________________________________________________

*Updates*

Additional information, updates, bug fixes may be available in the LDC catalog entry for this corpus at LDC2008T05.

As of December 12, 2012 the developers of the Penn Discourse Treebank Version 2.0 LDC2008T05 have updated this release to add metadata to the Wall Street Journal (WSJ) news stories in the corpus. The goal is to aid understanding PDTB files as texts and to support distinguishing texts from different genres within the WSJ.

The metadata includes of the below fields. Consult this metadata documentation for more information.

* DD: the date the article appeared in the WSJ
* AN: unique identifier for the article
* HL: the column name (for regular features such as Whos News, Marketing & Media, Technology), its headline and by-line
* SO: the source of the article
* IN: manually-assigned codes or keywords for the article
* CO: manually-assigned codes for companies or other organizations
* DATELINE: normally the location where the article was filed, but sometimes has very unexpected contents
* GV: Branch of Government or Government Agency mentioned in the article
* SBREAKS: the byte position of section breaks present in the file
* ARTICLEBREAK: separates files that contain more than one article

This update may be of value to discourse researchers. The meta-data can, for example, enable the texts to be distinguished by genre (news reports, editorials, etc. [Webber, 2009]) or by topic [Petrenz and Webber, 2011]. These can then be used, for example, in text segmentation and text summarization, or in testing hypotheses about domain adaptation [Plank and van Noord, 2011]. The data, on the other hand, can allow researchers to distinguish separate texts within a single file (e.g. the four separate letters to the editor in file wsj_0105) and thereby avoid, for example, attempting to produce one summary for the entire file.

As of February, 2017, 2,499 "raw" wsj files were added from Treebank-2 (LDC95T7), to make PDTB more useful.

All downloads after these date will contain the complete, updated corpus.

*Recognizing Textual Entailment Data*

These data have been used to run the textual entailment experiments described in: Sara Tonelli and Elena Cabrio Hunting for Entailing Pairs in the Penn Discourse Treebank, in Proceedings of Coling 2012, Mumbay, India. The files contain Text - Hypothesis pairs in the standard RTE xml format (for more details, see http://www.nist.gov/tac/2011/RTE/), which have been manually annotated as entailing or not entailing. All sentence pairs have been extracted from the Penn Discourse Treebank and are therefore connected by a discourse relation label.

For more information, consult the readme. The data are not included in the general release of Penn Discourse Treebank Version 2.0, but are freely available for download.

Written English news stories, edited so as to target a Canadian teenage audience. Text, questions and answers have been marked up automatically with 13 layers of linguistic knowledge. cdc4kids is a richly annotated subset of the 249-document MITRE Canadian Broadcasting Corpus (CBC) with 8-12 questions per text and their corresponding answers.