Dataset Card for "nli_tr"

Dataset Summary

The Natural Language Inference in Turkish (NLI-TR) is a set of two large scale datasets that were obtained by translating the foundational NLI corpora (SNLI and MNLI) using Amazon Translate.

Supported Tasks and Leaderboards

More Information Needed

Languages

More Information Needed

Dataset Structure

Data Instances

multinli_tr

• Size of downloaded dataset files: 75.52 MB
• Size of the generated dataset: 79.47 MB
• Total amount of disk used: 154.99 MB

An example of 'validation_matched' looks as follows.

This example was too long and was cropped:

{
    "hypothesis": "Mrinal Sen'in çalışmalarının çoğu Avrupa koleksiyonlarında bulunabilir.",
    "idx": 7,
    "label": 1,
    "premise": "\"Kalküta, sanatsal yaratıcılığa dair herhangi bir iddiaya sahip olan tek diğer üretim merkezi gibi görünüyor, ama ironik bir şek..."
}

snli_tr

• Size of downloaded dataset files: 40.33 MB
• Size of the generated dataset: 73.89 MB
• Total amount of disk used: 114.22 MB

An example of 'train' looks as follows.

{
    "hypothesis": "Yaşlı bir adam, kızının işten çıkmasını bekçiyken suyunu içer.",
    "idx": 9,
    "label": 1,
    "premise": "Parlak renkli gömlek çalışanları arka planda gülümseme iken yaşlı bir adam bir kahve dükkanında küçük bir masada onun portakal suyu ile oturur."
}

Data Fields

The data fields are the same among all splits.

multinli_tr

• idx : a int32 feature.
• premise : a string feature.
• hypothesis : a string feature.
• label : a classification label, with possible values including entailment (0), neutral (1), contradiction (2).

snli_tr

• idx : a int32 feature.
• premise : a string feature.
• hypothesis : a string feature.
• label : a classification label, with possible values including entailment (0), neutral (1), contradiction (2).

Data Splits

multinli_tr

train	validation_matched	validation_mismatched
multinli_tr	392702	10000	10000

snli_tr

train	validation	test
snli_tr	550152	10000	10000

Dataset Creation

Curation Rationale

More Information Needed

Source Data

Initial Data Collection and Normalization

More Information Needed

Who are the source language producers?

More Information Needed

Annotations

Annotation process

More Information Needed

Who are the annotators?

More Information Needed

Personal and Sensitive Information

More Information Needed

Considerations for Using the Data

Social Impact of Dataset

More Information Needed

Discussion of Biases

More Information Needed

Other Known Limitations

More Information Needed

Additional Information

Dataset Curators

More Information Needed

Licensing Information

More Information Needed

Citation Information

@inproceedings{budur-etal-2020-data,
    title = "Data and Representation for Turkish Natural Language Inference",
    author = "Budur, Emrah and
      "{O}zçelik, Rıza and
      G"{u}ng"{o}r, Tunga",
    booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    abstract = "Large annotated datasets in NLP are overwhelmingly in English. This is an obstacle to progress in other languages. Unfortunately, obtaining new annotated resources for each task in each language would be prohibitively expensive. At the same time, commercial machine translation systems are now robust. Can we leverage these systems to translate English-language datasets automatically? In this paper, we offer a positive response for natural language inference (NLI) in Turkish. We translated two large English NLI datasets into Turkish and had a team of experts validate their translation quality and fidelity to the original labels. Using these datasets, we address core issues of representation for Turkish NLI. We find that in-language embeddings are essential and that morphological parsing can be avoided where the training set is large. Finally, we show that models trained on our machine-translated datasets are successful on human-translated evaluation sets. We share all code, models, and data publicly.",
}

Contributions

Thanks to @e-budur for adding this dataset.