Dataset Card for "scifact"

Dataset Summary

SciFact, a dataset of 1.4K expert-written scientific claims paired with evidence-containing abstracts, and annotated with labels and rationales.

Supported Tasks and Leaderboards

More Information Needed

Languages

More Information Needed

Dataset Structure

Data Instances

claims

• Size of downloaded dataset files: 2.72 MB
• Size of the generated dataset: 0.25 MB
• Total amount of disk used: 2.97 MB

An example of 'validation' looks as follows.

{
    "cited_doc_ids": [14717500],
    "claim": "1,000 genomes project enables mapping of genetic sequence variation consisting of rare variants with larger penetrance effects than common variants.",
    "evidence_doc_id": "14717500",
    "evidence_label": "SUPPORT",
    "evidence_sentences": [2, 5],
    "id": 3
}

corpus

• Size of downloaded dataset files: 2.72 MB
• Size of the generated dataset: 7.63 MB
• Total amount of disk used: 10.35 MB

An example of 'train' looks as follows.

This example was too long and was cropped:

{
    "abstract": "[\"Alterations of the architecture of cerebral white matter in the developing human brain can affect cortical development and res...",
    "doc_id": 4983,
    "structured": false,
    "title": "Microstructural development of human newborn cerebral white matter assessed in vivo by diffusion tensor magnetic resonance imaging."
}

Data Fields

The data fields are the same among all splits.

claims

• id : a int32 feature.
• claim : a string feature.
• evidence_doc_id : a string feature.
• evidence_label : a string feature.
• evidence_sentences : a list of int32 features.
• cited_doc_ids : a list of int32 features.

corpus

• doc_id : a int32 feature.
• title : a string feature.
• abstract : a list of string features.
• structured : a bool feature.

Data Splits

claims

train	validation	test
claims	1261	450	300

corpus

train
corpus	5183

Dataset Creation

Curation Rationale

More Information Needed

Source Data

Initial Data Collection and Normalization

More Information Needed

Who are the source language producers?

More Information Needed

Annotations

Annotation process

More Information Needed

Who are the annotators?

More Information Needed

Personal and Sensitive Information

More Information Needed

Considerations for Using the Data

Social Impact of Dataset

More Information Needed

Discussion of Biases

More Information Needed

Other Known Limitations

More Information Needed

Additional Information

Dataset Curators

More Information Needed

Licensing Information

https://github.com/allenai/scifact/blob/master/LICENSE.md

The SciFact dataset is released under the CC BY-NC 2.0 . By using the SciFact data, you are agreeing to its usage terms.

Citation Information

@inproceedings{wadden-etal-2020-fact,
    title = "Fact or Fiction: Verifying Scientific Claims",
    author = "Wadden, David  and
      Lin, Shanchuan  and
      Lo, Kyle  and
      Wang, Lucy Lu  and
      van Zuylen, Madeleine  and
      Cohan, Arman  and
      Hajishirzi, Hannaneh",
    booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2020.emnlp-main.609",
    doi = "10.18653/v1/2020.emnlp-main.609",
    pages = "7534--7550",
}

Contributions

Thanks to @thomwolf , @lhoestq , @dwadden , @patrickvonplaten , @mariamabarham , @lewtun for adding this dataset.