Dataset Card for BookCorpus

Dataset Summary

Books are a rich source of both fine-grained information, how a character, an object or a scene looks like, as well as high-level semantics, what someone is thinking, feeling and how these states evolve through a story.This work aims to align books to their movie releases in order to providerich descriptive explanations for visual content that go semantically farbeyond the captions available in current datasets.

Supported Tasks and Leaderboards

More Information Needed

Languages

More Information Needed

Dataset Structure

Data Instances

plain_text

• Size of downloaded dataset files: 1.18 GB
• Size of the generated dataset: 4.85 GB
• Total amount of disk used: 6.03 GB

An example of 'train' looks as follows.

{
    "text": "But I traded all my life for some lovin' and some gold"
}

Data Fields

The data fields are the same among all splits.

plain_text

• text : a string feature.

Data Splits

name	train
plain_text	74004228

Dataset Creation

Curation Rationale

More Information Needed

Source Data

Initial Data Collection and Normalization

More Information Needed

Who are the source language producers?

More Information Needed

Annotations

Annotation process

More Information Needed

Who are the annotators?

More Information Needed

Personal and Sensitive Information

More Information Needed

Considerations for Using the Data

Social Impact of Dataset

More Information Needed

Discussion of Biases

More Information Needed

Other Known Limitations

More Information Needed

Additional Information

Dataset Curators

More Information Needed

Licensing Information

The books have been crawled from https://www.smashwords.com , see their terms of service for more information.

A data sheet for this dataset has also been created and published in Addressing "Documentation Debt" in Machine Learning Research: A Retrospective Datasheet for BookCorpus .

Citation Information

@InProceedings{Zhu_2015_ICCV,
    title = {Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books},
    author = {Zhu, Yukun and Kiros, Ryan and Zemel, Rich and Salakhutdinov, Ruslan and Urtasun, Raquel and Torralba, Antonio and Fidler, Sanja},
    booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
    month = {December},
    year = {2015}
}

Contributions

Thanks to @lewtun , @richarddwang , @lhoestq , @thomwolf for adding this dataset.