Dataset Card for "cardiffnlp/relentless"

RelEntLess is a new benchmark, in which entity pairs have to be ranked according to how much they satisfy a given graded relation. Essentially, the task is a ranking task where we provide five prototypical examples to each relation. Following brief description of each relation type is used in our baseline in addition to the prototypical examples. Please check our paper " A RelEntLess Benchmark for Modelling Graded Relations between Named Entities " for more detail.

{
    "friend/ally of": "entities that are friends or allies",
    "competitor/rival of": "entities that are competitors or rivals",
    "known for": "examples of what entities are known for",
    "influenced by": "what has influenced different entities",
    "similar to": "examples of entities that are similar"
}

Dataset Summary

Dataset Structure

Data Instances

{
    "pairs": [["Le Corbusier", "purism art"], ["Sean Connery", "Finding Forrester"], ...],
    "scores_all": [[4.0, 5.0, 3.0, 4.0, 5.0, 3.0, 5.0], [4.0, 5.0, 2, 5.0, 5.0, 4.0, 2], ...],
    "scores_mean": [4.142857142857143, 3.857142857142857, 4.857142857142857, ...],
    "relation_type": "known for",
    "ranks": [8.5, 11, 5, 14, 15, 5, 20, 13, 1.5, 18, 10, 1.5, 17, ...],
    "prototypical_examples": [ [ "Russell Crowe", "Gladiator" ], [ "Cadbury", "chocolate" ],...]
}

Citation Information

@misc{ushio2023relentless,
      title={A RelEntLess Benchmark for Modelling Graded Relations between Named Entities}, 
      author={Asahi Ushio and Jose Camacho Collados and Steven Schockaert},
      year={2023},
      eprint={2305.15002},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}