nkjp-pos

Description

NKJP-POS is a part the National Corpus of Polish ( Narodowy Korpus Języka Polskiego ). Its objective is part-of-speech tagging, e.g. nouns, verbs, adjectives, adverbs, etc. During the creation of corpus, texts of were annotated by humans from various sources, covering many domains and genres.

Tasks (input, output and metrics)

Part-of-speech tagging (POS tagging) - tagging words in text with their corresponding part of speech.

Input (' tokens' column): sequence of tokens

Output (' pos_tags' column): sequence of predicted tokens’ classes (35 possible classes, described in detail in the annotation guidelines)

Measurements : F1-score (seqeval)

Example :

Input: ['Zarejestruj', 'się', 'jako', 'bezrobotny', '.']

Input (translated by DeepL): Register as unemployed.

Output: ['impt', 'qub', 'conj', 'subst', 'interp']

Data splits

Class distribution

Citation

@book{przepiorkowski_narodowy_2012,
title = {Narodowy korpus języka polskiego},
isbn = {978-83-01-16700-4},
language = {pl},
publisher = {Wydawnictwo Naukowe PWN},
editor = {Przepiórkowski, Adam and Bańko, Mirosław and Górski, Rafał L. and Lewandowska-Tomaszczyk, Barbara},
year = {2012}
}

License

GNU GPL v.3

Links

HuggingFace

Source

Paper

Examples

Loading

from pprint import pprint

from datasets import load_dataset

dataset = load_dataset("clarin-pl/nkjp-pos")
pprint(dataset['train'][5000])

# {'id': '130-2-900005_morph_49.49-s',
#  'pos_tags': [16, 4, 3, 30, 12, 18, 3, 16, 14, 6, 14, 26, 1, 30, 12],
#  'tokens': ['Najwyraźniej',
#             'źle',
#             'ocenił',
#             'odległość',
#             ',',
#             'bo',
#             'zderzył',
#             'się',
#             'z',
#             'jadącą',
#             'z',
#             'naprzeciwka',
#             'ciężarową',
#             'scanią',
#             '.']}

Evaluation

import random
from pprint import pprint

from datasets import load_dataset, load_metric

dataset = load_dataset("clarin-pl/nkjp-pos")
references = dataset["test"]["pos_tags"]

# generate random predictions
predictions = [
    [
        random.randrange(dataset["train"].features["pos_tags"].feature.num_classes)
        for _ in range(len(labels))
    ]
    for labels in references
]

# transform to original names of labels
references_named = [
    [dataset["train"].features["pos_tags"].feature.names[label] for label in labels]
    for labels in references
]
predictions_named = [
    [dataset["train"].features["pos_tags"].feature.names[label] for label in labels]
    for labels in predictions
]

# transform to BILOU scheme
references_named = [
    [f"U-{label}" if label != "O" else label for label in labels]
    for labels in references_named
]
predictions_named = [
    [f"U-{label}" if label != "O" else label for label in labels]
    for labels in predictions_named
]

# utilise seqeval to evaluate
seqeval = load_metric("seqeval")
seqeval_score = seqeval.compute(
    predictions=predictions_named,
    references=references_named,
    scheme="BILOU",
    mode="strict",
)

pprint(seqeval_score, depth=1)

# {'adj': {...},
#  'adja': {...},
#  'adjc': {...},
#  'adjp': {...},
#  'adv': {...},
#  'aglt': {...},
#  'bedzie': {...},
#  'brev': {...},
#  'burk': {...},
#  'comp': {...},
#  'conj': {...},
#  'depr': {...},
#  'fin': {...},
#  'ger': {...},
#  'imps': {...},
#  'impt': {...},
#  'inf': {...},
#  'interj': {...},
#  'interp': {...},
#  'num': {...},
#  'numcol': {...},
#  'overall_accuracy': 0.027855061488566583,
#  'overall_f1': 0.027855061488566583,
#  'overall_precision': 0.027855061488566583,
#  'overall_recall': 0.027855061488566583,
#  'pact': {...},
#  'pant': {...},
#  'pcon': {...},
#  'ppas': {...},
#  'ppron12': {...},
#  'ppron3': {...},
#  'praet': {...},
#  'pred': {...},
#  'prep': {...},
#  'qub': {...},
#  'siebie': {...},
#  'subst': {...},
#  'winien': {...},
#  'xxx': {...}}