? + ? dbmdz BERT model

In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State Library open sources French Europeana BERT models ?

French Europeana BERT

We extracted all French texts using the language metadata attribute from the Europeana corpus.

The resulting corpus has a size of 63GB and consists of 11,052,528,456 tokens.

Based on the metadata information, texts from the 18th - 20th century are mainly included in the training corpus.

Detailed information about the data and pretraining steps can be found in this repository .

Model weights

BERT model weights for PyTorch and TensorFlow are available.

• French Europeana BERT: dbmdz/bert-base-french-europeana-cased - model hub page

Results

For results on Historic NER, please refer to this repository .

Usage

With Transformers >= 2.3 our French Europeana BERT model can be loaded like:

from transformers import AutoModel, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("dbmdz/bert-base-french-europeana-cased")
model = AutoModel.from_pretrained("dbmdz/bert-base-french-europeana-cased")

Huggingface model hub

All models are available on the Huggingface model hub .

Contact (Bugs, Feedback, Contribution and more)

For questions about our BERT model just open an issue here ?

Acknowledgments

Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). Thanks for providing access to the TFRC ❤️

Thanks to the generous support from the Hugging Face team, it is possible to download our model from their S3 storage ?