
<div class="course-tip course-tip-orange bg-gradient-to-br dark:bg-gradient-to-r before:border-orange-500 dark:before:border-orange-800 from-orange-50 dark:from-gray-900 to-white dark:to-gray-950 border border-orange-50 text-orange-700 dark:text-gray-400">
  <p><b>Update:</b> 
  This model has been moved to <a href="https://huggingface.co/linhd-postdata/alberti-bert-base-multilingual-cased">linhd-postdata/alberti-bert-base-multilingual-cased</a>, where it will be maintained and updated.
  </p>
</div>

# ALBERTI

ALBERTI is a set of two BERT-based multilingual model for poetry. One for verses and another one for stanzas. This model has been further trained with the PULPO corpus for verses using [Flax](https://github.com/google/flax), including training scripts.

This is part of the
[Flax/Jax Community Week](https://discuss.huggingface.co/t/open-to-the-community-community-week-using-jax-flax-for-nlp-cv/7104), organised by [HuggingFace](https://huggingface.co/) and TPU usage sponsored by Google.

## PULPO

PULPO, the Prodigious Unannotated Literary Poetry Corpus, is a set of multilingual corpora of verses and stanzas with over 95M words.

The following corpora has been downloaded using the [Averell](https://github.com/linhd-postdata/averell/) tool, developed by the [POSTDATA](https://postdata.linhd.uned.es/) team:

### Spanish
- [Disco v3](https://github.com/pruizf/disco)
- [Corpus of Spanish Golden-Age Sonnets](https://github.com/bncolorado/CorpusSonetosSigloDeOro)
- [Corpus general de poesía lírica castellana del Siglo de Oro](https://github.com/bncolorado/CorpusGeneralPoesiaLiricaCastellanaDelSigloDeOro)
- [Gongocorpus](https://github.com/linhd-postdata/gongocorpus) - [source](http://obvil.sorbonne-universite.site/corpus/gongora/gongora_obra-poetica)
### English
- [Eighteenth-Century Poetry Archive (ECPA)](https://github.com/alhuber1502/ECPA)
- [For better for verse](https://github.com/waynegraham/for_better_for_verse)
### French
- [Métrique en Ligne](https://crisco2.unicaen.fr/verlaine/index.php?navigation=accueil) - [source](https://github.com/linhd-postdata/metrique-en-ligne)
### Italian
- [Biblioteca italiana](https://github.com/linhd-postdata/biblioteca_italiana) - [source](http://www.bibliotecaitaliana.it/)
### Czech
- [Corpus of Czech Verse](https://github.com/versotym/corpusCzechVerse)
### Portuguese
- [Stichotheque](https://gitlab.com/stichotheque/stichotheque-pt)

Also, we obtained the following corpora from these sources:
### Spanish 
- [Poesi.as](https://github.com/linhd-postdata/poesi.as) - [source](http://www.poesi.as/)
### English
- [A Gutenberg Poetry Corpus](https://github.com/aparrish/gutenberg-poetry-corpus)
### Arabic
- [Arabic Poetry dataset](https://www.kaggle.com/ahmedabelal/arabic-poetry)
### Chinese
- [THU Chinese Classical Poetry Corpus](https://github.com/THUNLP-AIPoet/Datasets/tree/master/CCPC)
### Finnish
- [SKVR](https://github.com/sks190/SKVR)
### German
- [TextGrid Poetry Corpus](https://github.com/linhd-postdata/textgrid-poetry) - [source](https://textgrid.de/en/digitale-bibliothek)
- [German Rhyme Corpus](https://github.com/tnhaider/german-rhyme-corpus)
### Hungarian
- [verskorpusz](https://github.com/ELTE-DH/verskorpusz)
### Portuguese
- [Poems in Portuguese](https://www.kaggle.com/oliveirasp6/poems-in-portuguese)
### Russian
- [19 000 Russian poems](https://www.kaggle.com/grafstor/19-000-russian-poems)

## Team members

- Álvaro Pérez ([alvp](https://huggingface.co/alvp))
- Javier de la Rosa ([versae](https://huggingface.co/versae))
- Aitor Díaz ([aitordiaz](https://huggingface.co/aitordiaz))
- Elena González-Blanco
- Salvador Ros ([salva](https://huggingface.co/salva))

## Useful links

- [Community Week timeline](https://discuss.huggingface.co/t/open-to-the-community-community-week-using-jax-flax-for-nlp-cv/7104#summary-timeline-calendar-6)
- [Community Week README](https://github.com/huggingface/transformers/blob/master/examples/research_projects/jax-projects/README.md)
- [Community Week thread](https://discuss.huggingface.co/t/bertin-pretrain-roberta-large-from-scratch-in-spanish/7125)
- [Community Week channel](https://discord.com/channels/858019234139602994/859113060068229190)
- [Masked Language Modelling example scripts](https://github.com/huggingface/transformers/tree/master/examples/flax/language-modeling)
- [Model Repository](https://huggingface.co/flax-community/alberti-bert-base-multilingual-cased/)

## Acknowledgments

This project would not have been possible without the infrastructure and resources provided by HuggingFace and Google Cloud. Moreover, we want to thank POSTDATA Project (ERC-StG-679528) and the Computational Literary Studies Infrastructure (CLS INFRA No. 101004984) of the European Union's Horizon 2020 research and innovation programme for their support and time allowance.
{"language": "es", "license": "cc-by-4.0", "tags": ["multilingual", "bert"], "pipeline_tag": "fill-mask", "widget": [{"text": "\u00bfQu\u00e9 es la vida? Un [MASK]."}]}
transformers pytorch jax joblib safetensors bert fill-mask multilingual es license:cc-by-4.0 autotrain_compatible endpoints_compatible has_space region:us
TAGS
#transformers #pytorch #jax #joblib #safetensors #bert #fill-mask #multilingual #es #license-cc-by-4.0 #autotrain_compatible #endpoints_compatible #has_space #region-us 


<div class="course-tip course-tip-orange bg-gradient-to-br dark:bg-gradient-to-r before:border-orange-500 dark:before:border-orange-800 from-orange-50 dark:from-gray-900 to-white dark:to-gray-950 border border-orange-50 text-orange-700 dark:text-gray-400">
  <p><b>Update:</b> 
  This model has been moved to <a href="URL where it will be maintained and updated.
  </p>
</div>

# ALBERTI

ALBERTI is a set of two BERT-based multilingual model for poetry. One for verses and another one for stanzas. This model has been further trained with the PULPO corpus for verses using Flax, including training scripts.

This is part of the
Flax/Jax Community Week, organised by HuggingFace and TPU usage sponsored by Google.

## PULPO

PULPO, the Prodigious Unannotated Literary Poetry Corpus, is a set of multilingual corpora of verses and stanzas with over 95M words.

The following corpora has been downloaded using the Averell tool, developed by the POSTDATA team:

### Spanish
- Disco v3
- Corpus of Spanish Golden-Age Sonnets
- Corpus general de poesía lírica castellana del Siglo de Oro
- Gongocorpus - source
### English
- Eighteenth-Century Poetry Archive (ECPA)
- For better for verse
### French
- Métrique en Ligne - source
### Italian
- Biblioteca italiana - source
### Czech
- Corpus of Czech Verse
### Portuguese
- Stichotheque

Also, we obtained the following corpora from these sources:
### Spanish 
- URL - source
### English
- A Gutenberg Poetry Corpus
### Arabic
- Arabic Poetry dataset
### Chinese
- THU Chinese Classical Poetry Corpus
### Finnish
- SKVR
### German
- TextGrid Poetry Corpus - source
- German Rhyme Corpus
### Hungarian
- verskorpusz
### Portuguese
- Poems in Portuguese
### Russian
- 19 000 Russian poems

## Team members

- Álvaro Pérez (alvp)
- Javier de la Rosa (versae)
- Aitor Díaz (aitordiaz)
- Elena González-Blanco
- Salvador Ros (salva)

## Useful links

- Community Week timeline
- Community Week README
- Community Week thread
- Community Week channel
- Masked Language Modelling example scripts
- Model Repository

## Acknowledgments

This project would not have been possible without the infrastructure and resources provided by HuggingFace and Google Cloud. Moreover, we want to thank POSTDATA Project (ERC-StG-679528) and the Computational Literary Studies Infrastructure (CLS INFRA No. 101004984) of the European Union's Horizon 2020 research and innovation programme for their support and time allowance.
# ALBERTI

ALBERTI is a set of two BERT-based multilingual model for poetry. One for verses and another one for stanzas. This model has been further trained with the PULPO corpus for verses using Flax, including training scripts.

This is part of the
Flax/Jax Community Week, organised by HuggingFace and TPU usage sponsored by Google. ## PULPO

PULPO, the Prodigious Unannotated Literary Poetry Corpus, is a set of multilingual corpora of verses and stanzas with over 95M words.

The following corpora has been downloaded using the Averell tool, developed by the POSTDATA team: ### Spanish
- Disco v3
- Corpus of Spanish Golden-Age Sonnets
- Corpus general de poesía lírica castellana del Siglo de Oro
- Gongocorpus - source ### English
- Eighteenth-Century Poetry Archive (ECPA)
- For better for verse ### French
- Métrique en Ligne - source ### Italian
- Biblioteca italiana - source ### Czech
- Corpus of Czech Verse ### Portuguese
- Stichotheque

Also, we obtained the following corpora from these sources: ### Spanish 
- URL - source ### English
- A Gutenberg Poetry Corpus ### Arabic
- Arabic Poetry dataset ### Chinese
- THU Chinese Classical Poetry Corpus ### Finnish
- SKVR ### German
- TextGrid Poetry Corpus - source
- German Rhyme Corpus ### Hungarian
- verskorpusz ### Portuguese
- Poems in Portuguese ### Russian
- 19 000 Russian poems ## Team members

- Álvaro Pérez (alvp)
- Javier de la Rosa (versae)
- Aitor Díaz (aitordiaz)
- Elena González-Blanco
- Salvador Ros (salva) ## Useful links

- Community Week timeline
- Community Week README
- Community Week thread
- Community Week channel
- Masked Language Modelling example scripts
- Model Repository ## Acknowledgments

This project would not have been possible without the infrastructure and resources provided by HuggingFace and Google Cloud. Moreover, we want to thank POSTDATA Project (ERC-StG-679528) and the Computational Literary Studies Infrastructure (CLS INFRA No. 101004984) of the European Union's Horizon 2020 research and innovation programme for their support and time allowance.
TAGS
#transformers #pytorch #jax #joblib #safetensors #bert #fill-mask #multilingual #es #license-cc-by-4.0 #autotrain_compatible #endpoints_compatible #has_space #region-us 
 # ALBERTI

ALBERTI is a set of two BERT-based multilingual model for poetry. One for verses and another one for stanzas. This model has been further trained with the PULPO corpus for verses using Flax, including training scripts.

This is part of the
Flax/Jax Community Week, organised by HuggingFace and TPU usage sponsored by Google. ## PULPO

PULPO, the Prodigious Unannotated Literary Poetry Corpus, is a set of multilingual corpora of verses and stanzas with over 95M words.

The following corpora has been downloaded using the Averell tool, developed by the POSTDATA team: ### Spanish
- Disco v3
- Corpus of Spanish Golden-Age Sonnets
- Corpus general de poesía lírica castellana del Siglo de Oro
- Gongocorpus - source ### English
- Eighteenth-Century Poetry Archive (ECPA)
- For better for verse ### French
- Métrique en Ligne - source ### Italian
- Biblioteca italiana - source ### Czech
- Corpus of Czech Verse ### Portuguese
- Stichotheque

Also, we obtained the following corpora from these sources: ### Spanish 
- URL - source ### English
- A Gutenberg Poetry Corpus ### Arabic
- Arabic Poetry dataset ### Chinese
- THU Chinese Classical Poetry Corpus ### Finnish
- SKVR ### German
- TextGrid Poetry Corpus - source
- German Rhyme Corpus ### Hungarian
- verskorpusz ### Portuguese
- Poems in Portuguese ### Russian
- 19 000 Russian poems ## Team members

- Álvaro Pérez (alvp)
- Javier de la Rosa (versae)
- Aitor Díaz (aitordiaz)
- Elena González-Blanco
- Salvador Ros (salva) ## Useful links

- Community Week timeline
- Community Week README
- Community Week thread
- Community Week channel
- Masked Language Modelling example scripts
- Model Repository ## Acknowledgments

This project would not have been possible without the infrastructure and resources provided by HuggingFace and Google Cloud. Moreover, we want to thank POSTDATA Project (ERC-StG-679528) and the Computational Literary Studies Infrastructure (CLS INFRA No. 101004984) of the European Union's Horizon 2020 research and innovation programme for their support and time allowance.
passage: TAGS
#transformers #pytorch #jax #joblib #safetensors #bert #fill-mask #multilingual #es #license-cc-by-4.0 #autotrain_compatible #endpoints_compatible #has_space #region-us 
# ALBERTI

ALBERTI is a set of two BERT-based multilingual model for poetry. One for verses and another one for stanzas. This model has been further trained with the PULPO corpus for verses using Flax, including training scripts.

This is part of the
Flax/Jax Community Week, organised by HuggingFace and TPU usage sponsored by Google.## PULPO

PULPO, the Prodigious Unannotated Literary Poetry Corpus, is a set of multilingual corpora of verses and stanzas with over 95M words.

The following corpora has been downloaded using the Averell tool, developed by the POSTDATA team:### Spanish
- Disco v3
- Corpus of Spanish Golden-Age Sonnets
- Corpus general de poesía lírica castellana del Siglo de Oro
- Gongocorpus - source### English
- Eighteenth-Century Poetry Archive (ECPA)
- For better for verse### French
- Métrique en Ligne - source### Italian
- Biblioteca italiana - source### Czech
- Corpus of Czech Verse### Portuguese
- Stichotheque

Also, we obtained the following corpora from these sources:### Spanish 
- URL - source### English
- A Gutenberg Poetry Corpus### Arabic
- Arabic Poetry dataset### Chinese
- THU Chinese Classical Poetry Corpus### Finnish
- SKVR### German
- TextGrid Poetry Corpus - source
- German Rhyme Corpus### Hungarian
- verskorpusz### Portuguese
- Poems in Portuguese### Russian
- 19 000 Russian poems## Team members

- Álvaro Pérez (alvp)
- Javier de la Rosa (versae)
- Aitor Díaz (aitordiaz)
- Elena González-Blanco
- Salvador Ros (salva)## Useful links

- Community Week timeline
- Community Week README
- Community Week thread
- Community Week channel
- Masked Language Modelling example scripts
- Model Repository