# dant5-small

---
language:
- da
language_bcp47:
- da
- da-bornholm
- da-synnejyl
tags:
- t5
license: cc-by-4.0
datasets:
- dagw
widget:
- text: "Aarhus er Danmarks <extra_id_0>.<extra_id_2>"
co2_eq_emissions:
      training_type: "pretraining"
      geographical_location: "Copenhagen, Denmark"
      hardware_used: "4 A100 GPUs, 91 training hours"
      emissions: 23660
---

`dant5-small` is a 60M parameter model with architecture identical to `t5-small`. Training details are given in the paper [Training a T5 Using Lab-sized Resources](https://arxiv.org/abs/2208.12097).  It was trained for 10 epochs on the Danigh GigaWord Corpus ([official website](https://gigaword.dk), [paper](https://aclanthology.org/2021.nodalida-main.46/)).

## To use the model

```python
from transformers import AutoTokenizer, T5ForConditionalGeneration

model_name = "strombergnlp/dant5-small"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = T5ForConditionalGeneration.from_pretrained(model_name)

original_text = "Aarhus er Danmarks <extra_id_0> landets ældste. Under navnet Aros, som betyder å-munding, optræder den i skriftlige kilder i 900-tallet, men <extra_id_1> historie tilbage til 700-tallet.<extra_id_2>"
original_label = "<extra_id_0> næststørste by og en af <extra_id_1> arkæologiske fund fører dens <extra_id_2>"
input_ids = tokenizer(original_text, return_tensors="pt").input_ids
labels = tokenizer(original_label, return_tensors="pt").input_ids

loss = model(input_ids=input_ids, labels=labels).loss
print(f"Original text: {original_text}")
print(f"Original label: {original_label}")
print(f"Loss for the original label is {loss.item()}")

sequence_ids = model.generate(input_ids)
sequences = tokenizer.batch_decode(sequence_ids)
print(f"A sample generated continuation: ")
print(sequences[0])
```

You should see output similar to:

```
Original text: Aarhus er Danmarks <extra_id_0> landets ældste. Under navnet Aros, som betyder å-munding, optræder den i skriftlige kilder i 900-tallet, men <extra_id_1> historie tilbage til 700-tallet.<extra_id_2>
Original label: <extra_id_0> næststørste by og en af <extra_id_1> arkæologiske fund fører dens <extra_id_2>
Loss for the original label is 3.383681297302246
A sample generated continuation: 
<pad><extra_id_0> ældste og<extra_id_1> har sin<extra_id_2> Aarhus er Danmarks ældste<extra_id_3></s>
```
{}
transformers pytorch tensorboard safetensors t5 text2text-generation arxiv:2208.12097 doi:10.57967/hf/0012 autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #tensorboard #safetensors #t5 #text2text-generation #arxiv-2208.12097 #doi-10.57967/hf/0012 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

# dant5-small

---
language:
- da
language_bcp47:
- da
- da-bornholm
- da-synnejyl
tags:
- t5
license: cc-by-4.0
datasets:
- dagw
widget:
- text: "Aarhus er Danmarks <extra_id_0>.<extra_id_2>"
co2_eq_emissions:
      training_type: "pretraining"
      geographical_location: "Copenhagen, Denmark"
      hardware_used: "4 A100 GPUs, 91 training hours"
      emissions: 23660
---

'dant5-small' is a 60M parameter model with architecture identical to 't5-small'. Training details are given in the paper Training a T5 Using Lab-sized Resources.  It was trained for 10 epochs on the Danigh GigaWord Corpus (official website, paper).

## To use the model



You should see output similar to:


# dant5-small

---
language:
- da
language_bcp47:
- da
- da-bornholm
- da-synnejyl
tags:
- t5
license: cc-by-4.0
datasets:
- dagw
widget:
- text: "Aarhus er Danmarks <extra_id_0>.<extra_id_2>"
co2_eq_emissions:
      training_type: "pretraining"
      geographical_location: "Copenhagen, Denmark"
      hardware_used: "4 A100 GPUs, 91 training hours"
      emissions: 23660
---

'dant5-small' is a 60M parameter model with architecture identical to 't5-small'. Training details are given in the paper Training a T5 Using Lab-sized Resources.  It was trained for 10 epochs on the Danigh GigaWord Corpus (official website, paper). ## To use the model



You should see output similar to:
TAGS
#transformers #pytorch #tensorboard #safetensors #t5 #text2text-generation #arxiv-2208.12097 #doi-10.57967/hf/0012 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 # dant5-small

---
language:
- da
language_bcp47:
- da
- da-bornholm
- da-synnejyl
tags:
- t5
license: cc-by-4.0
datasets:
- dagw
widget:
- text: "Aarhus er Danmarks <extra_id_0>.<extra_id_2>"
co2_eq_emissions:
      training_type: "pretraining"
      geographical_location: "Copenhagen, Denmark"
      hardware_used: "4 A100 GPUs, 91 training hours"
      emissions: 23660
---

'dant5-small' is a 60M parameter model with architecture identical to 't5-small'. Training details are given in the paper Training a T5 Using Lab-sized Resources.  It was trained for 10 epochs on the Danigh GigaWord Corpus (official website, paper). ## To use the model



You should see output similar to:
passage: TAGS
#transformers #pytorch #tensorboard #safetensors #t5 #text2text-generation #arxiv-2208.12097 #doi-10.57967/hf/0012 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
# dant5-small

---
language:
- da
language_bcp47:
- da
- da-bornholm
- da-synnejyl
tags:
- t5
license: cc-by-4.0
datasets:
- dagw
widget:
- text: "Aarhus er Danmarks <extra_id_0>.<extra_id_2>"
co2_eq_emissions:
      training_type: "pretraining"
      geographical_location: "Copenhagen, Denmark"
      hardware_used: "4 A100 GPUs, 91 training hours"
      emissions: 23660
---

'dant5-small' is a 60M parameter model with architecture identical to 't5-small'. Training details are given in the paper Training a T5 Using Lab-sized Resources.  It was trained for 10 epochs on the Danigh GigaWord Corpus (official website, paper).## To use the model



You should see output similar to: