
# hebrew-bad_wiki-gpt_neo-tiny

## Table of Contents
- [Model Details](#model-details)
- [Uses](#uses)
- [Risks, Limitations and Biases](#risks-limitations-and-biases)
- [Training](#training)
- [Evaluation](#evaluation)
- [Environmental Impact](#environmental-impact)
- [How to Get Started With the Model](#how-to-get-started-with-the-model)

## Model Details
**Model Description:**

The model developer notes that the model is 
> Hebrew nonsense generation model which produces really bad wiki-abstract text. 


- **Developed by:** [Doron Adler](https://github.com/Norod)
- **Model Type:** Text Generation
- **Language(s):** Hebrew
- **License:** MIT
- **Resources for more information:**
- [GitHub Repo](https://github.com/Norod/hebrew-gpt_neo)
- [HuggingFace Space](https://huggingface.co/spaces/Norod78/Hebrew-GPT-Neo-Small)


## Uses

#### Direct Use

This model can be used for text generation.

#### Misuse and Out-of-scope Use


## Risks, Limitations and Biases
**CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes.**

Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)).




## Training


#### Training Data
 [Hebrew Wikipedia Dump](https://dumps.wikimedia.org/hewiki/latest/) (hewiki abstract) from May 2020




#### Training Procedure


This model was fined tuned upon [hebrew-gpt_neo-tiny](https://huggingface.co/Norod78/hebrew-gpt_neo-tiny) which was previously trained using [EleutherAI's gpt-neo](https://github.com/EleutherAI/gpt-neo). 

Fine-tuning on the wiki-absract text was done using [@minimaxir](https://twitter.com/minimaxir)'s [aitextgen](https://github.com/minimaxir/aitextgen).



## Evaluation


#### Configs

Model configs for the hebrew-gpt_neo-tiny is available on the [hebrew-gpt_neo model github](https://github.com/Norod/hebrew-gpt_neo/tree/main/hebrew-gpt_neo-tiny/configs) 

* **Activation Function:** gelu
* **Number_Head:** 12
* **Number_Vocab:** 50257
* **Train batch size:** 250
* **Eval batch size:** 64
* **Predict batch size:** 1




## Environmental Impact

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). We present the hardware type based on the [associated paper](https://arxiv.org/pdf/2105.09680.pdf).


- **Hardware Type:** [More information needed]

- **Hours used:** Unknown

- **Cloud Provider:** GCP  tpu-v8s

- **Compute Region:** europe-west4

- **Carbon Emitted:** [More information needed]


## How to Get Started With the Model

A Google Colab Notebook is also available [here](https://colab.research.google.com/github/Norod/hebrew-gpt_neo/blob/main/hebrew-gpt_neo-tiny/Norod78_hebrew_gpt_neo_tiny_Colab.ipynb)


​​
```
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Norod78/hebrew-bad_wiki-gpt_neo-tiny")

model = AutoModelForCausalLM.from_pretrained("Norod78/hebrew-bad_wiki-gpt_neo-tiny")


```



{"language": "he", "license": "mit", "thumbnail": "https://avatars1.githubusercontent.com/u/3617152?norod.jpg", "widget": [{"text": "\u05de\u05ea\u05de\u05d8\u05d9\u05e7\u05d4:"}, {"text": "\u05e2\u05dc\u05d9\u05d9\u05ea \u05d4\u05de\u05db\u05d5\u05e0\u05d5\u05ea"}, {"text": "\u05d5\u05d9\u05e7\u05d9\u05e4\u05d3\u05d9\u05d4 \u05d4\u05e2\u05d1\u05e8\u05d9\u05ea"}, {"text": "\u05d4\u05d0\u05d9\u05e8\u05d5\u05d5\u05d9\u05d6\u05d9\u05d5\u05df \u05d4\u05d5\u05d0"}, {"text": "\u05d3\u05d5\u05d3 \u05d1\u05df-\u05d2\u05d5\u05e8\u05d9\u05d5\u05df \u05d4\u05d9\u05d4"}]}
transformers pytorch coreml safetensors gpt_neo text-generation he arxiv:1910.09700 arxiv:2105.09680 license:mit autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #coreml #safetensors #gpt_neo #text-generation #he #arxiv-1910.09700 #arxiv-2105.09680 #license-mit #autotrain_compatible #endpoints_compatible #region-us 


# hebrew-bad_wiki-gpt_neo-tiny

## Table of Contents
- Model Details
- Uses
- Risks, Limitations and Biases
- Training
- Evaluation
- Environmental Impact
- How to Get Started With the Model

## Model Details
Model Description:

The model developer notes that the model is 
> Hebrew nonsense generation model which produces really bad wiki-abstract text. 


- Developed by: Doron Adler
- Model Type: Text Generation
- Language(s): Hebrew
- License: MIT
- Resources for more information:
- GitHub Repo
- HuggingFace Space


## Uses

#### Direct Use

This model can be used for text generation.

#### Misuse and Out-of-scope Use


## Risks, Limitations and Biases
CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes.

Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)).




## Training


#### Training Data
 Hebrew Wikipedia Dump (hewiki abstract) from May 2020




#### Training Procedure


This model was fined tuned upon hebrew-gpt_neo-tiny which was previously trained using EleutherAI's gpt-neo. 

Fine-tuning on the wiki-absract text was done using @minimaxir's aitextgen.



## Evaluation


#### Configs

Model configs for the hebrew-gpt_neo-tiny is available on the hebrew-gpt_neo model github 

* Activation Function: gelu
* Number_Head: 12
* Number_Vocab: 50257
* Train batch size: 250
* Eval batch size: 64
* Predict batch size: 1




## Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). We present the hardware type based on the associated paper.


- Hardware Type: [More information needed]

- Hours used: Unknown

- Cloud Provider: GCP  tpu-v8s

- Compute Region: europe-west4

- Carbon Emitted: [More information needed]


## How to Get Started With the Model

A Google Colab Notebook is also available here


​​




# hebrew-bad_wiki-gpt_neo-tiny ## Table of Contents
- Model Details
- Uses
- Risks, Limitations and Biases
- Training
- Evaluation
- Environmental Impact
- How to Get Started With the Model ## Model Details
Model Description:

The model developer notes that the model is 
> Hebrew nonsense generation model which produces really bad wiki-abstract text. 


- Developed by: Doron Adler
- Model Type: Text Generation
- Language(s): Hebrew
- License: MIT
- Resources for more information:
- GitHub Repo
- HuggingFace Space ## Uses #### Direct Use

This model can be used for text generation. #### Misuse and Out-of-scope Use ## Risks, Limitations and Biases
CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes.

Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)). ## Training #### Training Data
 Hebrew Wikipedia Dump (hewiki abstract) from May 2020 #### Training Procedure


This model was fined tuned upon hebrew-gpt_neo-tiny which was previously trained using EleutherAI's gpt-neo. 

Fine-tuning on the wiki-absract text was done using @minimaxir's aitextgen. ## Evaluation #### Configs

Model configs for the hebrew-gpt_neo-tiny is available on the hebrew-gpt_neo model github 

* Activation Function: gelu
* Number_Head: 12
* Number_Vocab: 50257
* Train batch size: 250
* Eval batch size: 64
* Predict batch size: 1 ## Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). We present the hardware type based on the associated paper.


- Hardware Type: [More information needed]

- Hours used: Unknown

- Cloud Provider: GCP  tpu-v8s

- Compute Region: europe-west4

- Carbon Emitted: [More information needed] ## How to Get Started With the Model

A Google Colab Notebook is also available here


​​
TAGS
#transformers #pytorch #coreml #safetensors #gpt_neo #text-generation #he #arxiv-1910.09700 #arxiv-2105.09680 #license-mit #autotrain_compatible #endpoints_compatible #region-us 
 # hebrew-bad_wiki-gpt_neo-tiny ## Table of Contents
- Model Details
- Uses
- Risks, Limitations and Biases
- Training
- Evaluation
- Environmental Impact
- How to Get Started With the Model ## Model Details
Model Description:

The model developer notes that the model is 
> Hebrew nonsense generation model which produces really bad wiki-abstract text. 


- Developed by: Doron Adler
- Model Type: Text Generation
- Language(s): Hebrew
- License: MIT
- Resources for more information:
- GitHub Repo
- HuggingFace Space ## Uses #### Direct Use

This model can be used for text generation. #### Misuse and Out-of-scope Use ## Risks, Limitations and Biases
CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes.

Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)). ## Training #### Training Data
 Hebrew Wikipedia Dump (hewiki abstract) from May 2020 #### Training Procedure


This model was fined tuned upon hebrew-gpt_neo-tiny which was previously trained using EleutherAI's gpt-neo. 

Fine-tuning on the wiki-absract text was done using @minimaxir's aitextgen. ## Evaluation #### Configs

Model configs for the hebrew-gpt_neo-tiny is available on the hebrew-gpt_neo model github 

* Activation Function: gelu
* Number_Head: 12
* Number_Vocab: 50257
* Train batch size: 250
* Eval batch size: 64
* Predict batch size: 1 ## Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). We present the hardware type based on the associated paper.


- Hardware Type: [More information needed]

- Hours used: Unknown

- Cloud Provider: GCP  tpu-v8s

- Compute Region: europe-west4

- Carbon Emitted: [More information needed] ## How to Get Started With the Model

A Google Colab Notebook is also available here


​​
passage: TAGS
#transformers #pytorch #coreml #safetensors #gpt_neo #text-generation #he #arxiv-1910.09700 #arxiv-2105.09680 #license-mit #autotrain_compatible #endpoints_compatible #region-us 
# hebrew-bad_wiki-gpt_neo-tiny## Table of Contents
- Model Details
- Uses
- Risks, Limitations and Biases
- Training
- Evaluation
- Environmental Impact
- How to Get Started With the Model## Model Details
Model Description:

The model developer notes that the model is 
> Hebrew nonsense generation model which produces really bad wiki-abstract text. 


- Developed by: Doron Adler
- Model Type: Text Generation
- Language(s): Hebrew
- License: MIT
- Resources for more information:
- GitHub Repo
- HuggingFace Space## Uses#### Direct Use

This model can be used for text generation.#### Misuse and Out-of-scope Use## Risks, Limitations and Biases
CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes.

Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)).## Training#### Training Data
 Hebrew Wikipedia Dump (hewiki abstract) from May 2020#### Training Procedure


This model was fined tuned upon hebrew-gpt_neo-tiny which was previously trained using EleutherAI's gpt-neo. 

Fine-tuning on the wiki-absract text was done using @minimaxir's aitextgen.## Evaluation#### Configs

Model configs for the hebrew-gpt_neo-tiny is available on the hebrew-gpt_neo model github 

* Activation Function: gelu
* Number_Head: 12
* Number_Vocab: 50257
* Train batch size: 250
* Eval batch size: 64
* Predict batch size: 1