
# Hindi language model
## Trained with ELECTRA base size settings

<a href="https://colab.research.google.com/drive/1R8TciRSM7BONJRBc9CBZbzOmz39FTLl_">Tokenization and training CoLab</a>

## Example Notebooks

This model outperforms Multilingual BERT on <a href="https://colab.research.google.com/drive/1UYn5Th8u7xISnPUBf72at1IZIm3LEDWN">Hindi movie reviews / sentiment analysis</a> (using SimpleTransformers)

You can get higher accuracy using ktrain + TensorFlow, where you can adjust learning rate and
other hyperparameters: https://colab.research.google.com/drive/1mSeeSfVSOT7e-dVhPlmSsQRvpn6xC05w?usp=sharing

Question-answering on MLQA dataset: https://colab.research.google.com/drive/1i6fidh2tItf_-IDkljMuaIGmEU6HT2Ar#scrollTo=IcFoAHgKCUiQ

A smaller model (<a href="https://huggingface.co/monsoon-nlp/hindi-bert">Hindi-BERT</a>) performs better on a BBC news classification task.

## Corpus

The corpus is two files:
- Hindi CommonCrawl deduped by OSCAR https://traces1.inria.fr/oscar/
- latest Hindi Wikipedia ( https://dumps.wikimedia.org/hiwiki/ ) + WikiExtractor to txt

Bonus notes:
- Adding English wiki text or parallel corpus could help with cross-lingual tasks and training

## Vocabulary

https://drive.google.com/file/d/1-6tXrii3tVxjkbrpSJE9MOG_HhbvP66V/view?usp=sharing

Bonus notes:
- Created with HuggingFace Tokenizers; you can increase vocabulary size and re-train; remember to change ELECTRA vocab_size

## Training

Structure your files, with data-dir named "trainer" here

```
trainer
- vocab.txt
- pretrain_tfrecords
-- (all .tfrecord... files)
- models
-- modelname
--- checkpoint
--- graph.pbtxt
--- model.*
```

## Conversion

Use this process to convert an in-progress or completed ELECTRA checkpoint to a Transformers-ready model:

```
git clone https://github.com/huggingface/transformers
python ./transformers/src/transformers/convert_electra_original_tf_checkpoint_to_pytorch.py
  --tf_checkpoint_path=./models/checkpointdir
  --config_file=config.json
  --pytorch_dump_path=pytorch_model.bin
  --discriminator_or_generator=discriminator
python
```

```
from transformers import TFElectraForPreTraining
model = TFElectraForPreTraining.from_pretrained("./dir_with_pytorch", from_pt=True)
model.save_pretrained("tf")
```

Once you have formed one directory with config.json, pytorch_model.bin, tf_model.h5, special_tokens_map.json, tokenizer_config.json, and vocab.txt on the same level, run:

```
transformers-cli upload directory
```

{"language": "hi"}
transformers pytorch tf electra feature-extraction hi endpoints_compatible region:us
TAGS
#transformers #pytorch #tf #electra #feature-extraction #hi #endpoints_compatible #region-us 


# Hindi language model
## Trained with ELECTRA base size settings

<a href="URL and training CoLab</a>

## Example Notebooks

This model outperforms Multilingual BERT on <a href="URL movie reviews / sentiment analysis</a> (using SimpleTransformers)

You can get higher accuracy using ktrain + TensorFlow, where you can adjust learning rate and
other hyperparameters: URL

Question-answering on MLQA dataset: URL

A smaller model (<a href="URL performs better on a BBC news classification task.

## Corpus

The corpus is two files:
- Hindi CommonCrawl deduped by OSCAR URL
- latest Hindi Wikipedia ( URL ) + WikiExtractor to txt

Bonus notes:
- Adding English wiki text or parallel corpus could help with cross-lingual tasks and training

## Vocabulary

URL

Bonus notes:
- Created with HuggingFace Tokenizers; you can increase vocabulary size and re-train; remember to change ELECTRA vocab_size

## Training

Structure your files, with data-dir named "trainer" here



## Conversion

Use this process to convert an in-progress or completed ELECTRA checkpoint to a Transformers-ready model:





Once you have formed one directory with URL, pytorch_model.bin, tf_model.h5, special_tokens_map.json, tokenizer_config.json, and URL on the same level, run:



# Hindi language model ## Trained with ELECTRA base size settings

<a href="URL and training CoLab</a> ## Example Notebooks

This model outperforms Multilingual BERT on <a href="URL movie reviews / sentiment analysis</a> (using SimpleTransformers)

You can get higher accuracy using ktrain + TensorFlow, where you can adjust learning rate and
other hyperparameters: URL

Question-answering on MLQA dataset: URL

A smaller model (<a href="URL performs better on a BBC news classification task. ## Corpus

The corpus is two files:
- Hindi CommonCrawl deduped by OSCAR URL
- latest Hindi Wikipedia ( URL ) + WikiExtractor to txt

Bonus notes:
- Adding English wiki text or parallel corpus could help with cross-lingual tasks and training ## Vocabulary

URL

Bonus notes:
- Created with HuggingFace Tokenizers; you can increase vocabulary size and re-train; remember to change ELECTRA vocab_size ## Training

Structure your files, with data-dir named "trainer" here ## Conversion

Use this process to convert an in-progress or completed ELECTRA checkpoint to a Transformers-ready model:





Once you have formed one directory with URL, pytorch_model.bin, tf_model.h5, special_tokens_map.json, tokenizer_config.json, and URL on the same level, run:
TAGS
#transformers #pytorch #tf #electra #feature-extraction #hi #endpoints_compatible #region-us 
 # Hindi language model ## Trained with ELECTRA base size settings

<a href="URL and training CoLab</a> ## Example Notebooks

This model outperforms Multilingual BERT on <a href="URL movie reviews / sentiment analysis</a> (using SimpleTransformers)

You can get higher accuracy using ktrain + TensorFlow, where you can adjust learning rate and
other hyperparameters: URL

Question-answering on MLQA dataset: URL

A smaller model (<a href="URL performs better on a BBC news classification task. ## Corpus

The corpus is two files:
- Hindi CommonCrawl deduped by OSCAR URL
- latest Hindi Wikipedia ( URL ) + WikiExtractor to txt

Bonus notes:
- Adding English wiki text or parallel corpus could help with cross-lingual tasks and training ## Vocabulary

URL

Bonus notes:
- Created with HuggingFace Tokenizers; you can increase vocabulary size and re-train; remember to change ELECTRA vocab_size ## Training

Structure your files, with data-dir named "trainer" here ## Conversion

Use this process to convert an in-progress or completed ELECTRA checkpoint to a Transformers-ready model:





Once you have formed one directory with URL, pytorch_model.bin, tf_model.h5, special_tokens_map.json, tokenizer_config.json, and URL on the same level, run:
passage: TAGS
#transformers #pytorch #tf #electra #feature-extraction #hi #endpoints_compatible #region-us 
# Hindi language model## Trained with ELECTRA base size settings

<a href="URL and training CoLab</a>## Example Notebooks

This model outperforms Multilingual BERT on <a href="URL movie reviews / sentiment analysis</a> (using SimpleTransformers)

You can get higher accuracy using ktrain + TensorFlow, where you can adjust learning rate and
other hyperparameters: URL

Question-answering on MLQA dataset: URL

A smaller model (<a href="URL performs better on a BBC news classification task.## Corpus

The corpus is two files:
- Hindi CommonCrawl deduped by OSCAR URL
- latest Hindi Wikipedia ( URL ) + WikiExtractor to txt

Bonus notes:
- Adding English wiki text or parallel corpus could help with cross-lingual tasks and training## Vocabulary

URL

Bonus notes:
- Created with HuggingFace Tokenizers; you can increase vocabulary size and re-train; remember to change ELECTRA vocab_size## Training

Structure your files, with data-dir named "trainer" here## Conversion

Use this process to convert an in-progress or completed ELECTRA checkpoint to a Transformers-ready model:





Once you have formed one directory with URL, pytorch_model.bin, tf_model.h5, special_tokens_map.json, tokenizer_config.json, and URL on the same level, run: