# Tokenizer

We trained our tokenizer using [sentencepiece](https://github.com/google/sentencepiece)'s unigram tokenizer. Then loaded the tokenizer as MT5TokenizerFast.

## Model

We used [MT5-base](https://huggingface.co/google/mt5-base) model.

## Datasets

We used [Code Search Net](https://huggingface.co/datasets/code_search_net)'s dataset and some scrapped data from internet to train the model. We maintained a list of datasets where each dataset had codes of same language.

## Plots

### Train loss

![train loss](https://i.ibb.co/x53Wm8n/train-loss.png)

### Evaluation loss

![eval loss](https://i.ibb.co/McB2jnf/eval-loss.png)

### Evaluation accuracy

![eval accuracy](https://i.ibb.co/YDGhLdn/eval-accuracy.png)

### Learning rate

![learning rate](https://i.ibb.co/CMStzWv/learning-rate.png)

## Fine tuning (WIP)

We fine tuned the model with [CodeXGLUE code-to-code-trans dataset](https://huggingface.co/datasets/code_x_glue_cc_code_to_code_trans), and scrapper data.

{}
transformers pytorch jax tensorboard safetensors mt5 text2text-generation autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #jax #tensorboard #safetensors #mt5 #text2text-generation #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

# Tokenizer

We trained our tokenizer using sentencepiece's unigram tokenizer. Then loaded the tokenizer as MT5TokenizerFast.

## Model

We used MT5-base model.

## Datasets

We used Code Search Net's dataset and some scrapped data from internet to train the model. We maintained a list of datasets where each dataset had codes of same language.

## Plots

### Train loss

!train loss

### Evaluation loss

!eval loss

### Evaluation accuracy

!eval accuracy

### Learning rate

!learning rate

## Fine tuning (WIP)

We fine tuned the model with CodeXGLUE code-to-code-trans dataset, and scrapper data.

# Tokenizer

We trained our tokenizer using sentencepiece's unigram tokenizer. Then loaded the tokenizer as MT5TokenizerFast. ## Model

We used MT5-base model. ## Datasets

We used Code Search Net's dataset and some scrapped data from internet to train the model. We maintained a list of datasets where each dataset had codes of same language. ## Plots ### Train loss

!train loss ### Evaluation loss

!eval loss ### Evaluation accuracy

!eval accuracy ### Learning rate

!learning rate ## Fine tuning (WIP)

We fine tuned the model with CodeXGLUE code-to-code-trans dataset, and scrapper data.
TAGS
#transformers #pytorch #jax #tensorboard #safetensors #mt5 #text2text-generation #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 # Tokenizer

We trained our tokenizer using sentencepiece's unigram tokenizer. Then loaded the tokenizer as MT5TokenizerFast. ## Model

We used MT5-base model. ## Datasets

We used Code Search Net's dataset and some scrapped data from internet to train the model. We maintained a list of datasets where each dataset had codes of same language. ## Plots ### Train loss

!train loss ### Evaluation loss

!eval loss ### Evaluation accuracy

!eval accuracy ### Learning rate

!learning rate ## Fine tuning (WIP)

We fine tuned the model with CodeXGLUE code-to-code-trans dataset, and scrapper data.
passage: TAGS
#transformers #pytorch #jax #tensorboard #safetensors #mt5 #text2text-generation #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
# Tokenizer

We trained our tokenizer using sentencepiece's unigram tokenizer. Then loaded the tokenizer as MT5TokenizerFast.## Model

We used MT5-base model.## Datasets

We used Code Search Net's dataset and some scrapped data from internet to train the model. We maintained a list of datasets where each dataset had codes of same language.## Plots### Train loss

!train loss### Evaluation loss

!eval loss### Evaluation accuracy

!eval accuracy### Learning rate

!learning rate## Fine tuning (WIP)

We fine tuned the model with CodeXGLUE code-to-code-trans dataset, and scrapper data.