# Model
This model is based on [nicoladecao/msmarco-word2vec256000-distilbert-base-uncased](https://huggingface.co/nicoladecao/msmarco-word2vec256000-distilbert-base-uncased) with a 256k sized vocabulary initialized with word2vec.

This model has been trained with MLM on the MS MARCO corpus collection for 400k steps. See train_mlm.py for the train script. It was run on 2x V100 GPUs. The word embedding matrix was frozen.


{}
transformers pytorch distilbert fill-mask autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #distilbert #fill-mask #autotrain_compatible #endpoints_compatible #region-us 

# Model
This model is based on nicoladecao/msmarco-word2vec256000-distilbert-base-uncased with a 256k sized vocabulary initialized with word2vec.

This model has been trained with MLM on the MS MARCO corpus collection for 400k steps. See train_mlm.py for the train script. It was run on 2x V100 GPUs. The word embedding matrix was frozen.


# Model
This model is based on nicoladecao/msmarco-word2vec256000-distilbert-base-uncased with a 256k sized vocabulary initialized with word2vec.

This model has been trained with MLM on the MS MARCO corpus collection for 400k steps. See train_mlm.py for the train script. It was run on 2x V100 GPUs. The word embedding matrix was frozen.
TAGS
#transformers #pytorch #distilbert #fill-mask #autotrain_compatible #endpoints_compatible #region-us 
 # Model
This model is based on nicoladecao/msmarco-word2vec256000-distilbert-base-uncased with a 256k sized vocabulary initialized with word2vec.

This model has been trained with MLM on the MS MARCO corpus collection for 400k steps. See train_mlm.py for the train script. It was run on 2x V100 GPUs. The word embedding matrix was frozen.
passage: TAGS
#transformers #pytorch #distilbert #fill-mask #autotrain_compatible #endpoints_compatible #region-us 
# Model
This model is based on nicoladecao/msmarco-word2vec256000-distilbert-base-uncased with a 256k sized vocabulary initialized with word2vec.

This model has been trained with MLM on the MS MARCO corpus collection for 400k steps. See train_mlm.py for the train script. It was run on 2x V100 GPUs. The word embedding matrix was frozen.