# roberta_python
---
language: code
datasets:
- code_search_net
- Fraser/python-lines
tags:
- python
- code
- masked-lm
widget:
- text "assert 6 == sum([i for i in range(<mask>)])"
---
# Details
This is a roBERTa-base model trained on the python part of [CodeSearchNet](https://github.com/github/CodeSearchNet) and reached a dev perplexity of 3.296

This model was used for the Programming Puzzles enumerative solver baseline detailed in [Programming Puzzles paper](https://arxiv.org/abs/2106.05784).

See also the [Python Programming Puzzles (P3) Repository](https://github.com/microsoft/PythonProgrammingPuzzles) for more details.

# Usage

You can either load the model and further fine-tune it for a target task (as done for the puzzle solver), or you can experiment with mask-filling directly with this model as in the following example:

```python
from transformers import AutoTokenizer, AutoModelWithLMHead, pipeline

tokenizer = AutoTokenizer.from_pretrained("tals/roberta_python")
model = AutoModelWithLMHead.from_pretrained("tals/roberta_python")

demo = pipeline("fill-mask", model=model, tokenizer=tokenizer)

code = """sum= 0
for i in range(<mask>):
    sum += i
assert sum == 6
"""
demo(code)
```

# BibTeX entry and citation info

```bibtex
@inproceedings{
      schuster2021programming,
      title={Programming Puzzles},
      author={Tal Schuster and Ashwin Kalyan and Alex Polozov and Adam Tauman Kalai},
      booktitle={Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1)},
      year={2021},
      url={https://openreview.net/forum?id=fe_hCc4RBrg}
}
```

{}
transformers pytorch roberta fill-mask arxiv:2106.05784 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #roberta #fill-mask #arxiv-2106.05784 #autotrain_compatible #endpoints_compatible #region-us 

# roberta_python
---
language: code
datasets:
- code_search_net
- Fraser/python-lines
tags:
- python
- code
- masked-lm
widget:
- text "assert 6 == sum([i for i in range(<mask>)])"
---
# Details
This is a roBERTa-base model trained on the python part of CodeSearchNet and reached a dev perplexity of 3.296

This model was used for the Programming Puzzles enumerative solver baseline detailed in Programming Puzzles paper.

See also the Python Programming Puzzles (P3) Repository for more details.

# Usage

You can either load the model and further fine-tune it for a target task (as done for the puzzle solver), or you can experiment with mask-filling directly with this model as in the following example:



# BibTeX entry and citation info



# roberta_python
---
language: code
datasets:
- code_search_net
- Fraser/python-lines
tags:
- python
- code
- masked-lm
widget:
- text "assert 6 == sum([i for i in range(<mask>)])"
--- # Details
This is a roBERTa-base model trained on the python part of CodeSearchNet and reached a dev perplexity of 3.296

This model was used for the Programming Puzzles enumerative solver baseline detailed in Programming Puzzles paper.

See also the Python Programming Puzzles (P3) Repository for more details. # Usage

You can either load the model and further fine-tune it for a target task (as done for the puzzle solver), or you can experiment with mask-filling directly with this model as in the following example: # BibTeX entry and citation info
TAGS
#transformers #pytorch #roberta #fill-mask #arxiv-2106.05784 #autotrain_compatible #endpoints_compatible #region-us 
 # roberta_python
---
language: code
datasets:
- code_search_net
- Fraser/python-lines
tags:
- python
- code
- masked-lm
widget:
- text "assert 6 == sum([i for i in range(<mask>)])"
--- # Details
This is a roBERTa-base model trained on the python part of CodeSearchNet and reached a dev perplexity of 3.296

This model was used for the Programming Puzzles enumerative solver baseline detailed in Programming Puzzles paper.

See also the Python Programming Puzzles (P3) Repository for more details. # Usage

You can either load the model and further fine-tune it for a target task (as done for the puzzle solver), or you can experiment with mask-filling directly with this model as in the following example: # BibTeX entry and citation info
passage: TAGS
#transformers #pytorch #roberta #fill-mask #arxiv-2106.05784 #autotrain_compatible #endpoints_compatible #region-us 
# roberta_python
---
language: code
datasets:
- code_search_net
- Fraser/python-lines
tags:
- python
- code
- masked-lm
widget:
- text "assert 6 == sum([i for i in range(<mask>)])"
---# Details
This is a roBERTa-base model trained on the python part of CodeSearchNet and reached a dev perplexity of 3.296

This model was used for the Programming Puzzles enumerative solver baseline detailed in Programming Puzzles paper.

See also the Python Programming Puzzles (P3) Repository for more details.# Usage

You can either load the model and further fine-tune it for a target task (as done for the puzzle solver), or you can experiment with mask-filling directly with this model as in the following example:# BibTeX entry and citation info