---
license: llama2
datasets:
- nvidia/OpenMathInstruct-1
language:
- en
library_name: transformers
tags:
- nvidia
- code
- math
---


# OpenMath-CodeLlama-7b-Python

## Description:

OpenMath models were designed to solve mathematical problems by integrating text-based reasoning with code blocks
executed by Python interpreter. The models were trained on [OpenMathInstruct-1](https://huggingface.co/datasets/nvidia/OpenMathInstruct-1),
a math instruction tuning dataset with 1.8M problem-solution pairs generated using permissively licensed
[Mixtral-8x7B](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) model.


| Model                                            | Size  | GSM8K     | MATH     |
|--------------------------------------------------|-------|-----------|----------|
| GPT-4 [1]                                        |   -   |    94.4   |   56.2   |
| GPT-4 + code [2]                                 |   -   |    92.9   |   69.7   |
| OpenMath-CodeLlama-7B ([nemo](https://huggingface.co/nvidia/OpenMath-CodeLlama-7b-Python), [HF](https://huggingface.co/nvidia/OpenMath-CodeLlama-7b-Python-hf))                 |   7B  |    75.9   |   43.6   |
| OpenMath-CodeLlama-7B + self-consistency (k=50)  |   7B  |    84.8   |   55.6   |
| OpenMath-Mistral-7B ([nemo](https://huggingface.co/nvidia/OpenMath-Mistral-7B-v0.1), [HF](https://huggingface.co/nvidia/OpenMath-Mistral-7B-v0.1-hf))                              |   7B  |    80.2   |   44.5   |
| OpenMath-Mistral-7B + self-consistency (k=50)    |   7B  |    86.9   |   57.2   |
| OpenMath-CodeLlama-13B ([nemo](https://huggingface.co/nvidia/OpenMath-CodeLlama-13b-Python), [HF](https://huggingface.co/nvidia/OpenMath-CodeLlama-13b-Python-hf))                           |  13B  |    78.8   |   45.5   |
| OpenMath-CodeLlama-13B + self-consistency (k=50) |  13B  |    86.8   |   57.6   |
| OpenMath-CodeLlama-34B ([nemo](https://huggingface.co/nvidia/OpenMath-CodeLlama-34b-Python), [HF](https://huggingface.co/nvidia/OpenMath-CodeLlama-34b-Python-hf))                           |  34B  |    80.7   |   48.3   |
| OpenMath-CodeLlama-34B + self-consistency (k=50) |  34B  |    88.0   |   60.2   |
| OpenMath-Llama2-70B ([nemo](https://huggingface.co/nvidia/OpenMath-Llama-2-70b), [HF](https://huggingface.co/nvidia/OpenMath-Llama-2-70b-hf))                              |  70B  |    84.7   |   46.3   |
| OpenMath-Llama2-70B + self-consistency (k=50)    |  70B  |    90.1   |   58.3   |
| OpenMath-CodeLlama-70B ([nemo](https://huggingface.co/nvidia/OpenMath-CodeLlama-70b-Python), [HF](https://huggingface.co/nvidia/OpenMath-CodeLlama-70b-Python-hf))                           |  70B  |  **84.6**      |   **50.7**    |
| OpenMath-CodeLlama-70B + self-consistency (k=50) |  70B  |  **90.8**    |   **60.4**    |


The pipeline we used to produce these models is fully open-sourced under a commercially permissive license.

- [Code](https://github.com/Kipok/NeMo-Skills)
- [Models](https://huggingface.co/collections/nvidia/openmath-65c5619de2ba059be0775014)
- [Dataset](https://huggingface.co/datasets/nvidia/OpenMathInstruct-1)

## How to use the models?

Try to [run inference with our models](/docs/inference.md) with just a few commands!

We provide [all instructions](/docs/reproducing-results.md) to fully reproduce our results.

If you want to improve your own models or to learn more about our pipeline, read through the relevant docs below.

- [Model evaluation](/docs/evaluation.md)
- [Generating synthetic data](/docs/synthetic-data-generation.md)
- [Finetuning models](/docs/finetuning.md)

## Training

This model is trained with [NVIDIA NeMo](https://www.nvidia.com/en-us/ai-data-science/generative-ai/nemo-framework/),
an end-to-end, cloud-native framework to build, customize, and deploy generative AI models anywhere.
It includes training and inferencing frameworks, guardrailing toolkits, data curation tools, and pretrained models,
offering enterprises an easy, cost-effective, and fast way to adopt generative AI.

Please see [NeMo-Skills Github repo](https://github.com/Kipok/NeMo-Skills) for training details.

## Contact

E-Mail: [Igor Gitman](mailto:igitman@nvidia.com)

## Citation

If you find this model useful, please cite the following works

TODO

## License
The use of this model is governed by the [Llama 2 Community License Agreement](https://ai.meta.com/llama/license/)