README.md

---

base_model: openlm-research/open_llama_3b_v2
tags:
  - llama-2
  - instruct
  - finetune
  - alpaca
  - gpt4
  - synthetic data
  - distillation
datasets:
  - teknium/openhermes
model-index:
  - name: openhermes-7b
    results: []
license: apache-2.0
language:
  - en
---

# OpenHermes-Llama-3B
*An OpenLlama-3B finetune for role-play, instruction following, and code generation*

<img src="https://cdn-uploads.huggingface.co/production/uploads/648a374f00f7a3374ee64b99/bDQ8ybHVS--CibUf4rvFB.png" width="400" height="400">

*Logo AI Generated by Dalle*

For a guide on how this was fine tuned, check out this blog [post](https://calebfahlgren.com/post/how-to-finetune-a-large-language-model).

# Model description
**OpenHermes-Lllama_3B** is a finetune of the Hermes dataset that has a fully open source dataset!
It was trained on 15,000 entries of primarily GPT-4 generated data, from open datasets across the AI landscape, including:

- GPTeacher - General Instruct, Roleplay v1, Roleplay v2, and Code Instruct Datasets, by Teknium
- WizardLM (v1, evol_instruct 70k), by WizardLM Team/nlpxucan
- Airoboros GPT-4 (v1.0), by JonDurbin
- Camel-AI's domain expert datasets, by the Camel-AI Team
- CodeAlpaca, by Sahil2801
- GPT4-LLM and Unnatural Instructions, by Microsoft


**OpenHermes-Lllama-3B** has improved very much improved coding abilities and general instruction following than the OpenLlama-3B base model. You can see the before and after outputs comparison below.


# Example Outputs

### Writing Code

**Q:** Write a Fast API endpoint with ratelimiting

**OpenHermes-Llama-3B Output**

<img src="https://cdn-uploads.huggingface.co/production/uploads/648a374f00f7a3374ee64b99/ZZxphP0JN1Y6HfcGDRwJ1.png" width="600" height="600">

**Base Model Output**

*Create a new FastAPI app

*Create a new endpoint

*Create a new ratelimit middleware

*Create a new ratelimit rule

*Create a new ratelimit backend

*Create a new ratelimit backend

### Planning an Itinerary

**Q:** Create a travel itinery for italy for 3 days for me.

**OpenHermes-Llama-3B Output**

Hi,

I am happy to help you create an itinery for Italy for 3 days.
Here is an example itinery:
Day 1:

Morning: Arrive in Rome, check into your hotel.

Afternoon: Visit the Colosseum, the Roman Forum, and the Pantheon.

Evening: Have a dinner at a local restaurant and enjoy the nightlife in Rome.

Day 2:

Morning: Visit the Vatican Museums, St. Peter's Basilica, and the Sistine Chapel.

Afternoon: Take a guided tour of the Italian capital.

Evening: Have a dinner at a local restaurant and enjoy the nightlife in Rome.

Day 3:

Morning: Visit the Fontana di Trevi, the Pantheon, and the Colosseum.

Afternoon: Take a guided tour of the Italian capital.

Evening: Have a dinner at a local restaurant and enjoy the nightlife in Rome.

You can also add visits to other attractions such as the Capitoline Museums, the Palatine Hill, and the Spanish Steps.

I hope this itinery helps you plan your trip to Italy. Let me know if you have any questions or if you need any further assistance.

**Base Model Output**

Hi,
I am a travel agent and I can help you with your travel itinerary.
I can help you with the following:
1. Flight booking
2. Hotel booking
3. Car rental
4. Tour booking
5. Visa
6. Travel insurance
7. Airport transfers
8. Airport pick up
9. Airport drop off
10. Airport lounge
11. Airport lounge pick up
12. Airport lounge drop off

### Explaining Concepts

**Q:** Explain golang structs

**OpenHermes-Llama-3B Output**

A golang struct is a way to define a data type in Go. It is a collection of fields, each of which is a value of a specific type. The fields are arranged in a hierarchy and are connected using interfaces.
Example:

```go
type User struct {
    Name  string
    Age   int
    Email string
}
```
In this example, the User struct contains a Name field, an Age field, an Email field, and a few other fields. The Name field is of type string, the Age field is of type int, and the Email field is of type string. The User struct is connected to other structs through interfaces, such as the interface for the User type.
The main advantage of structs in Go is that they can be used to represent data in a way that is more flexible and easier to use than traditional arrays or maps. Structs also allow for more complex data structures to be created without the need for nested arrays or maps.
Here's an example of how to use a User struct:
package main

```go
import (
    "fmt"
    "strings"
)

type User struct {
    Name  string
    Age   int
    Email string
}

func main() {
    user1 := User{"John", 25, "[email protected]"}
    user2 := User{"Mary", 22, "[email protected]"}

    // Print the users' names and ages
    fmt.Printf("User 1: %s (%d)\nUser 2: %s (%d)\n", user1.Name, user1.Age, user2.Name, user2.Age)

    // Print the users' emails separated by commas
    email1 := user1.Email
    email2 := user2.Email
    fmt.Printf("User 1's email: %s\nUser 2's email: %s\n", email1, email2)
}
```
In this example, the User struct is used to represent two different users. The Name field is used to store the users' names, the Age field is used to store their ages, and the Email field is used to store their emails. The main function prints out the users' names and ages, along with their emails separated by commas.
In summary, a golang struct is a way to define data types in Go. It is a collection of fields, each of which is a value of a specific type. The fields are arranged in a hierarchy and are connected through interfaces. The main advantage of structs in Go is that they can be used to represent data in a way that is more flexible and easier to use than traditional arrays or maps.

**Base Model Output**

*A struct is a named collection of variables.

# Benchmark Comparison
## HumanEval

| Model                          | pass@1 |
| ------------------------------ | ------ |
| openllama-3b-v2  (base)        | 8.1%   |
| openhermes-llama-3b (finetune) | 9.8%   |