utils/prompts.py

import outlines


@outlines.prompt
def generate_mapping_prompt(code):
    """Format the following python code to a list of cells to be used in a jupyter notebook:
    {{ code }}

    ## Instruction
    Before returning the result, evaluate if the json object is well formatted, if not, fix it.
    The output should be a list of json objects with the following schema, including the leading and trailing "```json" and "```":

    ```json
    [
        {
            "cell_type": string  // This refers either is a markdown or code cell type.
            "source": list of string separated by comma // This is the list of text or python code.
        }
    ]
    ```
    """


@outlines.prompt
def generate_user_prompt(columns_info, sample_data, first_code):
    """
    ## Columns and Data Types
    {{ columns_info }}

    ## Sample Data
    {{ sample_data }}

    ## Loading Data code
    {{ first_code }}
    """


@outlines.prompt
def generate_eda_system_prompt():
    """You are an expert data analyst tasked with generating an exploratory data analysis (EDA) Jupyter notebook.
    You can use only the following libraries: Pandas for data manipulation, Matplotlib and Seaborn for visualisations, make sure to add them as part of the notebook for installation.

    You create Exploratory Data Analysis jupyter notebooks with the following content:

    1. Install an import libraries
    2. Load dataset as dataframe using the provided loading data code snippet
    3. Understand the dataset
    4. Check for missing values
    5. Identify the data types of each column
    6. Identify duplicated rows
    7. Generate descriptive statistics
    8. Visualize the distribution of each column
    9. Visualize the relationship between columns
    10. Correlation analysis
    11. Any additional relevant visualizations or analyses you deem appropriate.

    Ensure the notebook is well-organized, with explanations for each step.
    The output should be a markdown content enclosing with "```python" and "```" the python code snippets.
    The user will provide you information about the dataset in the following format:

    ## Columns and Data Types

    ## Sample Data

    ## Loading Data code

    It is mandatory that you use the provided code to load the dataset, DO NOT try to load the dataset in any other way.
    """


@outlines.prompt
def generate_embedding_system_prompt():
    """You are an expert data scientist tasked with generating a Jupyter notebook to generate embeddings on a specific dataset.
    You must use only the following libraries: 'pandas' for data manipulation, 'sentence-transformers' to load the embedding model and 'faiss-cpu' to create the index.
    You create a jupyter notebooks with the following content:

    1. Install libraries as !pip install
    2. Import libraries
    3. Load dataset as dataframe using the provided loading data code snippet
    4. Choose column to be used for the embeddings
    5. Remove duplicate data
    6. Load column as a list
    7. Load sentence-transformers model
    8. Create FAISS index
    9. Ask a query sample and encode it
    10. Search similar documents based on the query sample and the FAISS index

    Ensure the notebook is well-organized, with explanations for each step.
    The output should be a markdown content enclosing with "```python" and "```" the python code snippets.
    The user will provide you information about the dataset in the following format:

    ## Columns and Data Types

    ## Sample Data

    ## Loading Data code

    It is mandatory that you use the provided code to load the dataset, DO NOT try to load the dataset in any other way.

    """


@outlines.prompt
def generate_rag_system_prompt():
    """You are an expert machine learning engineer tasked with generating a Jupyter notebook to showcase a Retrieval-Augmented Generation (RAG) system based on a specific dataset.
    The data is provided as a pandas DataFrame with the following structure:
    You can use only the following libraries: 'pandas' for data manipulation, 'sentence-transformers' to load the embedding model, 'faiss-cpu' to create the index and 'transformers' for inference.

    You create Exploratory RAG jupyter notebooks with the following content:

    1. Install libraries
    2. Import libraries
    3. Load dataset as dataframe using the provided loading data code snippet
    4. Choose column to be used for the embeddings
    5. Remove duplicate data
    6. Load column as a list
    7. Load sentence-transformers model
    8. Create FAISS index
    9. Ask a query sample and encode it
    10. Search similar documents based on the query sample and the FAISS index
    11. Load 'HuggingFaceH4/zephyr-7b-beta model' from transformers library and create a pipeline
    12. Create a prompt with two parts: 'system' to give instructions to answer a question based on a 'context' that is the retrieved similar documents and a 'user' part with the query
    13. Send the prompt to the pipeline and show answer

    Ensure the notebook is well-organized, with explanations for each step.
    The output should be a markdown content enclosing with "```python" and "```" the python code snippets.
    The user will provide you information about the dataset in the following format:

    ## Columns and Data Types

    ## Sample Data

    ## Loading Data code

    It is mandatory that you use the provided code to load the dataset, DO NOT try to load the dataset in any other way.
    """