|
--- |
|
language: |
|
- en |
|
license: apache-2.0 |
|
base_model: |
|
- google/gemma-2-2b-it |
|
datasets: |
|
- FinGPT/fingpt-fiqa_qa |
|
- FinGPT/fingpt-headline |
|
model-index: |
|
- name: gemma-2b-def |
|
results: |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: IFEval (0-Shot) |
|
type: HuggingFaceH4/ifeval |
|
args: |
|
num_few_shot: 0 |
|
metrics: |
|
- type: inst_level_strict_acc and prompt_level_strict_acc |
|
value: 26.93 |
|
name: strict accuracy |
|
source: |
|
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ell44ot/gemma-2b-def |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: BBH (3-Shot) |
|
type: BBH |
|
args: |
|
num_few_shot: 3 |
|
metrics: |
|
- type: acc_norm |
|
value: 4.59 |
|
name: normalized accuracy |
|
source: |
|
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ell44ot/gemma-2b-def |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: MATH Lvl 5 (4-Shot) |
|
type: hendrycks/competition_math |
|
args: |
|
num_few_shot: 4 |
|
metrics: |
|
- type: exact_match |
|
value: 1.74 |
|
name: exact match |
|
source: |
|
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ell44ot/gemma-2b-def |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: GPQA (0-shot) |
|
type: Idavidrein/gpqa |
|
args: |
|
num_few_shot: 0 |
|
metrics: |
|
- type: acc_norm |
|
value: 3.13 |
|
name: acc_norm |
|
source: |
|
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ell44ot/gemma-2b-def |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: MuSR (0-shot) |
|
type: TAUR-Lab/MuSR |
|
args: |
|
num_few_shot: 0 |
|
metrics: |
|
- type: acc_norm |
|
value: 5.31 |
|
name: acc_norm |
|
source: |
|
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ell44ot/gemma-2b-def |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: MMLU-PRO (5-shot) |
|
type: TIGER-Lab/MMLU-Pro |
|
config: main |
|
split: test |
|
args: |
|
num_few_shot: 5 |
|
metrics: |
|
- type: acc |
|
value: 6.36 |
|
name: accuracy |
|
source: |
|
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ell44ot/gemma-2b-def |
|
name: Open LLM Leaderboard |
|
--- |
|
This is a finetuned gemma2b model that is trained using FinGPT datasets |
|
|
|
Model Overview |
|
Model Name: Gemma 2B |
|
Version: 1.0 |
|
Date: November 2023 |
|
Task: Financial Data Analysis |
|
Framework: [Insert framework, e.g., TensorFlow, PyTorch] |
|
License: [Insert license type] |
|
|
|
Description |
|
Gemma 2B is a machine learning model designed to analyze and predict financial trends and behaviors using a comprehensive finance dataset. The model leverages advanced algorithms to provide insights into market movements, investment opportunities, and risk assessment. |
|
|
|
Intended Use |
|
Gemma 2B is intended for use by financial analysts, investors, and researchers looking to: |
|
|
|
Predict stock prices and market trends. |
|
Analyze financial statements and company performance. |
|
Assess portfolio risks and returns. |
|
Generate insights for strategic financial planning. |
|
Dataset Information |
|
Dataset: Finance Dataset |
|
Source: [Specify source, e.g., Yahoo Finance, SEC filings] |
|
Size: [Insert size, e.g., 100,000 records] |
|
Features: |
|
|
|
Historical stock prices |
|
Trading volumes |
|
Economic indicators |
|
Company financial metrics (e.g., revenue, earnings) |
|
News sentiment scores |
|
Performance Metrics |
|
The performance of Gemma 2B is evaluated using the following metrics: |
|
|
|
Mean Absolute Error (MAE) |
|
Root Mean Squared Error (RMSE) |
|
R-squared (R²) |
|
Benchmark Results: |
|
|
|
MAE: [Insert value] |
|
RMSE: [Insert value] |
|
R²: [Insert value] |
|
Limitations |
|
The model is trained on historical data and may not account for unprecedented market events. |
|
Performance can vary based on the selected features and parameters. |
|
Requires continuous updates with new data to maintain accuracy. |
|
Ethical Considerations |
|
Ensure compliance with financial regulations and ethical standards when using the model. |
|
Be aware of potential biases in the training data that may affect predictions. |
|
Future Work |
|
Future improvements may include: |
|
|
|
Incorporating additional datasets (e.g., macroeconomic data). |
|
Enhancing the model with deeper learning techniques or ensemble methods. |
|
Continuous monitoring and retraining to adapt to market changes. |
|
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) |
|
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_ell44ot__gemma-2b-def) |
|
|
|
| Metric |Value| |
|
|-------------------|----:| |
|
|Avg. | 8.01| |
|
|IFEval (0-Shot) |26.93| |
|
|BBH (3-Shot) | 4.59| |
|
|MATH Lvl 5 (4-Shot)| 1.74| |
|
|GPQA (0-shot) | 3.13| |
|
|MuSR (0-shot) | 5.31| |
|
|MMLU-PRO (5-shot) | 6.36| |
|
|
|
|