This repo contains a low-rank adapter for LLaMA-13b finetuned on Ntropy proprietary dataset (consumer financial transactions).

This version of the weights was trained with the following hyperparameters:

  • Base Model: decapoda-research/llama-13b-hf
  • Epochs: 10 (load from best epoch)
  • Batch size: 16
  • Cutoff length: 1024
  • Learning rate: 3e-4
  • Lora r: 16
  • Lora target modules: q_proj, k_proj, v_proj, o_proj

Instructions for running the adapter can be found at https://github.com/ntropy-network/enrichment_models/blob/main/notebooks/llama.ipynb

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no library tag.