richardr1126 commited on
Commit
32f3acc
·
1 Parent(s): 6dca0ce

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +56 -30
README.md CHANGED
@@ -1,32 +1,62 @@
1
  ---
2
- license: cc-by-nc-4.0
3
  tags:
4
- - generated_from_trainer
5
- model-index:
6
- - name: qlora-wizard-coder-adapter
7
- results: []
 
 
 
 
 
 
 
 
8
  ---
9
-
10
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
11
- should probably proofread and complete it, then remove this comment. -->
12
-
13
- # qlora-wizard-coder-adapter
14
-
15
- This model is a fine-tuned version of [WizardLM/WizardCoder-15B-V1.0](https://huggingface.co/WizardLM/WizardCoder-15B-V1.0) on the None dataset.
16
-
17
- ## Model description
18
-
19
- More information needed
20
-
21
- ## Intended uses & limitations
22
-
23
- More information needed
24
-
25
- ## Training and evaluation data
26
-
27
- More information needed
28
-
29
- ## Training procedure
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
 
31
  ### Training hyperparameters
32
 
@@ -41,10 +71,6 @@ The following hyperparameters were used during training:
41
  - training_steps: 1000
42
  - mixed_precision_training: Native AMP
43
 
44
- ### Training results
45
-
46
-
47
-
48
  ### Framework versions
49
 
50
  - Transformers 4.30.0.dev0
 
1
  ---
 
2
  tags:
3
+ - sql
4
+ - spider
5
+ - text-to-sql
6
+ - sql fine-tune
7
+ - LoRA
8
+ - QLoRa
9
+ - adapter
10
+ datasets:
11
+ - spider
12
+ - richardr1126/spider-skeleton-context-instruct
13
+ library_name: peft
14
+ license: bigcode-openrail-m
15
  ---
16
+ ### Spider Skeleton Wizard Coder QLoRA Adapter Summary
17
+
18
+ - This QLoRa adapter was created by finetuning [WizardLM/WizardCoder-15B-V1.0](https://huggingface.co/WizardLM/WizardCoder-15B-V1.0) on an enhanced Spider context training dataset: [richardr1126/spider-skeleton-context-instruct](https://huggingface.co/datasets/richardr1126/spider-skeleton-context-instruct).
19
+ - Finetuning was performed using QLoRa on a single RTX6000 48GB.
20
+ - If you want the merged model: [richardr1126/spider-skeleton-wizard-coder-merged](https://huggingface.co/richardr1126/spider-skeleton-wizard-coder-merged)
21
+
22
+ ### Spider Dataset
23
+
24
+ [Spider](https://arxiv.org/abs/1809.08887) is a large-scale complex and cross-domain semantic parsing and text-to-SQL dataset annotated by 11 Yale students
25
+ The goal of the Spider challenge is to develop natural language interfaces to cross-domain databases.
26
+
27
+ This dataset was used to finetune this model.
28
+
29
+ ## Citation
30
+
31
+ Please cite the repo if you use the data or code in this repo.
32
+
33
+ ```
34
+ @misc{luo2023wizardcoder,
35
+ title={WizardCoder: Empowering Code Large Language Models with Evol-Instruct},
36
+ author={Ziyang Luo and Can Xu and Pu Zhao and Qingfeng Sun and Xiubo Geng and Wenxiang Hu and Chongyang Tao and Jing Ma and Qingwei Lin and Daxin Jiang},
37
+ year={2023},
38
+ }
39
+ ```
40
+ ```
41
+ @article{yu2018spider,
42
+ title={Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task},
43
+ author={Yu, Tao and Zhang, Rui and Yang, Kai and Yasunaga, Michihiro and Wang, Dongxu and Li, Zifan and Ma, James and Li, Irene and Yao, Qingning and Roman, Shanelle and others},
44
+ journal={arXiv preprint arXiv:1809.08887},
45
+ year={2018}
46
+ }
47
+ ```
48
+ ```
49
+ @article{dettmers2023qlora,
50
+ title={QLoRA: Efficient Finetuning of Quantized LLMs},
51
+ author={Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke},
52
+ journal={arXiv preprint arXiv:2305.14314},
53
+ year={2023}
54
+ }
55
+ ```
56
+
57
+ ## Disclaimer
58
+
59
+ The resources, including code, data, and model weights, associated with this project are restricted for academic research purposes only and cannot be used for commercial purposes. The content produced by any version of WizardCoder is influenced by uncontrollable variables such as randomness, and therefore, the accuracy of the output cannot be guaranteed by this project. This project does not accept any legal liability for the content of the model output, nor does it assume responsibility for any losses incurred due to the use of associated resources and output results.
60
 
61
  ### Training hyperparameters
62
 
 
71
  - training_steps: 1000
72
  - mixed_precision_training: Native AMP
73
 
 
 
 
 
74
  ### Framework versions
75
 
76
  - Transformers 4.30.0.dev0