To avoid tuning the entire 7 billion parameters in LLaMA-2, we adopt LLaMA-Adapter in our fine-tuning procedure. LLaMA-Adapter achieves parameter-efficient fine-tuning using learnable adaption prompts: for each of the topmost L transformer layers, a learnable prompt of length K is prepended to the (embedded) word tokens. This procedure reduces the total number of trainable parameters to K × L × C, where C is the token embedding dimensions, set to 4096 by default in LLaMA. Throughout this study, K = 10 and L = 30, giving 1.2 million trainable parameters that can fit in a GPU of 24 GB memory in half precision.