Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,79 @@
+---
+library_name: pytorch
+license: llama2
+pipeline_tag: text-generation
+tags:
+- llm
+- generative_ai
+- quantized
+- android
+
+---
+
+![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/llama_v2_7b_chat_quantized/web-assets/banner.png)
+
+# Llama-v2-7B-Chat: Optimized for Mobile Deployment
+## State-of-the-art large language model useful on a variety of language understanding and generation tasks
+
+Llama 2 is a family of LLMs. The "Chat" at the end indicates that the model is optimized for chatbot-like dialogue. The model is quantized to 4-bit weights and 16-bit activations making it suitable for on-device deployment. For Prompt and output length specified below, the time to first token is Llama-PromptProcessor-Quantized's latency and average time per addition token is Llama-TokenGenerator-KVCache-Quantized's latency.
+
+This is based on the implementation of Llama-v2-7B-Chat found
+[here](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf). More details on model performance
+accross various devices, can be found [here](https://aihub.qualcomm.com/models/llama_v2_7b_chat_quantized).
+
+### Model Details
+
+- **Model Type:** Text generation
+- **Model Stats:**
+  - Number of parameters: 7B
+  - Model size: 3.6GB
+  - Model-1 (Prompt Processor): Llama-PromptProcessor-Quantized
+  - Max context length: 1024
+  - Prompt processor input: 1024 tokens
+  - Prompt processor output: 1 output token + KVCache for token generator
+  - Model-2 (Token Generator): Llama-TokenGenerator-KVCache-Quantized
+  - Token generator input: 1 input token + past KVCache
+  - Token generator output: 1 output token + KVCache for next iteration
+  - Decoding length: 1024 (1 output token + 1023 from KVCache)
+  - Use: Initiate conversation with prompt-processor and then token generator for subsequent iterations.
+  - QNN-SDK: 2.19
+
+
+| Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Precision | Primary Compute Unit | Target Model
+| ---|---|---|---|---|---|---|---|
+| Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | QNN Model Library | 117.812 ms | 66 - 238 MB | UINT16 | NPU | Llama-TokenGenerator-KVCache-Quantized 
+| Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | QNN Model Library | 2578.521 ms | 12 - 17 MB | UINT16 | NPU | Llama-PromptProcessor-Quantized 
+
+
+## License
+- The license for the original implementation of Llama-v2-7B-Chat can be found
+  [here](https://github.com/facebookresearch/llama/blob/main/LICENSE).
+
+## References
+* [LLaMA: Open and Efficient Foundation Language Models](https://arxiv.org/abs/2302.13971)
+* [Source Model Implementation](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf)
+
+## Community
+* Join [our AI Hub Slack community](https://join.slack.com/t/qualcomm-ai-hub/shared_invite/zt-2dgf95loi-CXHTDRR1rvPgQWPO~ZZZJg) to collaborate, post questions and learn more about on-device AI.
+* For questions or feedback please [reach out to us](mailto:[email protected]).
+
+## Usage and Limitations
+
+Model may not be used for or in connection with any of the following applications:
+
+- Accessing essential private and public services and benefits;
+- Administration of justice and democratic processes;
+- Assessing or recognizing the emotional state of a person;
+- Biometric and biometrics-based systems, including categorization of persons based on sensitive characteristics;
+- Education and vocational training;
+- Employment and workers management;
+- Exploitation of the vulnerabilities of persons resulting in harmful behavior;
+- General purpose social scoring;
+- Law enforcement;
+- Management and operation of critical infrastructure;
+- Migration, asylum and border control management;
+- Predictive policing;
+- Real-time remote biometric identification in public spaces;
+- Recommender systems of social media platforms;
+- Scraping of facial images (from the internet or otherwise); and/or
+- Subliminal manipulation