Update README.md

README.md

@@ -11,11 +11,7 @@ language:
 - vi
 pipeline_tag: text-generation
 ---
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1).
+# Model Card for WangChanGLM 🐘 - The Thai-Turned-Multilingual Instruction-Following Model
 
 ## Model Details
 
@@ -23,22 +19,20 @@ This modelcard aims to be a base template for new models. It has been generated
 
 <!-- Provide a longer summary of what this model is. -->
 
+WangChanGLM is a Thai-turned-multilingual, instruction-finetuned Facebook XGLM-7.5B using open-source, commercially permissible datasets (LAION OIG chip2 and infill_dbpedia, DataBricks Dolly v2, OpenAI TL;DR, and Hello-SimpleAI HC3; about 400k examples), released under CC-BY SA 4.0. The models are trained to perform a subset of instruction-following tasks we found most relevant namely: reading comprehension, brainstorming, and creative writing. We provide the weights for a model finetuned on an English-only dataset ([wangchanglm-7.5B-sft-en](https://huggingface.co/pythainlp/wangchanglm-7.5B-sft-en)) and another checkpoint further finetuned on Google-Translated Thai dataset ([wangchanglm-7.5B-sft-enth](https://huggingface.co/pythainlp/wangchanglm-7.5B-sft-enth)). We perform Vicuna-style evaluation using both humans and ChatGPT (in our case, `gpt-3.5-turbo` since we are still on the waitlist for `gpt-4`) and observe some discrepancies between the two types of annoators. All training and evaluation codes are shared under the [Apache-2.0 license](https://github.com/pythainlp/wangchanglm/blob/main/LICENSE) in our Github, as well as datasets and model weights on [HuggingFace](https://huggingface.co/pythainlp). See our live demo [here]().
 
+- **Developed by:** [PyThaiNLP](https://www.github.com/pythainlp) and [VISTEC-depa AI Research Institute of Thailand](https://huggingface.co/airesearch)
+- **Model type:** Finetuned [XGLM-7.5B](https://huggingface.co/facebook/xglm-7.5B)
+- **Language(s) (NLP)**: `en`, `th`, `ja`, `vi` capacibilities evaluated, theoretically all 30 languages of [XGLM-7.5B](https://huggingface.co/facebook/xglm-7.5B)
+- **License:** [CC-BY SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)
 
-- **Developed by:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
+### Model Sources
 
 <!-- Provide the basic links for the model. -->
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- **Repository:** [pythainlp/wangchanglm](https://www.github.com/pythainlp/wangchanglm)
+- **Blog:** [Medium]()
+- **Demo:** [Colab notebook]()
 
 ## Uses
 
@@ -48,25 +42,25 @@ This modelcard aims to be a base template for new models. It has been generated
 
 <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
 
-[More Information Needed]
+Intended to be use as an instruction-following model for reading comprehension, brainstorming and creative writing.
 
-### Downstream Use [optional]
+### Downstream Use
 
 <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
 
-[More Information Needed]
+The model can be finetuned for any typical instruction-following use cases.
 
 ### Out-of-Scope Use
 
 <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
 
-[More Information Needed]
+We do not expect the models to perform well in math problems, reasoning, and factfulness. We intentionally filter out training examples from these use cases.
 
 ## Bias, Risks, and Limitations
 
 <!-- This section is meant to convey both technical and sociotechnical limitations. -->
 
-[More Information Needed]
+We noticed similar limitations to other finetuned instruction followers such as math problems, reasoning, and factfulness. Even though the models do not perform on the level that we expect them to be abused, they do contain undesirable biases and toxicity and should be further optimized for your particular use cases.
 
 ### Recommendations
 
@@ -78,7 +72,40 @@ Users (both direct and downstream) should be made aware of the risks, biases and
 
 Use the code below to get started with the model.
 
-[More Information Needed]
+```
+name_model = "pythainlp/wangchanglm-7.5B-sft-en"
+model = AutoModelForCausalLM.from_pretrained(
+    model_name, 
+    return_dict=True, 
+    load_in_8bit=True ,
+    device_map="auto", 
+    torch_dtype=torch.float16, 
+    offload_folder="./", 
+    low_cpu_mem_usage=True,
+)
+text = "เล่นหุ้นยังไงให้รวย"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+batch = tokenizer(text, return_tensors="pt")
+with torch.cuda.amp.autocast(): 
+  output_tokens = model.generate(
+      input_ids=batch["input_ids"],
+      max_new_tokens=max_gen_len, # 512
+      begin_suppress_tokens = exclude_ids,
+      no_repeat_ngram_size=2,
+      
+      #oasst k50
+      top_k=50,
+      top_p=top_p, # 0.95
+      typical_p=1.,
+      temperature=temperature, # 0.9
+      
+      # #oasst typical3
+      # typical_p = 0.3,
+      # temperature = 0.8,
+      # repetition_penalty = 1.2,
+  )
+tokenizer.decode(output_tokens[0], skip_special_tokens=True)
+```
 
 ## Training Details
 
@@ -86,65 +113,31 @@ Use the code below to get started with the model.
 
 <!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
-[More Information Needed]
+Finetuning datasets are sourced from [LAION OIG chip2 and infill_dbpedia](https://huggingface.co/datasets/laion/OIG) ([Apache-2.0](https://github.com/pythainlp/wangchanglm/blob/main/LICENSE)), [DataBricks Dolly v2](https://github.com/databrickslabs/dolly) ([Apache-2.0](https://github.com/pythainlp/wangchanglm/blob/main/LICENSE)), [OpenAI TL;DR](https://github.com/openai/summarize-from-feedback) ([MIT](https://opensource.org/license/mit/)), and [Hello-SimpleAI HC3](https://huggingface.co/datasets/Hello-SimpleAI/HC3) ([CC-BY SA](https://creativecommons.org/licenses/by-sa/4.0/)).
 
 ### Training Procedure 
 
 <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
 
-#### Preprocessing [optional]
+#### Preprocessing 
 
-[More Information Needed]
+See [pythainlp/wangchanglm](https://www.github.com/pythainlp/wangchanglm).
 
 
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
+- **Training regime:** LoRA with 4 GPUs. See more details at [pythainlp/wangchanglm](https://www.github.com/pythainlp/wangchanglm).
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
 
 ## Evaluation
 
 <!-- This section describes the evaluation protocols and provides the results. -->
 
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Data Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
+We performed automatic evaluation in the style of [Vicuna](https://vicuna.lmsys.org/) and human evaluation. See more details from our [blog]().
 
 #### Summary
 
 
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
 ## Environmental Impact
 
 <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
@@ -175,7 +168,7 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 
 [More Information Needed]
 
-## Citation [optional]
+## Citation
 
 <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
 
@@ -183,24 +176,6 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 
 [More Information Needed]
 
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
 ## Model Card Contact
 
 [More Information Needed]