Update README.md

README.md

@@ -1,3 +1,40 @@
 ---
 license: apache-2.0
+datasets:
+- Crystalcareai/openhermes_200k_unfiltered
+- mlabonne/orpo-dpo-mix-40k
+- jondurbin/airoboros-3.2
+- abacusai/SystemChat-1.1
+- trollek/SimpleInstructionJudge-v01
+- cgato/SlimOrcaDedupCleaned
+language:
+- en
+library_name: transformers
+base_model: h2oai/h2o-danube3-4b-base
+tags:
+- mergekit
+- magpie
 ---
+# LittlePromptMaker-4B-v0.1
+
+A small model to create prompts the [Magpie](https://arxiv.org/abs/2406.08464) way.
+
+The secret sauce turned out to be also training on the prompts. I did that last with SystemChat-1.1 in order to be able to steer the prompt generation. It does not work without a system message.
+
+Now imagine, if you will, having this bad boy generating a bunch of different prompts right, and having another model like, I mean.. [LittleInstructionJudge](https://huggingface.co/trollek/LittleInstructionJudge-4B-v0.1) right, judge all of the instructions right, and then slam a serverfarm with the cream of the crop right.
+
+In other words, giving it a system prompt like "You are a creative writing partner", "You are an advanced coding assistant", "You are a damn good psychologist", etc, you can can quickly generate prompts for a niche dataset that can then be answered by large model. 
+
+In a different language: Ved hjælp at Husskades indsigt, hvor man udnytter sprogmodellers natur til at skabe tilpasningsdata, kan man med fordel bruge denne sprogmodel til at skrive instruktioner, og endda styre indholdet ved hjælp at system beskeden.
+
+## Training
+
+All the datasets were used seperately and merged together using [Model Stock](https://arxiv.org/abs/2403.19522), except for SystemChat-1.1 where I fine-tuned it using [LoRA+](https://arxiv.org/abs/2402.12354) with `train_on_prompt` set to True. 
+
+## Using
+
+```jinja2
+<|im_start|>system
+{{system_message}}<|im_end|>
+<|im_start|>user
+```