add Throughput, New Discoveries

README.md

@@ -8,7 +8,15 @@ Hardware requirements for ChatGPT GPT-4o level inference speed for the following
 
 Note: The following results are based on my day-to-day workflows only. My goal was to run private models that could beat GPT-4o and Claude-3.5 in code debugging and generation to ‘load balance’ between OpenAI/Anthropic’s free plan and local models to avoid hitting rate limits, and to upload as few lines of my code and ideas to their servers as possible.
 
-By a complex debugging task, I mean scenarios where you build library A on top of library B that requires library C as a dependency but the root cause was a variable in library C. In this case, the following workflow guided me to correctly identify the problem.
+An example of a complex debugging scenario is where you build library A on top of library B that requires library C as a dependency but the root cause was a variable in library C. In this case, the following workflow guided me to correctly identify the problem.
+
+<br>
+
+## Throughput
+
+![](./model_v0.1_throughput_comparison.png)
+
+IQ in model names mean Imatrix Quantizations. For performance comparison against regular GGUF, please read [this Reddit post](https://www.reddit.com/r/LocalLLaMA/comments/1993iro/ggufs_quants_can_punch_above_their_weights_now/).
 
 <br>
 
@@ -41,10 +49,12 @@ Think step by step. Solve this problem without removing any existing functionali
 
 <br>
 
-## Debugging with Reflection
+## New Discoveries
 
-The following are personal opinions.
+The following are tested, but may not generalize well to other workflows.
 
-In general, if there's an error in the code, copy pasting the last few rows of stacktrace to the LLM seems to work.
+- In general, if there's an error in the code, copy pasting the last few rows of stacktrace to the LLM seems to work.
+- Adding "Now, reflect." sometimes allows Claude-3.5-Sonnet to generate the correct solution.
+- If GPT-4o reasons correctly in its first response and the conversation is then sent to GPT-4-mini, the mini model can maintain comparable level of reasoning/accuracy as GPT-4o.
 
-Adding "Now, reflect." sometimes allows Claude-3.5-Sonnet to generate the correct solution.
+<br>