<story><title>LoRA from scratch: implementation for LLM finetuning</title><url>https://lightning.ai/lightning-ai/studios/code-lora-from-scratch?view=public&amp;section=all</url></story><parent_chain><item><author>r3trohack3r</author><text>I feel like it&amp;#x27;s the difference between something that has been engineered and something that has been discovered.&lt;p&gt;I feel like most of our industry up until now has been engineered.&lt;p&gt;LLMs were discovered.</text></item><item><author>rsweeney21</author><text>It&amp;#x27;s still strange to me to work in a field of computer science where we say things like &amp;quot;we&amp;#x27;re not exactly sure how these numbers (hyper parameters) affect the result, so just try a bunch of different values and see which one works best.&amp;quot;</text></item></parent_chain><comment><author>herval</author><text>LLMs were very much engineered... the exact results they yield are hard to determine since they&amp;#x27;re large statistical models, but I don&amp;#x27;t think that categorizes the LLMs themselves as a &amp;#x27;discovery&amp;#x27; (like say Penicilin)</text></comment>
<story><title>LoRA from scratch: implementation for LLM finetuning</title><url>https://lightning.ai/lightning-ai/studios/code-lora-from-scratch?view=public&amp;section=all</url></story><parent_chain><item><author>r3trohack3r</author><text>I feel like it&amp;#x27;s the difference between something that has been engineered and something that has been discovered.&lt;p&gt;I feel like most of our industry up until now has been engineered.&lt;p&gt;LLMs were discovered.</text></item><item><author>rsweeney21</author><text>It&amp;#x27;s still strange to me to work in a field of computer science where we say things like &amp;quot;we&amp;#x27;re not exactly sure how these numbers (hyper parameters) affect the result, so just try a bunch of different values and see which one works best.&amp;quot;</text></item></parent_chain><comment><author>SkyMarshal</author><text>If the Black Swan model of science is true, then most of the consequential innovations and advances are discovered rather than engineered.</text></comment>