Explain how it might be possible to host a large learning model on an average computer. Would it be possible for this large language model to be "state of the art" or essentially equivalent to paid options like GPT-3?
user
If by "large", you mean models with a higher amount of parameters, then No problem. to disappoint you.... it doesn't seem that likely for it to be possible to host them on an "average computer". At least not in the next five years. As of March 2023, the average computer around the world is the smartphone. The graphics processing on many high end computers would be inadequate to run large models. We would need radical advancements in GPUs to run something like GPT-3 on a high end consumer computer, let alone a smartphone. Such advancements in GPUs seems to be highly unlikely in the next 5 years at least, simply because the laws of physics are preventing us from manufacturing faster chips. I can tell you more about this if you are interested.  

However, what is looking to be possible is making lower parameter models much more capable. Again, as of today, that is March 2023, a lot of research in this field shows that models with lower parameters can be fine tuned to be extremely capable. However, it is too soon to tell whether these models would be equivalent to Gpt-3 or not. There is hope however. 

These smaller parameter models also can be run on higher end consumer hardware. We have seen this trend with other machine learning models like Stable Diffusion, where capable models could be run on relatively lower end hardware over time due to further fine tuning. It is quite likely that this trend would apply to languages models too! Only time will tell!
assistant
With average computer, I meant medium-end laptop. How long would a reply take on such a machine?
user
Got it, thanks for clarifying. To summarize:

Running moderately sized language models (millions of parameters) on a medium-end laptop is feasible today, but response times would likely be on the order of seconds, not real-time. The exact latency would depend on various factors, but sub-second response is unlikely with current hardware and algorithms.

As technology improves over time, response times could decrease and larger models could become practical on consumer laptops, but we are still quite far from running models equivalent to GPT-3, if ever. Significant hardware and software advances would be required to reach that scale.

If real-time response is a priority, it may be better to focus on innovations to make smaller models more capable, such as through training, pruning or other techniques. More progress is needed, but that could be a more promising path than just scaling up model size given current limitations.

Does that help summarize the key points regarding running language models on medium-end laptops? Let me know if you would like me to elaborate on any of the areas or address other questions.
assistant