An AI such as ChatGPT is an example of a Large Language Model (LLM). An LLM typically uses a transformer architecture, which employs the self-attention algorithm to consider context in long input sequences. The LLM predicts the next token in a sequence based on the context of the sequence so far. In this case, the sequence would be the conversation between user and AI.

It is very difficult to build an LLM from scratch. The process requires expensive hardware, huge volumes of data, and a great deal of time and expertise including the ability to write code. In the future services may be developed to build your own LLM without writing code. After training the LLM itself, it would also be necessary to to additional tuning, such as applying Reinforcement Learning from Human Feedback (RLHF) to enable the model to converse in human-like fashion.

One possibility to slightly reduce the expertise, time, hardware, and data requirements of an LLM is to fine-tune an existing, pre-trained model. Many open source models are available. However, this approach would still require writing code.
spam