The cost of training GPT-3 is not officially disclosed by OpenAI, but there are some estimates based on the available information about the model size, architecture, data, and hardware. According to one estimate by Lambda Labs, it would take 355 GPU-years and cost $4.6 million to train GPT-3 using a Tesla V100 cloud instance. Another estimate by TechTalks suggests that it would take 175 GPU-years and cost $12 million to train GPT-3 using a TPU v3 cloud instance. These estimates assume a single training run and do not account for other costs such as development, maintenance, and electricity. The actual cost of training GPT-3 may vary depending on the optimization techniques, parallelization strategies, and hardware efficiency used by OpenAI.
spam lang_mismatch pii not_appropriate hate_speech sexual_content quality toxicity humor creativity violence