Currently, for 2 billion parameters, it'll take $5 Million in just cloud computing costs to train a GPT-3 model from scratch.
Including the human labor and other infrastructure costs, this could easily balloon to 20% more, so total cost would be $6 Million to produce a GPT-3 model..
spam fails_task lang_mismatch pii not_appropriate hate_speech sexual_content quality toxicity humor helpfulness creativity violence