At what point are AWS GPU instances worth it compared to CPU, price-wise?
The answer is it depends.
Are you referring to GPU vs CPU when training, or running inference?
When you say CPU is it an x86 CPU or ARM CPU?
What algorithms are running?
Let's look at image classification. Suppose you select a Convolutional Neural Network(CNN) as your algorithm. CNN's were created in the 1980s, but only really saw use when GPU's were used. In 2004 K. S. Oh and K. Jung showed that a GPU was 20x faster than a CPU. In 2005 Steinkrau et al showed a 3x speed increase.
Dan Ciresan et al showed in 2010 and 2011 that a GPU can be up to 60 times faster than a CPU. They used the MNIST handwritten digits benchmark. A simple google search of performance CPU vs GPU will result in many papers that show how much faster a GPU is compared to a CPU.
In this paper Eric Lind and Avelin Parnigoso compare the performance differences between a CPU and a GPU using TensorFlow. In their study they used a Core i7-6500 2.5 GHz CPU and an Nvidia GTX 1070 GPU. In their study they used three CNN AlexNet, Text Classifiction, and the Mnist Digit Classification.
Eric and Avelin showed that GPU outperforms CPU when training, and that during inference the GPU can be slower. In their study AlexNet for example ran for 4 000 000ms on a CPU and under 100 000ms on a GPU. That's a 40x difference in training performance. Converting this to cost. Eric and Avelin did not mention how much RAM.
I am going to make some assumptions and select the following instance types from AWS for training:

ml.g4dn.xlarge [GPU Accelerated] \$0.736/hour [4 vCPU, 16 GiB RAM]
ml.c5n.xlarge \$0.302/hour [4 vCPU, 10.5 GiB RAM]
ml.c5n.2xlarge \$0.605/hour [8 vCPU, 21GiB RAM]

The c5n's are powered by Intel 3.0 Ghz Xeon CPUs and the g4dn feature Nvidia T4 GPUs.
Based on this GPU Accelerated will run at less than 100 000ms or 100s while non GPU accelerated will run at 4 000 000ms or 4 000s. The result is:

ml.g4dn.xlarge - estimated cost \$1.226
ml.c5n.xlarge - estimated cost \$20.133
ml.c5n.2xlarge - estimated cost \$40.333

Based on the experiments showed by Eric and Avelin the CPU instance must be 40x less in cost to be on par with a GPU instance. As you can see the standard CPU is 20 and 40 times more expensive than the GPU. The thought experiment above is based on the paper times for training, not inference, and naturally to get a better answer should be benchmarked.
However, AWS has released their own ARM based CPU's designed specifically for training, and inference.

AWS Graviton Processor
AWS Inferentia
AWS Trainium

Both the Inferentia and Trainium chips offer 30% higher throughput and 45% lower cost per inference than Amazon EC2 G4.