What is the logic behind "compute()" in Dask dataframes?
The core idea of delayed computations is to delay the actual calculation until the final target is known. This allows:

increased speed of coding (e.g. as a data scientist, I don't need to wait for every transformation step to complete before designing the workflow),
distribution of work across multiple workers,
overcoming resource constraints of my client, e.g. if I am using a laptop with limited memory, I can run heavy computations on dask workers that are in the cloud or another machine with more resources,
better efficiency if the final target requires only some tasks to be done (e.g. if the final calculation requires only a subset of the dataframe, then dask will load only the relevant columns/partitions).

Some of the alternatives to calling .compute are:

.visualize(): this helps visualize the task graph. The DAG can become hairy when there are lots of tasks, so this is useful to run on smaller subsets of the data (e.g. only loading two/three partitions of the dataframe)
using client.submit: this launches computations right away providing you with a future, an object that refers to results of a task being computed. This gives the advantages of scaling work across multiple workers, but it can be a bit more resource intensive (since dask doesn't know the full workflow, it might run computations that are not needed to achieve the final target).

With regards to distributed, I don't think there is a difference except for where the result will be: dask.compute will put the result in local machine, while client.compute will keep the result on a remote worker.
I'm struggling to understand when and when not to use compute() in Dask dataframes. I usually write my code by adding/removing compute() until the code works, but that's extremely error-prone. How should I use compute() in Dask? Does it differ in Dast Distributed?