Compute Shader vs CUDA/OpenCL So I just recently learned about the Compute Shader and it looks from what I have picked up the same idea as parallel programming you would do with CUDA or OpenCL, but in the shader pipeline.

If I want to draw a million cubes in a scene should I be using one method over the other or both. If both how do you split that up so the GPU isn't trying to parallel compute both the shader and another process at the same time
It's not quite correct, today, to think of compute shaders as being "in the shader pipeline" in the same sense that your vertex and fragment shaders are literally hooked up into a pipeline. Compute shaders are not "hooked up" to anything currently, cannot drive rasterization, or directly consume the outputs of rasterization. 

What it allows you to do, though, is consume and produce memory resources also used by draw calls in a relatively efficient way. OpenCL makes this rather difficult (see for example https://software.intel.com/en-us/articles/opencl-and-opengl-interoperability-tutorial). This allows you to efficiently integrate compute passes within your renderer, but it is not integrated into the rendering pipeline