How do Image Optimize pipelines coexist with core's Image toolkit optimization?
With the Image Optimize pipelines, I'm not sure if core's JPEG quality
settings are still used: does the image get core's optimization plus
the Image Optimize pipeline?

Yes

does the Image Optimize pipeline override core's optimization?

No
According to the module's creator you should set your core's image toolkit to 100%; otherwise, you will endup with double compression and lower image quality.