Is there a way to control the app and notification volumes at once on a phone like there is on a tablet?
That depends on your Android version and device, but based on what you said it seems like you should be able to. You can try Volume in Settings, or you can download Volume Control and control various different volume levels and settings.

Several attempts to get this are posted here, but it doesn't seem like some devices have it.