iPhone: is it possible to route calls to speaker only if answering via smartwatch?
No, that's unfortunately not currently possible.
As you indicate, you can use the Call Audio Routing functionality in Accessibility to always route any call to speaker. However, you cannot do that just for calls answered through a Bluetooth-enabled smart watch.