Sensor usage during manual control
The EV3-G programming language has mailbox blocks where it can receive bluetooth messages. You could write a program that until the touch sensor is pressed keeps processing the messages to control the robot.
The hard part here will be writing an app for the smartphone that sends the messages, however the BT protocol EV3 uses is documented and apps are already available implementing the messaging system, e.g. in C# here: https://github.com/jovabel/EV3Messenger
There's also the lego commander app, but I think the instructions sent by it bypass any program running on the EV3 brick, or rather they run in parallel so I'm inclined to think that the only solution is to write your own smartphone app...