How to remove my Mac from the speaker choices?
First off, it's not a &quot;speaker list,&quot; it's list of devices that can take or make phone calls.  If you paired a Bluetooth headset it would show up there as will your car's Bluetooth connection (if you have one and it's paired).  This is probably what's causing your confusion when trying to disable it.
So, to do so, you have to turn off WiFi calling on your &quot;other&quot; devices
Settings → Phone → Calls on Other Devices.  There will be a section called Allow Calls On.  Toggle off the device you don't want to allow access, in this case &quot;Jeffs Mac.&quot;  Once you do that, it will no longer show up as a device to send the call to.
Below are screen grabs from my iPhone that's configured to use my Mac as an audio device for phone calls.  Upon disabling it in Calls on Other Devices it's removed as an audio option on the call screen.