Where does the drush installtation command set the PROMPT_COMMAND variable?
The function that changes the .bash_profile or .bashrc file is drush_init_core_init(). The shell script that effectively sets the prompt is $home/.drush/drush.prompt.sh, where $home is the value returned from drush_server_home(). If the file doesn't exist, drush_init_core_init() creates it from the content of DRUSH_BASE_PATH . "/examples/example.prompt.sh", which contains the following line. (See example.prompt.sh.)

export PROMPT_COMMAND='__git_ps1 "\u@\h \w" "$(__drush_ps1 "[%s]") \\\$ "'


There is also the following line, which sets PS1 and that is is commented out by default. (See the comment before it.)

  # PROMPT_COMMAND is used in the example above rather than PS1 because neither
  # Git nor Drush color hints are compatible with PS1. If you don't want color
  # hints, however, and prefer to use PS1, you can still do so by commenting out
  # the PROMPT_COMMAND line above and uncommenting the PS1 line below:
  #
  # export PS1='\u@\h \w$(__git_ps1 " (%s)")$(__drush_ps1 "[%s]")\$ '


As for the error message you see, __drush_ps1() is a bash function used to change the prompt, defined in drush.complete.sh. bash completion/prompt broken explains exactly when and why you get that error. TLDR; in some circumstances, the shell script is not sourced, and the Bash function results not defined.
For what I understand it, the error has been has been fixed in recent versions, though, so you should just need to update Drush to fix the error.