drush fails to install drupal properly
Here is the answer to the riddle.
My live VM is on an older version of Debian, different document root, different apache2.conf.
It is much easier to edit apache2.conf in the newer version, and that is why it was working:

    Options Indexes FollowSymLinks

AllowOverride None

AllowOverride All
Require all granted



In older version, I copied apache2.conf to something else and replaced Deny for Allow, like so:

    Order allow,deny
    Allow from all
    Satisfy all

.
Completely different, I am mulling to move server to testing version, in spite that everybody advices against it.