Is supplying 5V from both external 5V and USB harmful?
No, it probably won't damage anything, as the PC's USB port is protected from overvoltage by the D1 schottky device on your arduino board.

Update, my previous answer contained the following paragraph, which is not related to your problem


  But you should take care of some things. 
  
  
  Make sure, the voltage applied to your +5V-Net is regulated properly elsewhere. I.e. choose a laboratory PSU or another good
  source. 
  Avoid having both your PC and your 5V-supply connected to line voltage. Ground levels may differ due to Y-capacitors present in the
  switching PSUs. Those provide a AC voltage of half your line voltage
  to DC ground (only with 0.3 to 0.7 mA, but enough to cause
  disturbances on your signals at 55 volts in the US or 115 V in the EU)
  If you have to use two switching PSU for both PC and arduino, provide an equipotential bonding between your PC's mass (e.g.
  VGA-port-screw) to your arduino's PSU's ground directly, to prevent
  leakage currents from the Y-capacitors wandering around in your
  circuitry.
  


Update according to your enhanced question

You are using a battery powered DC-DC supply for your Arduino. So, my previous notes do not apply to your problem. 


5.1 V seem exact enough to drive your arduino safely. No need for additional stabilisation measures
D1 (see genuine arduino schematic) separates USB-Power from +5V and prevents current flowing in reverse direction over your USB connection.
Arduino circuitry and USB port have common GND so there are always well defined voltage levels.


In my opinion, there's nothing with your setup, which may harm your circuitry or your PC.

Depending on the maximum power of your DC-DC converter you may consider adding a fuse to protect converter and components in case of short circuit.