When using a USB-C monitor with power delivery, does constantly having it plugged in damage the battery?
No, having it constantly plugged in won't damage the battery.

The MacBook contains power management circuitry that ensures that the battery is not "overcharged".

So having a USB-C monitor that charges your MacBook plugged in at all times is no different from having an ordinary charger plugged in at all times.