In Google Photos, when I try to delete a photo or video on SD card it tells me to go to gallery I have 2 Mi note 4g's and both of them have the same problem. I have also given access to all permissions in settings(though there is no permission for SD card). All the photos outside the SD card can be deleted all right.
Your issue is you haven't allow SD card access from the app itself:



Hamburger menu (Three dots) in the upper right. 
Settings 
SD Card Access


Regards.