STM32F7 programming problems
I think that you mess two things. One is update of ST-Link firmware, the second one is programming the target micro.

The message shows that your target uC does not respond and the question why is very difficult. You need to connect all the VDD and VDDA to the power supply with the proper capacitors. All the grounds have to be connected as well. Check if you do not have any short circuits on your board. Check with the multimeter if you have not mistaken any connections.

There is no simple solutions - you need to check your board, connections, paths, joints - check if you do net have any hidden bridges etc etc

I would suggest to use the old st-link utility instead.