Hard and noisy Shimano ST-EF29 front shifter with brake lever
My initial suspicions would be either the indexing is slightly too far outboard, so that the cage is not quite moving far enough to push the chain positively down to the smaller rings; or friction on the shift cable within the housing is delaying or impeding the inboard movement of the cage on downshifts.

Try adjusting the indexing. Front derailleur adjustment is a bit tricky so I always follow Park Tool's front derailleur process.

Check the shift cable is running freely. Detach the cable from the derailleur, pull on it to provide so tension and work the shifter, the cable should pay out easily and smoothly on downshifts.