Appropriate solenoid valve flyback diode
I have a 1N4007 diode parallel to the solenoid.

No you don't - not according to your schematic - the anode is connected to ground: -

The flyback diode needs to be in parallel with the solenoid. Look where you have it connected - it is ineffective how you have connected it. If you want to get rid of the capacitor fix how you connected it - i.e. place it where you have connected the capacitor (cathode to +24 volts). 1N400x is fine for this unless you are using anything significantly faster than 1 kHz PWM or the solenoid is taking more than an amps.