Can two IRLZ44N MOSFETs in parallel handle a current of 20 A without heat sink?
Consider the scenario without a heatsink first: -

The junction-to-ambient thermal resistance for the IRLZ44N is 62°C/watt
The on-resistance of the IRLZ44N is 0.022 Ω with a 10 volt gate-source drive voltage.

This means that if each is taking 10 amps drain current, the power dissipated is 2.2 watts in each. From that power we can conclude that the MOSFET junction will warm by 136.4°. But, given you are operating them on a 50% duty cycle, the average junction temperature will warm by 68.2°C.

The maximum junction temperature for the IRLZ44N MOSFET is 175°C (take note)

But, you have things fighting against you: -

Ambient temperature around the MOSFETs may begin at around 25°C but will rise due to heat not adequately being taken away. Only you can determine this but, it might be sensible to assume that in an ambient of 25°C, locally it may rise by another 40°C.
Both MOSFETs will not share current equally; you might find that one of them takes two-thirds of the current hence it will dissipate more power.

So, concentrating on the MOSFET that might have an on-resistance of only 0.015 Ω, it might take 13.3 amps and hence its power dissipation is 2.653 watts. Hence it's junction will warm by over 82°C on a 50% duty cycle. Given that the local ambient may become 25°C + 40°C, the junction might see a temperature of: -
82°C + 25°C + 40°C = 147°C and this doesn't leave a lot of comfort-factor.

Do I need to attach big heat sinks?

So, you have the math now, so you can decide.
I'd attach moderately sized heatsinks of circa 30°C/watt thermal resistance and make use of the MOSFET's junction-to-case specification of 1.4°C/watt. Net thermal resistance would be 31.4°C/watt and a much nicer scenario.
One of these: -

Is rated at 24.4°C/watt and so should be good-enough.