Big heatsink vs. small one with a fan
It all come down to thermal resistance. A larger heat sink will have a lower thermal resistance vs a small heat sink. Air flow will lower the thermal resistance of a given heat sink. What you need to do is calculate what thermal resistance you need by finding out the power dissipation and maximum junction temperature of the part you are trying to cool. Generally you want some margin below the maximum junction temperature. Once you know the thermal resistance you need you can select a heat sink and air flow that will meet it.