Should I ground the heatsink?
By the way, transistors(npn&amp;pnp) are used in a class B audio amplifier.


This is an important detail. The transistor tab is usually connected to the Collector pin, and your standard insulator will add about 15pF capacitance between the tab and the heat sink, so this is about knowing whether this capacitance can cause problems or not.

If the heat sink is grounded, then the collector will have 15pF to ground capacitance in series with the connection inductance. Then the heat sink will also capacitively couple to whatever traces or signals are around.

If your amp uses an emitter-follower output stage, then both power transistors collectors are connected to power supplies (or ground if it is a single supply amp). Since power supplies are decoupled to ground, there is already a low impedance HF path between collectors and ground, therefore grounding the heat sink should make no difference. However, if the heat sink is external, or part of the enclosure, and the enclosure is already grounded at another point, grounding the heat sink on the amp pcb will introduce a ground loop.

If your amp uses a common emitter output stage, then the collectors are connected to the output, and the capacitance will be between output and heat sink. Some amps omit the insulator for better thermal resistance and have the heat sink connected to the output voltage, but this requires it to be inside an enclosure and not accessible. Anyway, if you ground it, then you have to consider if the extra 15pF to ground could make your amp unstable. For an audio amp, this is unlikely.


  when I inspect the signals in the circuit via oscilloscope, I see there is some noise on waveforms, while trying to understand where it comes from, I have touched the heatsink (...) It becomes much smooter.


If the output voltage appears fuzzy on the scope, your amp could have stability issues. You can try single shot mode and see if you observe oscillations or just noise. If the amp does have stability issues, they are probably not related to case-to-heatsink capacitance, the problem would come from somewhere else, like layout or decoupling.