How do comets' nuclei stick together?
Van-der-Waals forces and gravitation are the main answer.
The main property keeping together bodies smaller than a few $100 {\rm km}$ are their constituents surface forces between the grains (van-der-Waals) and chemical/ molecular bonding within the grains themselves. It's the same mechanic which makes the dust and grease in your flat stick to surfaces and the fluffy dirt clumps under the bed stick together, thus cohesion/ adhesion.
The typical internal strength of such dirty iceball as comets are, or rubble piles as some minor asteroids like Ruygu seem to be (e.g. Wada et al, 2018), does not need to be large - there is nothing in deep space which pulls it apart. And even the few µg of self-gravitation help to keep smaller fragments and ejecta to return to the parent body when hit by the inevitable micro-meteorites after their intial formation (Blum et al, 2017) (so yes, there is still self-gravitation involved which helps form them; they are just too small to achieve hydrostatic equilibrium and differentiation).
67P Churyumov-Gerasimenke provided a nice cometary physics lab which allowed in-situ investigation. The strength of the material could be analysed directy (Heinisch et al, 2018), (Grousin et al, 2018) with the lander Philae, even though it landed somewhat precariously (see also this overview by Blum et al (2018) focussing on planet formation and lessons learned from 67P in particular).
In summary: most measurement, both in-situ and lab experiments, indicate that the internal strength of comets is within the range of $1 {\rm Pa} \ldots  ... 10 {\rm kPa}$ and most likely at the lower end of this range. Their strength is thus determined by their small gravity and the inter-particle forces, thus van-der-Waals which in part is possibly enhanced a bit by sintering together the different constituents grains.