How can I save my mussels?
Short answer:  It is safe, although by far the predominant practice is to remove them before cooking.

This is, of course, assuming the "bits of stuff" you refer to are the beards of the mussels, which they use to attach themselves to their growth substrate.  



According to ISSC, International Shellfish Safety Conference, you can leave the beards attached if you wish (emphasis added):


  HOW SHOULD FRESH MUSSELS BE HANDLED STORED AND COOKED?   
  
  Mussels
  should be scrubbed with a stiff brush or coarse sponge and rinsed with
  cold water to remove any debris. Beards can be removed or left
  attached.


Lesser quality sources concur:

According to Blue Ocean Mussels, you can remove the beards after cooking--assuming the bits of stuff you refer to are the beards.

According to Yahoo answers, its safe but I give them little credence.



Here is one article at The Kitchn describing how to prepare mussels and clean them, with photos, in case it helps in the future.  They also note, by the way:


  The beard isn't harmful or inedible (just not particularly desirable
  to our taste buds), so don't fret if there are a few little threads
  left that you can't grasp.