Having an infection of the belly button is the most common cause of bleeding or discharge. Getting an infection in your belly button is surprisingly easy as it can become an ideal host for bacteria by providing a warm, moist and dark place for it to grow. Symptoms of infection: An infection can be spotted early on.