Until your baby's umbilical cord stump falls off: 1  Keep the umbilical cord stump clean and dry. 2  Fold the baby's diaper away from the stump (or buy newborn diapers with a cut-out space for the stump) so it is exposed to the air and won't come in contact with urine.3  Give your baby sponge baths instead of tub baths. Keep the umbilical cord stump clean and dry. 2  Fold the baby's diaper away from the stump (or buy newborn diapers with a cut-out space for the stump) so it is exposed to the air and won't come in contact with urine.