Here are five ways to reduce your debt and improve your credit score. This is one of the fastest ways to improve your credit score, at least a little bit. Past due balances weigh heavily on your credit score. By paying them off, you can jumpstart your credit score quickly. While paying off very small past-due balances may not have much of a positive effect, paying off a larger balance, or several smaller ones, can jump your score by 20 or 30 points.