First-time homebuyers can buy a home with a minimum credit score of 580 and as little as 3.5 percent down or a credit score of 500 to 579 with at least 10 percent down. Unfortunately, you'll need to pay private mortgage insurance, or PMI, with FHA loans.