The Internal Revenue Service says you can figure out whether you have to pay taxes on Social Security by adding half of your annual benefit to your other income. If the total is more than $32,000 and you file a joint return, some of your benefits must be reported as taxable income on your tax return. For single filers the threshold is $25,000. For example, if you file as single and your annual benefit is $15,000, half of your benefit amount is $7,500. You you can earn up to $17,500 in additional income before you hit the $25,000 mark and have to pay taxes on Social Security benefits. Taxable Percentage When Social Security is taxable at all, you have to pay taxes on at least 50 percent of your benefits. As your income increases the percentage can reach a maximum of 85 percent of your annual benefit amount.