6. Eating foods rich in sugar (bread, candy, cookies, pasta, soda, etc.) causes rapid increases in your blood sugar. This, in turn makes your testosterone levels plummet. Studies find that one sugar rich meal can cause testosterone levels to decrease by as much as 25%.