Graduate in three years and then apply for Ph.D. programs?
If it really is a risk that you might not graduate at all if you stay on another year, then maybe you could consider graduating early and hedging your bets. There are other ways to catch up and make yourself both more prepared and more desirable to PhD programs of your choice. For example, you could try and apply for a research position if you see any openings in labs, and either take courses simultaneously or or as someone else suggested, apply to a non PhD program first. 

Either way, if there is an academic you know you would want to work with, it doesn't hurt to write them an email and ask them if they would be willing to take someone with only three years of college and if not, what would kind of skills and strengths they might be looking for. I did this when I switched fields right after college, and I was told that though I did not have quite enough qualifications, if I managed to do x, y, z I would be considered. That's what I ended up doing while working part time and two years later I got into the program (of course if you need a visa to work that's a different story).

However, if you can manage it financially, I would probably advise staying in college the extra year, and prepping for the GRE, taking courses relevant to your field, and getting those very important recommendation letters by making an impression on your professors. Also, most colleges have the option of doing a senior thesis, and if you want to go into a research career, while not a deal breaker, writing one can be very beneficial, no matter what area you write it in. Showing that you have the ability to do independent work is always a good thing when applying for doctoral programs. And it gives you a chance to work one-on-one with your advisors and can be as challenging and fulfilling as you make it! 

Everyone is different, so you will have to decide on your own, but I'm always in favor of a little bit more rather than less time between college and grad school to be both psychologically and academically prepared. But that's just me. Good luck to you!