How to turn down a Postdoc scholarship offer after signing a contract
I upvoted both of the other answers, but in my heart I agree with xLeitix: academics live and die by their honor in a much more extreme way than is the norm in the contemporary Western world.  I should admit that my views on this are more absolute than those of some of my colleagues (with whom I have had occasion to discuss issues of reneging on signed contracts).  When I hear people say that it is okay to behave absolutely unethically -- even illegally, but counting on the fact that your former employer is not going to hunt you down and pursue a surely thankless legal action -- because they have to look out for themselves first, it really worries me.  Such a person is baldly stating that ethics can be thrown out the window when the consequences are important...which is of course the time when ethical considerations matter most.

What stops a person who has reneged on a job from reneging on a future job?  What stops them from engaging in unethical academic behavior in any of a hundred different ways?  Beyond the student level, academia is almost entirely self-policing.  In my field, if someone releases a preprint, and I respond by saying that I have independently proved the result but hadn't released it publicly yet, there will be some followup questions and discussion, but unless I give them an excellent reason not to, people will believe me. In science, every time you write a paper describing an experiment you did, you don't submit a videotape of yourself doing the experiment, you submit a written description of the experiment...and people believe you unless they have an excellent reason not to.  Whenever you submit a paper, it gets sent to a referee, whom you trust not to try to steal the work.  And so forth: only a small amount of "justified unethical behavior" brings this crashing down.  

Once upon a time I was myself in the position of having accepted -- via an email correspondence, not via filling out a contract -- a postdoctoral position.  Just a few days later I got offered another postdoctoral position.  The first position was a great one, but it involved moving to Canada.  The second position was probably even better: it was at a top 10 math department in the US.  (Financially the second position was significantly better.)  But the first position was one that I hadn't even applied for originally; rather, my thesis advisor had made some phone calls and the offer came quickly: people had taken some trouble on my behalf.  What did I do?  I told the people offering me the second job that I was grateful for their offer but had to turn it down because I had already committed to another.  In retrospect, I still feel like the second offer might have been even better for me.  But a tremendously eminent mathematician went on the line and offered me a job when -- for a few days -- no one else would.  Anyone who knows me knows who this person is.  I can be a prickly person at times -- I certainly felt like the "loud American" during my 2.5 years in Canada -- and I imagine that this guy and most of his colleagues know me as slightly eccentric but professionally reliable.  Having people know you as professionally reliable is really priceless if you plan on staying in the profession: for instance, we later arranged for someone to transfer part of his NSF postdoc from this Canadian department to my own department.  

What should you do in academia if you want to back out of an agreement?  It's simple: you immediately contact the party you've made the agreement with, you explain why you want to back out of the agreement, and you see what they have to say about it.  They're not going to be thrilled, but they can convey to you how much hardship they will actually incur by your backing out of the agreement.  If it is only a moderate amount, they will probably give you their blessing and the agreement can be mutually dissolved.  If however backing out of the agreement would turn out to really be a significant, tangible, medium or long-term loss for them, then you should honor it.  The OP has done something in the comments which he surely does not realize sounds absolutely obnoxious to more senior personnel: he's made all kinds of assumptions that his reneging will not be so bad for his employer.  And he's making them in the face of evidence that this employer actually did go to trouble: applying for a fellowship is certainly trouble!  If this is an externally funded position then it cannot easily be filled with someone else, and the faculty supervisor may well already have made plans and committed resources for the OP's arrival.  Or maybe not, of course.  Ask!

Finally, the OP doesn't seem to understand why getting a tenure-track job offer might make a difference.  The difference is that every postdoc wants to get a permanent job eventually, and every postdoctoral supervisor and institution wants that out of their postdocs.  Therefore, if you ask to renege on a postdoctoral job because you'd like to take a tenure track job instead, it is much more likely that the response will be: "Bummer for us, but congratulations for you!" or "Well, can you come work here even for a very short time?  That will inconvenience us minimally, you'll get some further training and experience, and we'll be able to say that we placed our postdoc in a tenure-track job.  It's a win-win!" It is however still possible that reneging on a postdoc for a tenure-track job may be unacceptable to the employer.  If they tell you that, then you go back to the tenure-track job and say "Look, I really want this job, but I have some prior commitments.  I am the type of person who honors my prior commitments even when it conflicts with my short-term best interests.  Surely we can work something out?" It is likely that the position can be held open for an extra semester or year.