Just started a postdoc, but it went REALLY bad, REALLY fast. Stay or go?
With all else the same, given an choice between staying in a bad situation that cannot be resolved or leaving it, the decision is easy.

Of course, all is never exactly the same. When the balance is uneven, I hold at the point where you are that following your passions and desire for healthy growth is the most important factor. You can and will end up in situations where leaving has worse consequences than staying and you must therefore resolve the issues at hand or let go (and stay where you are). Is this case really to that point for you? I sense not.

As you weigh the various concerns, realize that you owe the lab only the respect that you will never be dishonest. The promptly, professionally delivered statement that "I am leaving because this is not the fit that I really wanted" is all that is required at the end of the day.