How much does a FAIL grade in thesis credits affect PhD admission chances?
To be successful in applying for PhD programmes, you need to demonstrate your ability to perform research.

Unfortunately, failing a thesis research course says rather the opposite to the admissions committee. By the sounds of it, your lukewarm recommendation letter from your supervisor will have done nothing to help matters either.

To improve your chances if you are planning to apply again, I would suggest trying to publish your work (you hint in your question that this may be possible) and, if you can, finding someone else who can write you a much stronger letter of recommendation.