Does using your test set ultimately burn your data set in case of failure?
I would try one more time to split the data &amp; train a new algo. This is similar to p-hacking as you've stated so try to avoid doing this more than once but if the alternative is the project stops there doesn't seem to be much more you can do.
This time when you split the data I would try stratifying it by some feature (the target? Some category of the input variables?). This ensures the training/validation/testing sets have the same proportion of some criteria which might help your performance be a little more consistent when comparing the validation set to the testing set.