When doing comparative evaluation among different research tools should we debug others research tool?
You could do that, but then you wouldn't be doing a comparative study anymore. You would, in effect, be comparing one thing you created with something else that you modified. It would bring your comparison into doubt even if it didn't invalidate it. 

Report that your tool produces X and tool A produces no result for such cases. That is a valid comparison. 

But improvement of the tools of others can also be a valuable contribution, though of a different sort. Decide what it is you are after. 

You could also point out the flaws to the developers of the other tools and see if they wish to update. If so, you can compare with their new versions. 

You can also, perhaps, enter into a tool-building collaboration with some of the others.