Currently empty test results are just a warning, but it would be great if they could be configured as failures. In the case of phpunit, when it starts running, it creates an empty xml file. So if there is a fatal php error, it will just result in an empty, but new, file, and xUnit will happily warn but ignore this problem. For us it makes sense to always consider these errors, but maybe there should be a checkbox for "Fail the build if there are empty test files" or something.
Attached is the patch we are using locally to accomplish this. Along with the previous patch, it makes our xUnit runs really robust and is the only way to ensure nothing is going wrong with the builds.
In my opinion, we can't set a fail status for one empty test result file.
Nevertheless, if there are only empty test result files, set the result status to failed would be great.
Regarding adding a checkbox, I think that's would be good idea but there already lots of checkboxes. Adding new checkboxes could make the plugin too complex.
Gregory Boissinot added a comment - 2010-04-23 13:16 In my opinion, we can't set a fail status for one empty test result file. Nevertheless, if there are only empty test result files, set the result status to failed would be great. Regarding adding a checkbox, I think that's would be good idea but there already lots of checkboxes. Adding new checkboxes could make the plugin too complex.
Hm yeah, I can see your point. In our case though with phpunit fatal errors it is absolutely a failure and we want to consider it as such, and I can see where others who try to keep a clean/robust build would consider that a failure as well.
Can you think of a way to have it such that we don't have to apply a patch on each release, while keeping the plugin to your liking? You don't find it worthy of a configuration option?
mcrooney added a comment - 2010-05-04 14:09 Hm yeah, I can see your point. In our case though with phpunit fatal errors it is absolutely a failure and we want to consider it as such, and I can see where others who try to keep a clean/robust build would consider that a failure as well. Can you think of a way to have it such that we don't have to apply a patch on each release, while keeping the plugin to your liking? You don't find it worthy of a configuration option?
I need to include it in the new xUnit architecture.
Gregory Boissinot added a comment - 2010-07-15 10:08 Ok for the option. I need to include it in the new xUnit architecture.
Are the possibility to test it?
Gregory Boissinot added a comment - 2011-02-13 00:55 Available in trunk? Are the possibility to test it?