I'm trying to get MbUnit to play nicely TeamCity 3 (so it will display Tests passed / failed and show statistics as it does with JUnit). My projects are all MSBuild projects and they current run MbUnit correctly and generate an html report that i can include as an artifact.
From reading around the docs, my plan was to generate a teamcity-info.xml with the MbUnit statistics (in the hope that TeamCity would automagically pick them up and process them).
Which seems to be automatically included as an artifact by TeamCity. However, there doesn't seem to be any processing beyond that. Am I approaching this in the correct way, and if so, what properties do I need to put in my teamcity-info file to get this to work?
Aside from this issue I'm very impressed - we've been using CCNET for the last couple of years and this has been far easier to get up and running and extend!
The part you are missing is at the very bottom where you need to add statisticValue elements inside the build element.
Tests passed: 2 * * You also need to edit the TeamCity data directory/config/main-config.xml file to include the new stats you want to track. So if you want to add the 2 keys above you would add: The ]]> element that you specified will just modify the build status text (usually just says "Successful", "Failed", etc).
I had looked at the page but couldn't work out whether there were some standard keys to make this work or whether i had to define custom keys. Anyway - all looks good now!
The graphs are now showing up in the statistics tab but they don't display any data! The status labelling doesn't appear to have any effect. Is there something else I need to do?
In fact, there is a nasty bug with teamcity-info.xml processing, and this bug was fixed right after the last EAP was released. Your setup looks correct, but I'm afraid you need to wait for the next EAP build (which hopefully will be released on the next week).
I have MbUnit working, yes. The simplest way to do it is just to ensure MbUnit runs as part of the build script (we use MsBuild to run our .csproj projects and have MbUnit run as part of that using the MbUnit MsBuild task) - this generates a html file which you can include in the build artifacts for you TeamCity build configuration.
If you want to go further than that, as we have, you could get your build script to produce a teamcity.info file including the number of tests passed, failed, etc. To get the numbers out of MbUnit requires a patch to the MsBuild task (which you can find on the MbUnit Jira site or I can email to you) and we've written a couple of MsBuild tasks to produce a teamcity.info file which we could also make public.
Once thats done, the test statistics can be viewed directly in the build history in TeamCity.
I think I'll put a blog post up about this as it's a little involved, but the end result is quite nice. I'll post here again when it's up.
teamcity.info file which we could also make public.
Does your custom change show passed,failed,new counts under the projects tab?
I currently use NUNIT for some .NET testing. But I want to use a framework called zanebug and still have all the stats and numbers show up in TeamCity like NUNIT results do currently.
The output looks like this under the project tab for NUNIT.
And this is how I want zanebug to look.
Sorry I meant to reply to the parent node of this discussion.
Y I made an MsBuild task to generate the teamcity-info.xml which is parsed to display these stats - I'll put them on my blog soon but have been busy this week and haven't got around to it.
Once you have the teamcity-info.xml being generated in the build working directory, status messages get automatically added to the dashboard. You can also put stats in there which can be used in graphs by editing the server configuration files.
Can you please give a link to where you upload the file.
I actually just started working on a gallio teamcity testrunner. I was going to just do a mbunit runner but with gallio being able to run multiple testing frameworks and being able to do some cool reporting I figured I would target that. I am hoping to have something in the next couple weeks. I need to check about licensing, but it will probably get included as an opensource module inside of gallio.
It will give you rpc style reporting like the nunit teamcity testrunner, include standard team city test reporting and charts, as well as add a "Test Details" tab with whatever extra reporting you decide to configure from the gallio runner.
One problem I will have is using it in the build. I don't think I will have the luxury of patching the build files like current TC runner does for NUnit. Maybe I can work something in with the standard Gallio runners and if it detects its in a TC build it can just add my TC logger and progress monitor.
I will post here when I have something working.
Adam - it may be a dirty hack but you can look for the presence of the teamcity environment variables as an indication that you're running in TC. I'm working on putting a blog up post about MbUnit integration with TC as soon as I get the time and also adding the MSBuild tasks to output teamcity log files to the MSBuild Community Tasks project. If you want any help with Gallio to integration, drop me an email.
an indication that you're running in TC.
I wouldn't call it dirty, that is what I was planning on using. If it detects them it will just append my tc rpc test logger and progress monitor.
I was going to do the same with ncover, I just want to figure out how to get better charts first.
Not sure what you mean about teamcity log files, currently tc wraps msbuild so all logging gets captured and sent back to the build server.
If you want any help with Gallio to integration, drop me an email.
Thanks Will, I am take you up on that. I just want to spike something first and get into the gallio code base, then everyone can work on patches for it. I already talked to Jeff Brown and he wants to include as part of the Gallio base if licensing permits.
Please do not hesitate to contact TeamCity Team if you have any questions while your work on the plugin.
I've added the MSBuild tasks and a blog post at http://bluewireblog.wordpress.com/2008/01/24/teamcity-msbuild-tasks/ - let me know of any probs.
I started the gallio test monitor, from my end its pretty complete I just can't get it to work with TC. I sent an email asking jetbrains for help.
I created both an rpc monitor and a console monitor for the new service messages in tc 3.1. I can't get service messages to work under msbuild and I have no idea where or why my rpc calls are failing. As far as I can tell its the exact same calls and xml as the jetbrains nunit runner.
Hopefully we will get something soon.
Bumping this... I sent an email last week to support asking for some help in tracing and debugging rpc calls to the TC build agent service.
Any word? Below is a copy of the email.
settings to get more verbose output.
same cookcomputing.xmprc.dll. But nothing seems to get captured and logged.
however if I run local with /v:diag I see the Console.Out messages.
It will be oss so I have no problem sending you my code either.
resend it to you again.
in tracing and debugging rpc calls to the TC build agent service.
Ah... if it was sent to gmail it probably bounced if there were any binaries in it.
I've forwarded email to the new address.
Did you try using the XSLT support on the MbUnit console?
I was thinking of going this route instead, although I currently have the NANT custom task running MbUnit just fine.
Adam, you have mentioned you've been working on gallio test runner. Is that available somewhere? Do you have some problems doing it? Have you managed to get any positive result?
If you're using NAnt and MbUnit with TeamCity you may want to take a look at the NAnt-Extensions project.
It includes a NAnt task for MbUnit that integrates nicely with TeamCity.
The project also includes additional tasks that make it easy to interact with TeamCity from a NAnt build script.