Standalone profiling of code (code coverage for third party testing tools)

If you come across  a requirement where you need to calculate code coverage for third party testing tools like nunit or cppunit or even mstest( for Mstest.exe it is a crude way as testsetting file is already there for it), etc in Microsoft TFS build.. then you should not worry as below simple steps would help you in completing the requirement… :)

Step 1: Create “In” and “Out ” folders in the “Anydrive:(builds folder):\.. testresults\codecoveragefiles” location.

Step 2: Copy Binaries folder to a separate folder on the build server. Here Binaries folder is the location from where you would like to execute your unit test cases.

Step 3: Instrument production binaries with below tool. Please ensure you instrument binaries at the location where you have copied your binaries folder. Let’s call this new folder as “Instrumentation”. The reason why you need to create a replica of binaries folder is to avoid instrumented binaries going in application package… :)

%(SystemDir)%\Program Files\Microsoft Visual Studio 10.0\Team Tools\Performance Tools\vsinstr.exe

Step 4: You would also like to sign your binaries with private key… This is optional..

Step 5: Now copy the content from “Instrumentation” folder to “Out” folder of test results…

Step 6: Start the perfmon from the below location…

C:\Program Files\Microsoft Visual Studio 10.0\Team Tools\Performance Tools\VSPerfcmd.exe

Step 7: Execute the unit tests.

Step 8: Stop the perfmon.

C:\Program Files\Microsoft Visual Studio 10.0\Team Tools\Performance Tools\VSPerfcmd.exe

Please go to “In” folder  you will find your data.coverage file…

Now let’s talk about the need to have standalone profiling…Sometimes you may come across the need to run testsuites of different products on to the same test machine. That means if you try calculate code coverage and more specifically at a time when both test suite run on the same machine (perhaps a single build running in parallel on the same machine) then you will find conflict, which would lead to build failure..

For this global unique identified is there for the rescue… :)

In Step 3. Instrument all binaries with a GUID. Use the same GUID for starting and stopping of perfmon…:)

%(SystemDir)%\Program Files\Microsoft Visual Studio 10.0\Team Tools\Performance Tools\vsinstr.exe “production-1.dll” /coverage /Exclude:”stringType” (optional) /DS: Guid

In Step 4:  Sn.exe –R “production-1.dll”  “private key”

In Step 6: C:\Program Files\Microsoft Visual Studio 10.0\Team Tools\Performance Tools\VSPerfcmd.exe /start:coverage –waitstart:10 /output:..testresults\codecoveragefiles\In\data.coverage /ds:Guid

In Step 8: C:\Program Files\Microsoft Visual Studio 10.0\Team Tools\Performance Tools\VSPerfcmd.exe /shutdown:400  /ds:Guild

One thought on “Standalone profiling of code (code coverage for third party testing tools)

Leave a Reply

Your email address will not be published. Required fields are marked *