Earlier this week at work I got an email asking me to install Pex, Microsoft Research's White box Unit Testing framework for .Net. While my company falling into the "early adopter" category (we're already on .Net 4 and VS2010), this one surprised me since it's a Research project. I briefly looked into Pex last fall and didn't get too into it since I was content working with NUnit and my current unit testing tools. Now that it appears that I may be using it to some extent at work, I decided to take a more meaningful look at it.
Like I said previously, Pex is a MS Research project for doing White Box Unit Testing in .Net. White Box unit testing means that the tests are focused to cover the current code. This is one of the shining aspects of Pex; it will scan the code to be tested and generate unit tests that cover all logical paths within it. This is a great way to bring your code coverage up if you aren't getting the coverage rate with your traditional habits. After Pex scans and tests the code, you can drill into the tests themselves as well as the results. Pex allows you to update your own code with conditions specific for code under test too.(e.g. Pex can add null argument validation code if you so desire) When everything is all set with the tests that have ran, you can save them out into a separate or existing project so that they can be ran again later. In addition to all of these things, Pex comes wired up and ready to use an assortment of Unit Testing frameworks beyond MSTest so that it can play nicely with your existing tests and test runners also.
While those elements are awesome and helpful, there are a few things that turn me off of the framework. Pex will dive into all dependencies found with in the code under test. Where you traditionally may mock or stub out a dependency, if there's a hard dependency found, Pex will dive into it. Now, this can be averted using some form of IoC container or Isolation Framework (more on this below); however, if you don't use one of these, that code will be tested as well. In addition, Pex only tests Publicly exposed code. If your code is marked as anything other than public, Pex will inform you that it can't test it. This doesn't bode well for software businesses who have a lot of internal classes. During my tests, I even tried to use the InternalsVisibleTo assembly attribute and it still wouldn't work with it. Lastly, I was a bit disappointed in Pex only scans the currently active file inside of the IDE. While there's a command line way of running Pex, supposedly, from within Visual Studio itself, Pex can only work with 1 file at a time.
Finally, Pex has this nice little buddy called Moles. Moles is an isolation framework for handling dependencies. You can use either without Pex; however, they are closely tied together from a project and documentation standpoint (if you find info on Moles, you'll almost always find info on Pex and visa versa). Moles focuses on generating Stubs in the same way as frameworks like Rhino Mocks and NMock2; however, it also has what it calls Mole objects which focus on Sealed, Static, and or Privately scoped classes. I did not dive too deep into Moles; however, it would appear that it does not mock these types but rather intercept and detour the method calls into the implementations you wire up. From my understanding this is similar to how TypeMock handles the same task.
Overall, I think Pex is a good product for people trying to get their code coverage up and are in a Test-Last mentality. It works great under that situation and makes for a great candidate for automating unit test generation of DTO/Domain Model classes that tend to be public due to how they get passed around and usually consist of just getters and setters. It's not a tool I see myself using personally, though. I'm human and can see a very difficult argument in justifying a Test-First approach beyond design when Pex can/will write the tests for you. Applying that logic to a business environment and justifying a few minutes to write some tests to a manager vs a few seconds to run Pex and generate more tests is a hard sell.
No comments:
Post a Comment