Saturday, May 15, 2010

My Thoughts on Pex

Earlier this week at work I got an email asking me to install Pex, Microsoft Research's White box Unit Testing framework for .Net.  While my company falling into the "early adopter" category (we're already on .Net 4 and VS2010), this one surprised me since it's a Research project.  I briefly looked into Pex last fall and didn't get too into it since I was content working with NUnit and my current unit testing tools.  Now that it appears that I may be using it to some extent at work, I decided to take a more meaningful look at it.


Like I said previously, Pex is a MS Research project for doing White Box Unit Testing in .Net.  White Box unit testing means that the tests are focused to cover the current code.  This is one of the shining aspects of Pex; it will scan the code to be tested and generate unit tests that cover all logical paths within it.  This is a great way to bring your code coverage up if you aren't getting the coverage rate with your traditional habits.  After Pex scans and tests the code, you can drill into the tests themselves as well as the results.  Pex allows you to update your own code with conditions specific for code under test too.(e.g. Pex can add null argument validation code if you so desire)  When everything is all set with the tests that have ran, you can save them out into a separate or existing project so that they can be ran again later. In addition to all of these things, Pex comes wired up and ready to use an assortment of Unit Testing frameworks beyond MSTest so that it can play nicely with your existing tests and test runners also.

While those elements are awesome and helpful, there are a few things that turn me off of the framework.  Pex will dive into all dependencies found with in the code under test.  Where you traditionally may mock or stub out a dependency, if there's a hard dependency found, Pex will dive into it.  Now, this can be averted using some form of IoC container or Isolation Framework (more on this below); however, if you don't use one of these, that code will be tested as well.  In addition, Pex only tests Publicly exposed code.  If your code is marked as anything other than public, Pex will inform you that it can't test it.  This doesn't bode well for software businesses who have a lot of internal classes.  During my tests, I even tried to use the InternalsVisibleTo assembly attribute and it still wouldn't work with it.  Lastly, I was a bit disappointed in Pex only scans the currently active file inside of the IDE.  While there's a command line way of running Pex, supposedly, from within Visual Studio itself, Pex can only work with 1 file at a time.

Finally, Pex has this nice little buddy called Moles.  Moles is an isolation framework for handling dependencies.  You can use either without Pex; however, they are closely tied together from a project and documentation standpoint (if you find info on Moles, you'll almost always find info on Pex and visa versa).  Moles focuses on generating Stubs in the same way as frameworks like Rhino Mocks and NMock2; however, it also has what it calls Mole objects which focus on Sealed, Static, and or Privately scoped classes.  I did not dive too deep into Moles; however, it would appear that it does not mock these types but rather intercept and detour the method calls into the implementations you wire up. From my understanding this is similar to how TypeMock handles the same task.

Overall, I think Pex is a good product for people trying to get their code coverage up and are in a Test-Last mentality.  It works great under that situation and makes for a great candidate for automating unit test generation of DTO/Domain Model classes that tend to be public due to how they get passed around and usually consist of just getters and setters.  It's not a tool I see myself using personally, though.  I'm human and can see a very difficult argument in justifying a Test-First approach beyond design when Pex can/will write the tests for you.  Applying that logic to a business environment and justifying a few minutes to write some tests to a manager vs a few seconds to run Pex and generate more tests is a hard sell.

Iowa Code Camp - Spring '10 Review


Back on the first of May, many from the .Net community in the Midwest flocked to Iowa City, IA in order to attend the Iowa Code Camp.  For those that aren't aware, Code Camps are community driven events where speakers volunteer to give presentation on a variety of topics for people to come and learn.  The Iowa Code Camp happens twice a year and every time it's been a great event.

This Spring's Code Camp saw me presenting my talk on the Managed Extensibility Framework (MEF).  The talk went fairly well in my opinion though we are our own worst critics sometimes.  The feedback I got on the talk was very constructive and gave me a lot of things I need to work on beyond what I was able to find from personal reflection.  I also ended up recording my talk using my Flip MinoHD camera which works very well other then the fact that it's hard to tell how the HD video truly looks on a tiny screen.  If you present, I highly recommend that you record yourself either in practice and/or presentation in order to obtain additional feedback on yourself; since you know the subject matter, you'll find more things about your presentation style and quirks that many evaluation forms miss.

Beyond my presentation, there were 4 other time slots/talks available for me to attend.  This specific code camp I wanted to focus on solidifying my knowledge of topics that I have pretty much taught myself.  While I was very confident in my abilities with the code-related topics, I wanted to see another person's perspectives of such so that I can gauge my own understanding and learn a few new tips as well.  Below is a rundown of the sessions I attended:


IoC with StructureMap - By Brad Tutterow

Inversion of Control and Dependency Injection have been topics that some people get and some people still don't fully see the value of it.  I got it pretty quickly because of my TDD experience; however, am always interested to learn new things about IoC containers  and the patterns in general.  Going into this talk I figured there'd be some discussion on the principles and patterns; however, there'd be a lot of discussion on how to use StructureMap.  What the presentation turned out to be was a jewel.  The polished presentation focused probably 75% on IoC and DI with the last 25% being about StructureMap.  Brad did an excellent job in not only explaining the "What" of the topics but the "Why" as well.  Looking throughout the audience, I could see some people finally understand it and it's importance.  In addition, Brad provided a great, evolving example that went from tightly coupled to decoupled to implementing StructureMap for fully handling all dependencies.  The time with StructureMap may have been less than I expected, but the value of the presentation as a whole completely outweighed my curiosity on the "How". 


TDD Dojo - By John Teague

I've been unit testing my own code on side projects for a while now.  I don't always push them in my blog post examples (I should though; I know); however, seeing a true TDD with pair programming session was something I wanted to observe and partake in.  In my opinion, a person's adoption rate of TDD and specifically Test-First is directly related to how close they are to finding that spot in Red-Green-Refactor where they write enough tests and code to feel meaningful.  I've seen a lot of people get turned off with the RGR pattern and TDD's principle of doing the bare minimum because it seems too simple and impractical; however, if you can find a level between writing all of the code and writing the bare minimum, usually they begin to adopt it more.

This issue is one of the things I felt from John's session.  John's session was broken up into 2 sessions and I only stayed for the first half.  In the first half, there was a wide assortment of experience levels among attendees.  Some people have been doing TDD for a while; however, there were a number that either rarely unit test or have never unit tested at all.  This range made some people question to simplicity of TDD's cycles and the perception of the impracticality of it all.  Despite what the session abstract said, I believe people were expecting slides on how to do TDD instead of a more hands on approach...the different format, which included pairing in front of everyone, seemed to make some uncomfortable.

With that said though, John did a great job in handling all of the questions and comments and an even better job in helping people pair up and begin to get familiar with TDD.  It was really good to see a talk session in a different format and a practical flow.  It was really good to see how attendees who finished their pairing time to come back with a "I get it" look on their face. 


Open Session

There were no open sessions on the schedule for Iowa Code Camp this time around.  I had every intent to continue with John's Dojo for the second half; however, a few of my friends were in the session had never written a unit test and were a bit overwhelmed.  When I ended up doing instead is taking one aside and worked with him on a one on one basis to help him understand unit testing, in general, a bit better.  One thing I wanted to make sure though was to use the same frameworks that John used in the first half of his Dojo since he covered the basics of seeing up a test class and test methods. The developer I worked with absorbed these aspects of John's session; however, was having a hard time seeing how to apply it.  For example: show a person who's never unit tested a test that validates a method returns true is great; however, they tend to get lost in the practicality of the test when the method body just reads "return true;".  I worked with him on an example of splitting a paragraph into various lines after a certain length, ensuring words were not broken as well.  My example was well; however, I didn't do so well on demonstrating stuff.  Towards the end though, he had written his first couple unit tests though and saw it working.  Now for me to go through more katas myself - practice makes perfect right?


Being a Technology Entrepreneur - By Scott Davis

This session was packed. I've looked into doing some side projects and freelance work in the past and this talk helped provide some insight on how I should go about doing such correctly from the start in the event that I wanted to go from simple freelancing to turning it into a true business.  Scott's advice was fantastic and the information he provided on the types of things to be mindful of was invaluable.  He kept everything relevant from things you should do when dealing with an attorney or accountant to what and how long it takes to setup a business with your local state.  His information on what types of businesses you should be looking at as well as how companies like Microsoft can help out as well.  Scott's talking again at the Kansas City Developers Conference in June and I may be attending the talk again since I know there'll be more things I didn't pick up on the first time around.


Learning Distributed Source Control with Mercurial - By Chris Sutton

Chris was kind enough to allow me record his talk despite the fact that I went to a different session (Thanks again, Chris!).  I have very little knowledge of Mercural (Hg) or Git so I was hoping to attend a talk on one of these while there; however, there were so many good sessions I had to resort to my recording.  Chris did a great job on presenting Hg.  He provided a great explanation of distributed version control systems and how they different from more traditional options like Subversion or Team Foundation Server.  His examples showed how to use Hg in a variety of tasks.  There were a lot of good questions during the discussion in terms of branching, merging, and comparisons which Chris fielded very well.  This talk wanted me to move all of my code to a dvcs instead of the systems I've been using up to this point.



After looking over my notes and the conversations I had surrounding the event; I wish it was longer.  Being able to bounce ideas from other developers and also focus on ways to make the event better was great.  Everyone involved in putting on the event was fantastic and for that Thank You!  I'm looking forward to (at least) attending the next Iowa Code Camp in November and see what may come from that.

Outside of the great event, there were a lot of things that I noticed.  The talks with the most interest were associated with either mobile development or Scott's Entrepreneurship talk.  Mobile is a hugely hot topic in the industry and the entrepreneurship talk's attendance matches what I've been seeing on Twitter and other conversations.  There seems to be a desire for information on how to be better outside of the code.  This is not to say that there should be talks on how to be a manager; however, how to be better at business, interactions with others, and skills that are important but not about really writing code.  I've been seeing this for a while; however I'm seeing the trend increase for things like Scott's talk as well as Brian H. Prince's Soft Skills talk.

In addition, I noticed the experience gap.  Now, I'm not expecting everyone to be super star developers; however, I see a need for something additional to be made available to help those understand concepts they may not fully understand before or even after a topic.  There isn't a magic bullet that provides the solution to this, but it's something that I am brainstorming on.  Should there be a 100% open spaces track dedicated to "mentoring" and helping people out with different topics or projects than just a discussion of random things like some open spaces turn into?  Should there be prerequisites suggested for every session?  Am I actually just imagining this gap?

Lastly, this code camp brought a lot of insight into myself.  After seeing the presentations and also this TED talk on how leaders inspire action, I'm going to be changing my MEF presentation around a lot. Brad did a great job in his presentation in building an understanding of why things are important and why he's passionate about such.  This is something that I felt was lacking from my presentation.  In addition, I'm hoping to get more involved in the near future.  Do I know specifically what that means, no but that's the beauty of community, people can fill where the need is.  In addition, I may be moving my presentation focus away from tools and into a more community focused array of topics; like Interviewing Skills, getting involved in Open Source Software, and ways to become a better developer and getting involved in the community.  Needless to say, there's no shortage of things I can improve upon with in myself and my abilities.  I can't wait to see how I'll grow and be ready for the next Iowa Code Camp.


Contact Information: