It is always almost always on a Monday, when you fix that long standing (10 minutes) production bug that requires you to stand on your head, during a blue moon while balancing an egg on your foot to get it to render an image 1 pixel to the left when you find that someone just deployed code that broke every thing… except the image always renders in the correct spot. What broke and why?
That is why you want unit tests. Being able to run unit tests helps you understand why things broke when something as simple as changing ++x to x++ makes a difference. Having a whole suite of tests that can check your code base can save hours of testing to ensure things still work after your changes and before the testers get their hands on the next release. It is not a silver bullet to stop all errors, but it helps.
It should come as no surprise that I encourage people to write unit tests and I often get questions like “do you use nunit or mstest?”, “do I really need to write negative unit tests?”, “should I write unit tests or integration tests?”, “which is better Unit testing, TDD or BDD?” The problem is all of these question are really a matter of opinion and sometimes we polarize them to extremes where we end up paralyzed trying to make the “right” decision.
Let me answer all of these question with one sweeping answer – it doesn’t matter. Don’t get hung up on tooling or methodology. Tooling changes and new ways of testing come and go. There is no one size fits all and what works for any given developer may not work for the next one. What is important is that you write tests.
With that said, let me share my skewed opinion...
Do you use NUnit or MsTest?
I use MsTest. Yes, MsTest takes a “kind of a one size fits all” approach to everything. I have heard the debates of it is a jack of all trades and a master of none and I have seen NUnit and XUnit in action. The thing is I have never really had to fight with configuration issues, IDE plugins, or getting the “right” test runner on my build server when I use MsTest. MsTest has never failed a test because of a bug in it or the test runner and I have yet to find something I could not test in MsTest that I could in NUnit. That is not to say that such issues are not possible with MsTest, but I have yet to encounter them. I have run into –some of those- issues with other test frameworks. I like having my unit tests in my IDE and with Visual Studio I don't have to do anything to get MsTest to work. There are Visual Studio extensions for other test frameworks, like NUnit, that are free or have a cost with them, but I haven't really found a compelling reason to switch. Also, if you happen to have new or not-so-technical Developers using MsTest lowers the “day 0 on-boarding” and learning curve.
So there you have it, I use MsTest because it is there, easy, and it just works.
Do I really need to write negative tests?
Yes and No. Write tests that make sense. Cover the “happy path” and the use cases that you expect; if you expect invalid inputs –because… you know… users – write negative tests. If you try to write a unit test for every possible scenario you’ll go nuts and never get anything done. Don't forget the company wants working software on time and there are two things they will cut if they think it is taking too long- unit tests and developer documentation. Yes that might mean you will encounter bugs that you could have caught if you had those extra tests - It’s possible. But, I have found that by keeping my tests focused on the happy path for the unit of work under test that I get things done quicker, I have a base line of unit tests and the company gets their software on time.
Negative tests have their place and I write them when they make sense –because user input should never be trusted-, but I do not go out of my way to test that I get a DivideByZeroException when there is division within the unit under test.
In case you were wondering what a a negative test is here is a handy link that can help you in your research http://www.aptest.com/glossary.html
Should I write Unit Test or Integration Test?
Yes. That is not really an either or question; they are both useful in different situations. However, integration tests should not be used in place of unit test. Use interfaces in your code to abstract the integration away from your code and a mock framework to simulate that integration in your unit tests. If you can not do that I suggest revisiting your codes structure to make it more testable. On a side note I do not run integration tests as part of CI build. Just because Mark unplugged the payment gateway in the datacenter so he could brew a pot of tea does not mean I will get a 418 status code back, that I should be looking for it or that it should fail the build. However, I do schedule my integration tests to run on the last successful build every x days
When a bug occurs I create a new test for that bug, include the URL to the issue in the tracking system in the remarks ///<remarks></remarks> and we are off to the races. When the bug is fixed I can run all the tests and make sure I did not break anything,
Which is better Unit testing, TDD or BDD?
This question always bugs me because it confuses a best practice with a methodology. Unit Testing is a “best practice” while TDD and BDD are methodologies. If you are doing TDD or BDD you are by nature doing Unit Testing. <opinion>The methodology used -TDD or BDD- does not matter nearly as much as Unit Testing. If you are not already writing tests just start and you are are already ahead of the curve</opinion>. The problem with unit testing is the code base has to be written in a way that it is testable. For developers new to unit testing this can be extremely difficult and hard to retrofit after you have working software. I am not going to dive into writing testable code in this post but there are a lot of articles about the topic in the wild.
Fine, which is better TDD or BDD
I am not going to dive into the best practices of TDD or BDD but I will give a bit of a high level overview.
I use BDD but I still do TDD; let me explain…
TDD – Test Driven Development
The important part here is the “Driven”. TDD is really a form of "Test First Design”. Developers write the tests first and those tests drive the development of the software. For me this is very hard. I want to build it, see it work, tweak it so it works better. But to do TDD I have to write my tests first; otherwise, it is not TDD. Sorry, writing the tests after the code is “code driven testing”…and I am not sure that is a thing.
TDD forces you to write testable code where that quick POC might not be so testable when it is finished. I do not like the way most unit tests read. Have you ever run a batch of 800 unit tests and have one fail? When you went to look at the code, even if it follows AAA ( arrange, act, assert) how long did it take you to understand what it was doing?
BDD – Behavior driven development
Make no mistake BDD is not a replacement of TDD, it is an extension. So even when I do BDD I am still doing TDD. Remember how I said I use MsTest because I do not have to install anything? Well BDD is a bit more than just a methodology. It uses a language called cucumber. That means there are extensions to be installed and configurations to be adjusted… but I see a value here so I use it. Depending on the work environment I either can use it or not. For my personal projects I use cucumber to write my tests and 80% of the time I do BDD. Yeah yeah. TDD is hard for me so sometimes I just use cucumber. But you get human readable test instead of developer readable tests. When I have not looked at my code in 9 months I can go to my tests and see my agile user story acceptance criteria. When I write my agile stories I write my acceptance criteria in the “Given, when, then” format; which happens to be the same format as cucumber. If the requirements change the expected behavior it is clear and easy to update the tests accordingly. The downside is this takes a bit more work to configure and until you get used to it –and the BDD tool you are using- it takes a little extra time. The big advantage is the test themselves become a verifiable record of the requirements and acceptance criteria, one that you can show to the business who can review it and say “yes that is right” or “No, that is not how that should work”
Blog