Test-driven development

Test-Driven Development (TDD) is a software development technique that involves repeatedly first writing a test case and then implementing only the code necessary to pass the test. Test-driven development gives rapid feedback. The technique began to receive publicity in the early 2000s as an aspect of Extreme Programming, but more recently is creating more general interest in its own right.

Practitioners emphasize that test-driven development is a method of designing software, not merely a method of testing.

Along with other techniques, the concept can also be applied to the improvement and removal of software defects from legacy code that was not developed in this way .

Requirements

Test-driven development requires that an automated unit test, defining requirements of the code, is written before each aspect of the code itself. These tests contain assertions that are either true or false. Running the tests gives rapid confirmation of correct behaviour as the code evolves and is refactored. Testing frameworks based on the xUnit concept (see the list of unit testing frameworks for an exhaustive list) provide a mechanism for creating and running sets of automated test cases.

Test-Driven Development Cycle

The following sequence is based on the book Test-Driven Development by Example , which many consider to be the original source text on the concept in its modern form.

1. Add a test

In test-driven development each new feature begins with writing a test. This test must fail since it is written before the feature is implemented. In order to write a test, the developer must understand the specification and the requirements of the feature clearly. This may be accomplished through use cases and user stories to cover the requirements and exception conditions. This could also imply an invariant, or modification of an existing test.

2. Run all tests and see the new one fail

This validates that the test harness is working correctly and that the new test does not mistakenly pass without requiring any new code.

The new test should also fail for the expected reason. This step tests the test itself, in the negative.

3. Write some code

The next step is to write some code that will cause the test to pass. The new code written at this stage will not be perfect and may, for example, pass the test in an inelegant way. That is acceptable as later steps will improve and hone it. It is important that the code written is only designed to pass the test; no further (and therefore untested) functionality should be predicted and 'allowed for' at any stage.

4. Run the automated tests and see them succeed

If all test cases now pass, the programmer can be confident that the code meets all the tested requirements. This is a good point from which to begin the final step of the cycle.

5. Refactor code

Now the code can be cleaned up as necessary. By re-running the test cases the developer can be confident that refactoring is not damaging any existing functionality. The concept of removing duplication is an important aspect of any software design. In this case, however, it also applies to removing any duplication between the test code and the production code—for example magic numbers or strings that were repeated in both, in order to make the test pass in step 3.

The cycle is then repeated, starting with another new test to push forward the functionality. The size of the steps can be as small as the developer likes, or get larger if s/he feels more confident. If the code written to satisfy a test does not fairly quickly do so, then the step-size may have been too big, and maybe the increment should be split into smaller testable steps. When using external libraries (such as Microsoft ADO.NET) it is important not to make increments that are so small as to be effectively testing the library itself .

Development style

There are various aspects to using test-driven development, for example the principles of "Keep It Simple, Stupid" (KISS) and "You Ain't Gonna Need It" (YAGNI). By focusing on writing only the code necessary to pass tests, designs can be cleaner and clearer than is often achieved by other methods.

To achieve some advanced design concept (such as a Design Pattern), tests are written that will generate that design. The code may remain simpler than the target pattern, but still pass all required tests. This can be unsettling at first but it allows the developer to focus only on what is important.

Test-driven development requires the programmer to first fail the test cases. The idea is to ensure that the test really works and can catch an error. Once this is shown, the normal cycle will commence. This has been coined the "Test-Driven Development Mantra", known as red/green/refactor where red means fail and green is pass.

Test-driven development constantly repeats the steps of adding test cases that fail, passing them, and refactoring. Receiving the expected test results at each stage reinforces the programmer's mental model of the code, boosts confidence and increases productivity.

Benefits

Programmers using pure TDD on new ("greenfield") projects report they only rarely feel the need to invoke a debugger. Used in conjunction with a Version control system, when tests fail unexpectedly, reverting the code to the last version that passed all tests is almost always more productive than debugging.

Test-driven development can help to build software better and faster. It offers more than just simple validation of correctness, but can also drive the design of a program. By focusing on the test cases first, one must imagine how the functionality will be used by clients (in this case, the test cases). Therefore, the programmer is only concerned with the interface and not the implementation. This benefit is complementary to Design by Contract as it approaches code through test cases rather than through mathematical assertions or preconceptions.

The power test-driven development offers is the ability to take small steps when required. It allows a programmer to focus on the task at hand as the first goal is to make the test pass. Exceptional cases and error handling are not considered initially. Tests to create these extraneous circumstances are implemented separately. Another advantage is that test-driven development, when used properly, ensures that all written code is covered by a test. This can give the programmer, and subsequent users, a greater level of trust in the code.

While it is true that more code is required with TDD than without TDD because of the unit test code, total code implementation time is typically shorter. Large numbers of tests help to limit the number of defects in the code. The early and frequent nature of the tests helps to catch defects early in the development cycle, preventing them from becoming endemic and expensive problems. Eliminating defects early in the process usually avoids lengthy and tedious debugging later in the project.

TDD can lead to more modularized, flexible, and extensible code. This effect often comes about because the methodology requires that the developer think of the software in terms of small units that can be written and tested independently and integrated together later. This leads to smaller, more focused classes, looser coupling, and cleaner interfaces. The use of the Mock Object design pattern also contributes to the overall modularization of the code because this pattern requires that the code be written so that modules can be switched easily between mock versions for unit testing or "real" version for deployment.

Limitations

Test-driven development, as a design technique, can produce inappropriate and low-quality code just as rapidly as high-quality code. A team should use other techniques, such as discussion and code reviews for example, to ensure not only that the correct features are implemented, but that the resulting code uses acceptable styles. James Carr has created a catalog of TDD anti-patterns [1], as discussed on the TDD mailing list, and others have proposed that Tester Driven Development (note: TestER driven development, as distinct from Test-driven development) itself is a design anti-pattern.

Test-driven development only proves correctness of design and functionality according to the test cases written. An incorrect test that does not represent the specifications will produce incorrect code. Some of the emphasis on correctness and design shifts to designing test cases since they are the drivers. As a result, a test-driven development is only as good as its tests. Due to the rapid alternation of tasks described above, of course each developer is responsible for their own tests—they should not be written in bulk, in advance, for example.

(Contrarily, code that submits to tests through all phases of its growth will achieve many Design For Test goals implicitly. When developers only write code that can indeed be tested, verification with formal unit tests becomes easier.)

As a design technique, test-driven development is unlikely to produce new algorithms, although it will verify their behaviour. Computer science research should therefore only use it in conjunction with other techniques. Automated testing may not be feasible at the limits of computer science (cryptography, robustness, artificial intelligence, computer security etc).

TDD is difficult to use in some situations, such as graphical user interfaces or relational databases, where systems with complex input and output were not designed for isolated unit testing or refactoring.

The use of the Mock Object design pattern is necessary in order to control the scope of dependencies involved in unit tests. However, when creating mock objects to interact with the module being tested, it is necessary that the developer have a good understanding of the of the behavior of the entities that are being mocked. Failure to achieve this understanding can lead to problems when the modules are deployed into a "real life" environment and they receive input they were not expecting our give output the environment can not handle. In some instances creating mock objects that function realistically enough to allow for appropriate unit testing can be difficult and time consuming.

Code visibility

Test-suite code clearly has to be able to access the code it is testing. On the other hand normal design criteria such as information hiding, encapsulation and the separation of concerns should not be compromised. Thus, unit test code for TDD is usually written within the same project or module as the code being tested. In object oriented design this still does not provide access to private data and methods. In some languages these can be made internal to allow test-suite access, but this is still a distortion of the original intent. Where the facility is available in the language, it may be better to provide the test code with privileged access to private features via partial classes during development.

In all cases, thought must be given to the question of deployment. When the final library or executable is assembled or built, is it desirable to include in it all the test code that was used in its development? If it is written within the project or module, maybe as partial classes, then it is likely that it would be included by default. This will make the delivered run-time much larger than necessary, and may introduce security and other unexpected problems. It is unlikely that test code compiled into a delivered binary will ever be run; it is mainly useful to a developer with full access to the source code.

In many compiled languages, this can largely be solved by the use of compiler directives surrounding all test code to prevent deployment.

#if DEBUG
...//test class
#endif

In scripting languages, keeping all test code in separate folders may help remind users not to deploy it, for example to live web servers, especially if the folders have, for example '_DO_NOT_DEPLOY' as part of their name.

Fakes, mocks and integration tests

If a code project is going to be developed by the creation of several hundred test cases, and the developers are expected to rerun these tests very frequently throughout the development, then it is necessary for these tests to run extremely quickly. For this reason unit tests, as used in TDD, should never cross process boundaries, i.e. they should never lead to any calls to code running in a different process to the one under test. Much more so, they should never cause and wait for any network access, across host boundaries.

This appears to lead to a problem when the code under development relies on a database or a web service or any other external process or service. This is not a problem, however, but an opportunity and a driving force to design more modular, more testable and more re-usable code. Two steps are necessary:

1. Whenever external access is going to be needed in the final design, an interface should be defined that prescribes the access that will be available.
2. The interface should be implemented in two ways, one of which really accesses the external process, and the other is a fake or mock object. Fake objects need do little more than log a message such as "Person object saved" to a log file or to the console, if that. Mock objects differ in that they themselves may contain test assertions that will make the test fail, for example, if the person's name and other data are not valid. Fake and mock object methods that return data, ostensibly from a data store or user, can help the test process by always returning the same, realistic data that the tests can rely upon.

The disadvantage of this approach is that the actual database or other external-access code is never tested by the TDD process. This must be avoided, and other tests are needed that instantiate the test-driven code with its 'real' implementations of the interfaces discussed above. Many find it useful if these tests are kept quite separate from the TDD unit tests, and are referred to as integration tests. There will be fewer of them, and they need be run less often than the unit tests. They can nonetheless be implemented using the same, for example xUnit, testing framework.

Integration tests that test writing to a database should be careful always to clean their test data out after the test, even if the test fails for some other reason. This can be achieved using some combination of the following techniques where relevant and available to the developer:

• the TearDown method integrated into many test frameworks
• try...catch...finally exception handling structures where these are available
• database transactions where a transaction atomically includes perhaps a write, a read and a matching delete operation..

See also

• Aegis a software change management system which supports a test-driven development workflow
• Software testing
• Test case
• Unit test
• Mock object
• List of software development philosophies
• List of unit testing frameworks

External links

• Three Rules of TDD
• Extreme perl - Unit testing
• TDD is worthless if you can't write good unit tests - Unit Testing guidelines
• Write Maintainable Unit Tests That Will Save You Time And Tears
• Improving Application Quality Using Test-Driven Development by Craig Murphy. This article provides an introduction to Test-Driven Development with concrete examples using NUnit
• testdriven.com on-line test-driven development community
• Test-driven Development using NUnit tutorial (also Java version available)
• c2.com Test-driven development from WikiWikiWeb
• Brief explanation of the Qualities of an Ideal Test
• Failing Tests Meaningfully : TDD Process and The Karate Kid by Scott Bellware. This article explains TDD and how the red-green-refactor cycle in TDD acts along with unit tests as a method of calibration to ensure the tests and the code both function correctly.
• team test development A non-programmer driven approach towards test driven development.
• Test Driven Development with Visual Studio 2005 Team System - Doug Seven, 3/10/2006
• TDD Anti-Patterns Common mistakes and mishaps when starting out with TDD... a catalog of how to NOT do TDD.
• Introduction to Test-Driven Design (TDD)
• XP Episode By Bob Martin and Bob Koss