Agile Purists
I find a number of software concepts have religious following where a passionate leader can find a lot of followers even when the ideas are not very sound or have limited application. There are number of examples like OO, agile methodologies, dynamic languages, Microsoft, etc. In a number of cases
purists ignore real world or refuse to have a dialog with opposite side.
I read post of Cedric Beust, in which he raised interesting points about :
- Tests are (executable) specs.
- If it’s not testable, it’s useless.
I have been thinking about these as well and I agree that these purists have not seen most of the code in real world. Working for a large eCommerce site with millions of lines of code, I understand that you cannot expect 100% coverage. Though, personally I like writing tests and have even wrote a framework TestPlayer that sniffs legacy applications and automatically generate:
- Unit Tests with Mocks using jMock, JUnit 3.8.1, JUnit 4.0 and TestNG 4.0
- Integration Tests using JUnit 3.8.1, JUnit 4.0 and TestNG 4.0
- Acceptance Tests using FitNesse
- Stress/Load Tests
The fact that leader of TDD, Kent Beck works for a company that writes tools for exploratory testing (though I am skeptical about it) shows that having 100% coverage is unrealistic. In my experience I have found that 100% coverage takes about 2-3 times the time that it takes to write the software and in some cases up to 4 times. This is where Cedric recommends risk based approach. I also read reply of Uncle Bob where he suggests “Either make sure it works, or don’t ship it. Period.” Again such hard nose
attitude shows little experience with real life code. This tells me that unless you don’t have 100% coverage, dont’ ship your software. Yet, with over ten years of industry experience I have never seen this actually worked. I see buggy code released all the time, it’s just matter of risk factor, i.e., you ship the software as long as there aren’t any serious bugs. Also, when agilists talk about TDD they mean unit tests with mocks, but in real world you also need integrations tests to test against real services and load tests to make sure that you don’t have any unexpected performance problems when the software is shipped. Also, for any online software, it is also important to do security testing or penetration testing.
However, later three get very little attention. In fact, in order to cover all facets, you need to start
with acceptance test (using Fit/FitNesse), then unit tests and followed by integration/performance/security tests. Clearly, you need to balance all this and focus on high risk components of your software or your software may never ship if you stick to 100% coverage. Despite what you think 100% coverage does not mean elimination of your QA department, it’s just that they will be doing a lot more exploratory
testing.
The second point about using tests as specs is equally unrealistic. For any one who has worked with over 500 developers and millions of lines of code, you can’t expect anyone to understand software by reading tests. In fact, I had to learn a number of commercial and open source software where developers had same attitude that just read the code or tests (if you are lucky) and though with enough time you can understand the software by reading code and tests, but with good brief documentation it can be done much faster. Also, in many cases code or tests don’t tell show why that code is there. In many cases, there could be more than one way to do things, without documentation it is hard to see which way is preferred under what conditions.
Though, I like TDD and Agile methodologies specially their emphasis on people and collaboration over processes and technology, but each team and each project is different. It is time that agilists stop proposing prescriptive methodologies. In the end, agility means how fast you respond to change and reduce time to market.