Shahzad Bhatti

June 13, 2006

Agile Purists

Filed under: Computing — admin @ 9:26 am

Agile Purists
I find a number of software concepts have religious following where a passionate
leader can find a lot of followers even when the ideas are not very sound or
have limited application. There are number of examples like OO, agile
methodologies, dynamic languages, microsoft, etc. In a number of cases
purists ignore real world or refuse to have a dialog with opposite side.

I read post of Cedric Beust, in which he raised interesting points about :

  • Tests are (executable) specs.
  • If it’s not testable, it’s useless.

I have been thinking about these as well and I agree that these purists have
not seen most of the code in real world. Working for a large eCommerce site
with millions of lines of code, I understand that you cannot expect 100%
coverage. Though, personally I like writing tests and have even wrote a
framework TestPlayer that sniffs
legacy applications and automatically generate:

  • Unit Tests with Mocks using jMock, JUnit 3.8.1, JUnit 4.0 and TestNG 4.0
  • Integration Tests using JUnit 3.8.1, JUnit 4.0 and TestNG 4.0
  • Acceptance Tests using FitNesse
  • Stress/Load Tests

The very fact that leader of TDD, Kent Beck works for a company that writes
tools for exploratory testing (though I am skeptical about it) shows that
having 100% coverage is unrealistic. In my experience I have found that 100%
coverage takes about 2-3 times the time that it takes to write the software
and in some cases up to 4 times. This is where Cedric recommends risk based
approach. I also read reply of Uncle Bob where he suggests
“Either make sure it works, or don’t ship it. Period.” Again such hard nose
attitude shows little experience with real life code. This tells me that
unless you don’t have 100% coverage, dont’ ship your software. Yet, with
over fifteen years of industry experience I have never seen this actually
worked. I see buggy code released all the time, it’s just matter of risk
factor, i.e., you ship the software as long as there aren’t any serious bugs.
Also, when agilists talk about TDD they mean unit tests with mocks, but
in real world you also need integrations tests to test against real
services and load tests to make sure that you don’t have any unexpected
performance problems when the software is shipped. Also, for any online
software, it is also important to do security testing or penetration testing.
However, later three get
very little attention. In fact, in order to cover all facets, you need to start
with acceptance test (using Fit/FitNesse), then unit tests and followed by
integration/performance/security tests. Clearly, you need to balance all
this and focus on high risk components of your software or your software
may never ship if you stick to 100% coverage.
Finally, despite what you think 100% coverage does not mean elimination of
your QA department, it’s just that they will be doing a lot more exploratory

The second point about using tests as specs is equally unrealistic. For any
one who has worked with over 500 developers and millions of lines of code, you
can’t expect anyone to understand software by reading tests. In fact, I had to
learn a number of commercial and open source software where developers had
same attitude that just read the code or tests (if you are lucky) and
though with enough time you can understand
the software by reading code and tests, but with good brief documentation it
can be done much faster. Also, in many cases code or tests don’t tell show
why that code is there. In many cases, there could be more than one way to
do things, without documentation it is hard to see which way is preferred under
what conditions.

Though, I like TDD and Agile methodologies specially their emphasis on
people and collaboration over processes and technology, but each team
and each project is different. It is time that agilists stop proposing
prescriptive methodologies. In the end, agility means how fast you respond
to change and reduce time to market.

No Comments

No comments yet.

RSS feed for comments on this post. TrackBack URL

Sorry, the comment form is closed at this time.

Powered by WordPress