I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). If I don’t typically make a kind of mistake (like setting the wrong variables in a constructor), I don’t test for it. I do tend to make sense of test errors, so I’m extra careful when I have logic with complicated conditionals. When coding on a team, I modify my strategy to carefully test code that we, collectively, tend to get wrong.
Different people will have different testing strategies based on this philosophy, but that seems reasonable to me given the immature state of understanding of how tests can best fit into the inner loop of coding. Ten or twenty years from now we’ll likely have a more universal theory of which tests to write, which tests not to write, and how to tell the difference. In the meantime, experimentation seems in order.
The original StackOverflow topic : http://stackoverflow.com/questions/153234/how-deep-are-your-unit-tests/153565#153565
Another blog lost in the interweb... 
Emmanuel,
Freelance opportunity in Eurocontrol, in Brussels.
Is this something that you might be interested in? If yes please let me know ASAP. Contact me at +32 2 285 40 10 or contract@eurocity.be
Looking forward to hearing from you.
Lucian
There is a fallacy in the argumentation. The argumentation uses a petitio principii, in English we would say “Begging the question”: the first proposition contains an hypothesis, not proven, which becomes the guiding principle of all the argumentation.
In this case, this is the proposition you’ve chosen as blog article “I get paid for code that works, not for tests”. This is a variant of the idea development quality should be a pragmatic trade-off between software quality and available time.
The argumentation then teaches us the Kent Beck post isn’t an objective theory of the idea test coverage (at the end, he considers we still need 5 or 15 years to get that) and that his vision of tests is an help to mainly detect his own errors: “If I don’t typically make a kind of mistake, I don’t test for it. ”.
But another idea of unit testing is tests are an helper tool for future maintainability, to ensure future corrections won’t break working stuff. And there, Kent Beck does that only reluctantly (“When coding on a team, I modify my strategy to carefully test code that we, collectively, tend to get wrong.”).
So all this post covers is ONE (and a very frequent one) scenario of development:
– a one-shot commercial development
– one or very few developers
– long-term maintainability isn’t a priority
This wouldn’t apply to long-term community open source projects like Apache, MediaWiki (but of course would totally apply to short open source projects).
I think you’ve totally missed the point of the article.
I will never say, and I don’t say, testing is useless or a waste of money. It’s a kind of project suicide to say something like this. I myself work in lot with TDD so the question is not there and I’m convinced by most kind of testing.
The idea here is to stop a trend leading to overtest everything ! Some guys are so TDD they write too much tests. If it’s a big app, ok, I have no problems with projects i’ve worked on and 25K+ unit tests you need to run before every single checkin in the source control, If they are RELEVANT !
I know a project where every simple String field (just data structure, no logic triggered, events, anything) is tested to be sure that the assignment is ok. It’s the well known point of framework testing, you’re too deep in the testing, and you’re not paid for that.
The idea here is to point out the situation where testing cost much, even in a long time point of view, then the benefits, I don’t even speak about maintaining the tests.
It’s a bit like scrum meetings in Agile. It’s a great stuff, but if you’re a team of 35 person and do a 30min scrum meet every day… Well it cost you 5 mandays per month of meets. And if the project is delayed… it’s again worst…
So we always need with great idea to find a kind of pragmatical point of view in the application. That’s the point of the article. The title is just a keeper 😉
For me, we need to develop more smoke testing and integration testing and reduce a bit the number of unit tests. And, by experience, smoke testing can usually replace dozens of unit tests with the same effect, less maintainance and a reduced cost. But thats only my point of view and some of ex employeers
This is why it’s great to have as strong of a type system as you can get – strong typing eliminates scores of boring-to-write and boring-to-run unit tests that just verify input types, output types, and basic usage.
Integrating Flow into our JS projects has reduced the amount of testing we need by a very large percentage, and catches errors All. The. Time. Embracing strong typing is probably one of the best investments you can make for your personal sanity and your code’s stability.
>>>It’s a bit like scrum meetings in Agile. It’s a great stuff, but if you’re a team of 35 person and do a 30min scrum meet every day… Well it cost you 5 mandays per month of meets. And if the project is delayed… it’s again worst…
First you should have a Scrum team more than 5 People. If it takes a 1 minute per person that meeting should not be more than 10 min a day. So if you had 35 people in a team you should break it into 7 teams. Don’t ramble about anything and everything in a Scrum standup. Try standing up in a Scrum meeting. Say what you did yesterday, say what you will be working on today, and tell the team if you have any blockers. Scrum daily standup meeting is not a overhead. If your project is delayed do not blame Scrum for it.
Its almost like saying today I got lot of work so I will skip shower, breakfast, lunch, dinner, and just sleep under the desk because all of that is an overhead. 8 hours of sleep a night and for 35 developers thats a whole lot of hours a month. Daily Standup in Scrum is as important to the team as you brushing your teeth every morning.
I’ve seen teams that allocate full sprints to improve their code coverage to reach a preset level (e.g. 80%) by writing pointless tests on their published code. This is busywork. If your success is measured by quantity, you will sacrifice the quality, and that is not going to give you any quality assurance per se.
I’ve seen code that is delivered with unit tests (enforced by team-lead through non-functional requirements), without actually solving customer’s problem (as requested in functional requirements).
We should ask WHY before writing a test case, rather than HOW (e.g. how to Mock this and that). JEST, I think, is a good rule of thumb to follow when it comes to TDD.