Getting Feedback from Test-Driven Development and Testing in Production

Teams are increasingly relying on strong unit and integration tests instead of end-to-end tests, using Test-Driven Development (TDD), pair programming, and good design to ship small changes often, test in production for real feedback, and use feature toggles to reduce risk, according to Ola Hast and Asgaut Mjølne Söderbom in their talk about continuous delivery with pair programming at QCon London.

Hast mentioned that they trust their unit tests and integration tests individually, and all of them together as a whole. They have no end-to-end tests: "We achieved this by using good separation of concerns, modularity, abstraction, low coupling, and high cohesion. These mechanisms go hand in hand with TDD and pair programming. The result is a better domain-driven design with high code quality."

Previously, they had more HTTP application integration tests that tested the whole app, but they have moved away from this (or just have some happy cases) to more focused tests that have shorter feedback loops, Hast mentioned.

Since test environments will always be an approximation of production, and usually struggle with long supply chains and bad test data, they have more or less stopped using them in their entirety, Mjølne Söderbom explained: "We prefer to test in production since that is where we get the feedback with the best quality."

They reduce risks by putting new features behind toggles and deploying small pieces at a time. It is something they have done for several years now, and it works really well, Mjølne Söderbom said. If something breaks in production, it's easy to find, fix, and roll back/forward, he added.

In an earlier article, Hast and Mjølne Söderbom mentioned that their team uses pair and mob programming with TDD; there are no solo tasks or separate code reviews. This approach boosts code quality, reduces waste, and enables the sharing of knowledge: "After years of practicing, we have ended up working together, doing TDD, and then deploying to production. We rarely test the application running locally or in the test environment. This was never our main intent; it is just a (happy) consequence of the way we work."

Pair programming and continuous integration can go hand-in-hand. Pushing to main multiple times a day is hard in isolation, leading to delays, large PRs, and merge issues. Pairing enables instant code review, easier refactoring, fewer bugs, and higher team resilience, Hast and Mjølne Söderbom explained.

Hast mentioned that they used to have more tests that tested the entire application running, but they have reduced this to a minimum; they usually just have a happy path test, and additional tests in case there are any special error situations: "What we cannot test with unit tests, we prefer to test in production. Tests will always be an approximation of reality, and we always struggled with long supply chains and bad test data. The best feedback we always get is from production."

It was never a goal in itself to stop using the test environment, but it was just another happy side effect. The most important thing is getting the feedback loops in place, Mjølne Söderbom mentioned. Feedback is what helps to navigate and choose a direction, and change direction when necessary: "We get the fastest feedback from our tests and the best feedback from production."

When something is painful, they do it more often, Hast explained: "The way our company got to where we are today was that we increased the cadence of our deployments to productions and we got feedback quickly where it hurt the most, then we fixed those things. This process has been ongoing for 10 years now."

We are very focused on fast feedback loops on all levels in the development process, Mjølne Söderbom said. TDD is one of our most important tools for getting early and fast feedback, he explained: "If you have code that is difficult to test, it usually means that there is a problem with the design. The code 'talks' to us, and this drives the design. Many think that TDD is a testing tool, but in reality, it is a design tool. Having proper tests that enable fast flow is more of a (really good) side-effect, Mjølne Söderbom concluded."

This approach represents a significant shift in how teams think about testing and deployment. By eliminating end-to-end tests and relying instead on unit and integration tests combined with production testing, teams can achieve faster feedback loops and higher code quality. The key is to design systems with good separation of concerns, modularity, and low coupling, which makes them easier to test and deploy.

The use of feature toggles allows teams to deploy code to production without exposing it to users, reducing risk while still getting real-world feedback. Pair programming and continuous integration work together to enable fast, frequent deployments without the overhead of traditional code reviews.

This approach isn't without risks, of course. Testing in production requires careful monitoring and the ability to quickly roll back changes if something goes wrong. But for teams that can make it work, the benefits in terms of speed, quality, and developer satisfaction can be significant.