Efficient unit-testing with a containerised database

In my previous post I explained why we should write more unit tests for our database code. Databases contain crucial business logic that is often developed and maintained in tandem with the code that depends on it. If we want to validate complex SQL statements in isolation and in detail, that means writing much more automatic tests than we normally do. Testing against a real database server (not an in-memory emulation) is by nature slower and more cumbersome to set up and run than basic unit tests. Here I will explain some practical strategies to make that process faster and more manageable.

The tips and tricks fall into two categories. First, I’ll look at ways to arrange your code under test so that the database is only spun up and accessed when it is actually needed. Secondly there are tips on how to prepare your database container with schemas and data, so setup time is kept to a minimum.

Container terminal, Port of Rotterdam
Continue reading “Efficient unit-testing with a containerised database”

Your database needs unit tests: no excuses

The test pyramid is a well-known visualization aid to classify software tests. As you climb the steps towards greater integration of components, you proceed from many, detailed, fast and isolated tests towards fewer, slower and more global tests that validate the system as a whole. It makes good sense in principle but it’s harder to explain how the stratification between unit-, integration and end-to-end tests is supposed to work. Opinions differ on what parts to integrate at which layer in the pyramid. You would think the database belongs in the upper strata, because it is expensive to set up and run. Yet it also makes sense to integrate it at the lower, detailed stage in the pyramid when it contains business-critical logic that requires detailed validation. It often needs the kind of rigorous validation that you cannot leave to a few slow, global integration tests. But let’s start with a recap of the definition of a unit test.

Continue reading “Your database needs unit tests: no excuses”

User-friendly API publishing and testing with Retrofit

SUMMARY: Any web service needs to export their public API if consumers want to make the best use of that service. A developer-friendly approach to do so if you work in the Java ecosystem is to package DTOs and endpoint interfaces in an API jar file and use the Retrofit framework to create type-safe clients for integration testing. This article discusses a complete example.

If you’ve worked in enterprise Java projects you will remember good old Web Services Description Language, the XML based format for describing network services by IBM and Microsoft. Maybe you still work with it? WSDL and its twin XML Schema are among those W3C standards that seasoned developers love to hate. Its specification files are not very human readable, let alone human writable. Fortunately you don’t have to. They can be generated by your server endpoint and fed straight into a code generator to create transfer objects (dtos) and service stubs.

Retrofest in Tenterden, Kent. World War 2 Theme Day.
Continue reading “User-friendly API publishing and testing with Retrofit”

Test metrics give false security. Focus on your dangerous code first.

Summary
Pushing for test coverage is a bad incentive. It favors quantity over quality. You should prioritize your code with regard to test-worthiness. I discuss three such categories. First there is code that processes unmanaged and possibly malicious (user)input. You should test this for robustness. Secondly there is complex business logic with possible implementation errors. Test this for correctness. Thirdly there are quirks with trusted frameworks running code that doesn’t perform at scale. Run performance tests under realistic production loads to capture these.

Continue reading “Test metrics give false security. Focus on your dangerous code first.”

We’re doing code reviews all wrong. It’s about sharing, not finding bugs.

I love reviews. When it comes to consumer equipment and whether it’s worth your money, it’s a blessing to have so much free advice to choose from. Last year I bought a new 3000-euro Casio Celviano digital piano online during the pandemic, purely based on internet reviews. Not my choice, but retail was locked down and I couldn’t try it out anywhere. Most reviews were favourable to raving, so I found safety in numbers there. Now you have to accept that reviews by retailers of their own wares are biased, but then again nobody would go to the trouble of a scathing review simply because they don’t like digital pianos, musical instruments in general or have it in for Casio as a company. Pianos are not controversial, in other words. What on earth does this have to do with code reviews? Bear with me.

Heijplaat, Port of Rotterdam
Continue reading “We’re doing code reviews all wrong. It’s about sharing, not finding bugs.”