Is CrowdStrike’s Mishap A Blow for Test Driven Development Enthusiasts?
TDD, or Test Driven Development, is a coding practice where tests are written before the actual code. It’s designed to validate functionality and improve code quality. However, a deep dive into the recent CrowdStrike incident reveals potential weaknesses in this approach.
A critical error was outlined in their analysis:
“The new IPC Template Type defined 21 input parameter fields, but the integration code that invoked the Content Interpreter with Channel File 291’s Template Instances supplied only 20 input values to match against.”
Also, on July 19, 2024:
“Two additional IPC Template Instances were deployed, introducing a non-wildcard matching criterion for the 21st input parameter. These new Template Instances resulted in a version of Channel File 291 that required the sensor to inspect the 21st input parameter—a condition not met by previous versions.”
Essentially, the test expected 21 parameters, and always filled in the 21st parameter, even if one was not provided. This allowed the test to pass, but when only 20 were delivered, the app failed leading to a null reference for the missing 21st parameter. This highlights a oversight in the testing process where the test did not accommodate real-world application changes.
While TDD and unit testing are valuable, they are not foolproof. I find development teams often write poorly designed tests sometimes with the goal of meeting a quote on code coverage, or with the idea that they can update the test later. This incident serves as a reminder that quality in testing should never be sacrificed for coverage metrics.
Do you employ TDD in your projects? What has been your experience with balancing test quality and coverage?’