Little Leaks Sink Your Tests*
"The tests pass on my machine." "Wait, it was working a minute ago." "Oh, that test is flaky sometimes." Unpredictable tests are toxic for our productivity. They undermine confidence in our code. They encourage us to wallpaper over the immediate problem, rather than fixing the underlying cause.
In this presentation, we'll talk about a chief cause of flaky tests: leaky global state.
When global state leaks between test cases, one test can affect subsequent ones. The result is a test failing mysteriously, or passing when it shouldn’t, or alternating between passing and failing (based on timing or random ordering).
Some sources of global state—such as files and databases—are obvious. Others—mock objects, misconfigured test helpers, and custom patches to system libraries—are more subtle. During our time together, we’ll see several types of global state, how to detect them, how to solve them, and how to keep them from cropping up in the future.
Examples will be in Ruby, but the concepts are universal.
testing, maintenance, Ruby
I've spoken several times at Open Source bridge, OSCON, PNSQC, JRubyConf, and ESC. Here's last year's Open Source Bridge talk: http://opensourcebridge.org/sessions/1612
Ian Dees was first bitten by the programming bug in 1986 on a Timex Sinclair 1000, and has been having a blast in his software apprenticeship ever since. By day, Ian slings code, tests, and puns at New Relic. By night, he dons a cape and keeps watch over the city as Sidekick Man. In his (heh) “spare time,” he converts espresso into programming books, including the team effort Seven More Languages in Seven Weeks.
- Title: Little Leaks Sink Your Tests
- Track: Practice
- Room: B201
- Time: 1:30 – 2:15pm
“The tests pass on my machine.” “Wait, it was working a minute ago.” “Oh, that test is flaky sometimes.” Unpredictable tests are toxic for our productivity. They undermine confidence in our code. They encourage us to wallpaper over the immediate problem, rather than fixing the underlying cause.
In this presentation, we’ll talk about a chief cause of flaky tests: leaky global state.
- Speakers: Ian Dees