Arrakis teaches the attitude of the knife—chopping off what’s incomplete and saying: “Now it’s complete because it’s ended here.”
—from “Collected Sayings of Maud’Dib” by the Princess Irulan, in Frank Herbert’s Dune
Testing isn’t explicitly mentioned this time, but most testers who have been around the block a few times will probably recognize why this caught my eye. Often it feels like there are an infinite number of things to test, and the only way testing can be “done” is by putting down the knife.
Is there any testing that the “attitude of the knife” doesn’t apply to?
Of course, we like to attribute more agency to ourselves than that. Maybe we say our testing is complete when we’ve tested all the aspects described in our well-reasoned test plan. You know, that perfect test plan that takes into consideration all the stakeholders’ needs, the risks involved, the impact to the users, and timeliness required to arrive at just the right amount of testing for this particular context. At the end you can push back from your terminal and say, “Yes, I tested everything I set out to test. Everything about it has been validated and verified to the extent reasonable, and I’m not just saying that because we’ve run out of time.”
But then again, did you not come up with your test plan or strategy knowing how much time would be reasonable to spend on this? Did you know that the knife was going to come down on a certain day? Did you have a sense of the tolerance of your stakeholders, and balance that against the risks?
We all know that exhaustive testing is impossible. Most of us probably realize that exhaustive testing isn’t the goal anyway, and not just because we don’t have infinite time. The knife has to come down sometime, and our testing will be complete because it ended.*
Though… does anybody ever finish testing anything? Short of moving on to another product or company entirely, I don’t think I do. Maybe we don’t live on Arrakis.
I recently read Frank Herbert’s Dune and was surprised to find a couple passages on testing. I thought it would be fun to take them completely out of context (that’s a good habit for testers, right?) and try to apply them to software instead of spice. Here’s the first:
Any road followed precisely to its end leads precisely nowhere. Climb the mountain just a little bit to test that it’s a mountain. From the top of the mountain, you cannot see the mountain.
—Bene Gesserit proverb, from Frank Herbert’s Dune
This proverb appeared in one of the excerpts from in-universe books that open each chapter, all meant to have been written after the main events of the story and usually highlighting some important aspects of the plot or characters in the chapter to come.
To be honest, I’m not really sure what the relevance of this passage to the story was. It comes immediately before the chapter where Lady Jessica finds the greenhouse in their new home on Arrakis. In it, she finds a warning that there are double agents among them advancing a plot to overthrow the Duke. Maybe these initial clues at treachery are like the first steps on the road, or the first bit of the mountain. I don’t know. It doesn’t matter.
Is there any way we can say this proverb applies to software testing?
One interesting bit for us is the advice to “climb the mountain just a little bit to test that it’s a mountain.” My first thought was that it could be about how you can’t practically test everything completely, but it equally implies that you could climb the mountain or follow the road to its end (whereas you can’t test everything). This isn’t saying anything about what’s possible or not. I’d be inclined to say that it means we don’t need to climb the mountain—that we can tell it’s a mountain without going to the summit—except for the other half of the proverb:
From the top of the mountain, you cannot see the mountain.
This isn’t as easy to recklessly apply to software. With the road, once you’ve gone to the end of it, it goes nowhere else. For the mountain, it sounds like the essence of the mountain is that it is this large thing towering above the landscape. It’s something that goes up. So you can’t appreciate its mountain-ness from the top. Does this apply to software? Do we lose something about the software by testing it to its end? It might work on something like scientific study—once you understand everything there’s no more science to do—but a piece of software doesn’t stop being useful just because we’ve already exercised everything it can do.
Alternatively, perhaps it means that you lose sight of what the mountain or the road are once you’re that far along them. When you’re fully immersed in something, when you’ve studied that software every way you can think of and think you understand it inside and out, you lose the ability to see it as a beginner or an outsider. As a tester, when you become an expert on a particular system, do you risk losing the ability to see the software for what it is? I don’t feel like that has ever happened to me, but it’s also the sort of thing that you wouldn’t notice happening. It is certainly true that the things I think to test on a product after two years are different from what I would test if I were seeing it for the first time. Hopefully it’s because I know more and have a better idea of where the risks are, but maybe not.
This would be where it would be helpful to understand what the passage actually meant in the context of the novel. Maybe there’s nothing here about testing at all even though it mentions how to test something. I’d love to hear other interpretations!
In a future post, when I don’t have a real testing topic for the week, I’ll post the second passage from the book that is quite the opposite: there’s no specific mention of testing but it will be quite familiar to how we test. (Update: part two is here.)
I want to start by introducing why I decided to start blogging about an area of work that seems to have no shortage of blogs and communities and podcasts and companies all clamoring for attention. As a relative newbie on the scene, how much new is there that I can add to an already very active conversation?
I came into testing as a profession in 2014, just by being in the right place at the right time. It wasn’t something I planned on doing, and not something I had any training in. I’ve taken precisely one computer science course, 10 years ago. In the meantime, I had been pursuing an academic career in physics. I’ve had a lot of catching up to do.
In science, you can go to any one of a hundred introductory textbooks and start learning the same fundamentals. There are books on electromagnetism that all have Maxwell’s equations and there’s classes on quantum mechanics that all talk about Hamiltonians. There’s really only one “physics” until you get the bleeding edge of it, and even then there’s an underlying assumption that even where there are different competing ideas, they’ll eventually converge on the same truth. We all agree that we’re studying the same universe.
Software testing is nothing like that.
We all do software testing, but none of us are testing the same software. Even though we use a lot of the same terms, there are as many ideas about what they mean as testers using them. There’s a vast array of different ideas about just about every aspect of what we do. That’s part of what makes it exciting! But it also makes it difficult to feel like I know what I’m doing. How do I actually learn about a discipline that has so much information in so many different places with so many different perspectives without just completely overwhelming myself?
That’s where curling comes in.
In case you didn’t already know that I’m Canadian, I’m also a curler. In a lot of ways, curling is physics-as-sport. And what does it have to do with blogging or testing?
Curling is all about sliding rocks down over 100 feet (30 meters) of ice and having them land in the right place. The two biggest variables are simply the direction and how fast you throw it. Once it’s out of the thrower’s hands, it’s the job of the sweepers to tell if it’s going the right speed to stop in the right place or not, and the job of the skip at the far end of the ice to watch if it’s going in the right direction. They need to communicate, since if either one of those variables is off, the sweepers can brush the ice to affect where the rock goes.
As the guy who’s walking down the ice trying guess where this thing is going to land 100 feet from now, I can tell you it’s damn hard to get that right. When I first started playing, it was very easy to escort 47 rocks down the ice and still not have any idea where the 48th was going to land until I got a very simple, but oddly frightening, piece of advice.
Just commit to something.
Experienced curlers have a system for communicating how fast a rock is moving by shouting a number from 1 to 10. A “four” means it’s going to stop at the top of the outermost ring. A “seven” means it’ll be right on the button (a bulls-eye, so to speak). I knew this system and I would think about the numbers in my head as I walked beside those rocks, but it wasn’t until I started committing to specific numbers by calling them out to my team that I started to actually get it.
What made it frightening was that those first few times I called out a number I was way off. And I knew that I was going to be way off. I knew I stood a good chance of being wrong, loudly, in front of everybody else on the ice. But by doing it, I actually started to see how the end result compared to what I committed to. Not in the wishy-washy way I did when I would run through those numbers in my head (“that’s about what I would have guessed), but in a concrete way. It’s similar to how you think you know all the answers when watching Jeopardy, but it’s a lot harder when you have to say the answers out loud. I started to think through the numbers more, pay more attention to how the rocks were moving, committed out loud to something, and took in the feedback to learn something.
Can you see where I’m going with this now?
Even though software testing blogs are a dime a dozen, if I want to actually become an expert in this field I think it’s time to start forcing myself to get my thoughts together and commit to something.
My goal with this blog, then, is to think through testing concepts and my experiences and commit those thoughts to paper. I’m not going to try to explain basic terms as if I’m an authority, but I might try to talk through whether some of those concepts are useful to me or not and how I see them actually being used. I plan to talk about my experiences and views as a tester, as “a QA”, as a developer, and and as a member of this community, so that I can commit to growing as a professional.
If nobody else reads this it’ll still be a useful exercise for myself, but I do hope that there’s occasionally a skip on the other end of the ice who’ll hear my “IT’S A TWO!” and shout back “OBVIOUSLY TEN!”