After witnessing a string of situations where people proposed what were, to me at least, wildly improbable theories of a problem, I began to question why it was that people kept throwing out these elaborate explanations of seemingly straightforward problems. Upon reflection, I believe it has a lot to do with people's egos, and their desire to be part of something historical. These three factors drive the generation of complex, unlikely theories in favor of simple, probable ones:

  1. The War Story Factor - You can't constantly retell a story about the time that you had a frustrating off-by-one error, even if the debugging process was arduous and the stakes were high. People only want to hear "fix-it" stories where the solution to the problem either required some seemingly mystical leap of logic, or where the actual underlying cause turned out to be incredibly bizarre and improbable. Thus, in proposing an unlikely hypothesis, we hope that it's true so that we get a new war story to add to our arsenal.
  2. The Genius Factor - We all want to look smart in front of our peers, and what better way to do so than by proposing a chimerical theory of the problem, only to be proven right in the end! If your theory is wrong, then you shrug your shoulders and move on; it's likely that no one will hold you to account for the hours or days you spent pursuing this theory. In this way, improbable theories have really only an upside for the proposers: you'll look like a genius if you are right, and just another wrong guesser if you're not.
  3. The Hero Factor - Related to the genius factor is a desire by many software people (and I assume trade and service people of all stripes) to be seen as the savior or hero. You want people to say, "We were really stuck. Thank goodness that you came up with that crazy theory that we hadn't considered. We would still be struggling with this problem!" You get to swoop in and save the day with your fantastic theory.

So we want our problems to be solved, and solved quickly, but we secretly hope they have complex, mind-bending solutions so that we can boost our egos when we solve them. We are rewarded for doing the opposite of what is generally effective.

How can we change this behavior from a team standpoint? For a start, keep track of who proposes which theories, and who tends to be right more often than not. Offer kudos to people who solve problems quickly and efficiently rather than those who solve "hard" or "unusual" problems. We tend to judge the "hardness" retroactively based on the perceived unlikeliness or difficulty of the solution, even though most debugging problems look hard when you don't know the solution, and this penalizes people who try simple solutions first and plow through lots of fixes in a shorter period of time.

The second thing to do is to make people feel bad about improbable theories. This can be as simple as encouraging mild ridicule of team members who consistently make bizarre leaps of faith in their hypotheses. At the very least, force people to make their reasoning explicit. Is their hypothesis really the one that is least inconsistent with the evidence, or is there a much more parsimonious explanation? With a carrot for rapid debugging and a stick for improbable theorizers it should be possible to eliminate the ego factor and improve your team's hypothesis-making.

I've encountered a frustrating roadblock in few distance debugging situations recently, and I have looking for a term to describe the problem. Here's the issue, in a nutshell:

A sticky problem on a remote system has been narrowed down to a likely cause, such as the interaction of two different applications. A test has been proposed to verify this assessment, which is to remove one of the applications, and run the test procedure again. However, the remote resource balks, saying "But we'll need both of those applications on the real system!" The problem is that they have conflated testing and fixing. In other words, they have taken the suggestion of "remove an application to test the theory" as "remove the application to fix the problem", when one does not have to imply the other.  Unfortunately, with these issues mixed together it creates a mental roadblock where you can no longer make headway by ruling a particular cause in or out.

So what can be done? I've found that the first thing to do is make the test/fix separation explicit. Tell the remote resource, "I understand that this is a not a permanent solution, but until we've verified that the application interaction is the problem, there is no point in pursuing other solutions." Another tactic is to offer a test procedure that involves undoing and then redoing the piece that your remote resource insists is untouchable to address their fears.  In the conflicting applications example, this might be, "Remove one of the applications and check to see if the problem still occurs.  Then reinstall that application and try it again to verify that the problem is back again."  Finally, make it clear that you understand the constraints by saying things like, "I know that you will need both applications in the long run.  I'm sure that once we've narrowed it down, we can find out what the conflict is and resolve it."  Once you have shown your focus on getting to the cause rather than proposing a solution, your remote resource should be more receptive to allowing you to try something out.