Embracing Trial and Error Within Complex Systems

As a Research Fellow with Coburn Ventures I am privileged to receive their periodic missives. The group covers a lot of ground and their grasp of the "changes" in the technology world are always insightful and compelling. Between portfolio strategy and coverage of individual companies members also contribute book reviews and thought pieces. These are lucid, clearly written articles that spawn a lot of discussion.

Earlier this week Coburn Analyst Brynne Thompson penned her take-aways from a TED presentation by Tim Harford. Tim's presentation discussed the need to embrace trial and error when working in complex systems. Brynne extends the subject matter to investing (there's plenty of complexity to deal with in that arena!), highlights some of the main points of the talk and then provides a neat summary of how she plans to apply it to her work.

The first part of this is a lead-in from Pip Coburn - the main man at Coburn Ventures. His comments are always worth a read:

What follows is a note from Brynne through our friend Julien. Thank you Julien.

I am immediately reminded of a time long ago when at Lynch+Mayer the MOMENT of "taking a position" was a VERY VERY big deal that implied you had figured it out, had crossed ALL your t's and dotted ALL your I's... 15 years later I realize that I am rarely if ever in my LIFE (of which my work is a critical subset)at a point of all t's crossed and "all" my work is done AND even more so I seem to see that when I am deluded to consider that I have ALL "set" that a rude awakening is juuuuust around the corner.

Somewhere in 1996 I had a realization in doing tech - which was far more immature and far far far far far more volatile than today for others who lived it -- that when I scanned my portfolio of investments during my regular checks and thought, "No, that one is good... No need to make a follow up check," that I had somehow alerted the Gods that I didn't know my humanness and that I was due for a painful lesson... Sure enough an earnings pre-announcement would spring. (This was BEFORE the days of what I think my friend Arnie Berman dubbed as "earnings pre announcement season" - which has stuck.)

So... EVERYTHING we do today I consider work-in-process... I am today reviewing half of our short pre determined game plans to consider with Dave and Brynne especially what we might be missing...

And I love the work I get to do...

Thanks Brynne for this note... It is well-timed for me :)

...Pip

And here is Brynne's note:

Subject: **Tim Harford at TED (video from Julien): The take away for me and my investing:

Develop a stronger muscle for trail and error in problem solving. Understand there is no one right answer, *especially* in complex systems.

Brynne...

I watched the 18min. TED talk that Julien passed on. It is on the topic of using and embracing trial and error. Fantastic video (link below).

The main point for me was that trial and error itself - or rather the thought that there can be errors at all - can be incredibly difficult, even excruciating, for most of us. And if we cannot stand to admit mistakes or defeat, we may do anything possible to avoid that experience.

I don't think I have often, if ever, heard of "trial and error" applied to investing. Investors in particular aren't fond of "error", whether tracking error or analytical error. In my mind, it's tied so closely to "poor performance!" or "lost performance!" that I want to do everything in my process to avoid error.

But, I think taking a step back from the very literal considerations around "error" can be very valuable to allow some space for all the exploration and openness that trial and error can bring to an investment process. When stripping it down, trial and error is really just iteration, and learning from each iteration, and THAT is something that we are finding to be ever more powerful, whether it is in our own investment process, or with companies that can be at "home" doing work iteratively instead of cocooning themselves off from the possibility of ever making a mistake, or companies that iterate so well that it becomes a key competitive advantage and an ability to Observe, Orient, Decide and Act (OODA) much more effectively than others.

Here are a few more points from the piece, and the video link below:

• The "God Complex": An avoidance of mistakes drives us to become "Gods" of our own little worlds - our own mental models we've created to name and explain what's our very complex world all about. When we do that, we can cocoon ourselves off, miss seeing clearly what is going on in "reality", and make poor decisions.


• Iteration/Trial and Error helps remove the "need" for a God Complex. His example is that he thinks one reason the US produces some very good businesses is that the rate of failure of US businesses is higher than the rest of the world: 10% of US businesses fail every year. There is iteration inherent in a system that allows such "failure". The turnover starts to chip away at a God Complex. There's an acceptance that starting and growing a business is tough; not everyone "wins", but also, there is no "God" example, and often, no one person in charge to say what a "successful" business is, so there is a freer definition of what business "can" be. Other business cultures around the world rely more on a few "God" examples of how to do things.


• Education: If we want to develop iterative skills and remove the cocoon of the God Complex, start teaching children in schools that there are some problems that *do not have ONE right answer*. Instead, we give exams that have question and answer with the focus on "only one right answer", that is adequate for learning some skills and memorization but inadequate for problem-solving, creativity and learning to "sit with" the uncertainty of not having an answer but being able to stick with the iteration and exploration of the problem.


• Why is the ability to use trial and error ever more important or at least, useful? The world is complex, and increasingly so! We have very interesting problems right now, and if we are so stuck and unable to admit our own infallibility, or think there is only "one right answer", we will likely be stuck with the same problems, stuck and stagnant.

TED Talks -- Tim Harford: Trial, error and the God Complex


So what does this really mean?

I loved this video, if only as a great reminder of being patient in sitting through some of the tough questions that come up for me with regard to investment process or company analysis, giving myself some space to work through the problem from different angles like a complicated puzzle instead of a short-sighted conclusion that I'm wrong if I don't "know" all the answers right away, or more likely, if I don't always see very direct, linear patterns of change, but a number of points that form a more complicated model that takes an openness to many different points of view about what a mental model is, and some injection from different mental models, disciplines and perspectives to really generate insight.

*.*

My $0.02:

I too loved the video….

The comments about education reminded me of some of the things that were said during a panel that honored Jack Kilby and Richard Smalley at an '06 Nanotechnology conference in Dallas, Texas. After the panelists, all Nobel Laureates, described what drove them to their honored result they dove in to the topic of education. To a man they agreed that "problem solving" skills were not being developed in our educational system. This is not an exact quote but the message went something like this: "We aren't teaching our students to solve problems. We are teaching them to memorize and regurgitate." One panelist, the late Alan MacDiarmid, suggested that students should question everything - going as far to say that everything a student read from today's science text book would be wrong in 20 years.

Reader comments are more than welcome!