For more than a quarter century, Michael Mauboussin has been producing research on decision making and investment process. Dan Callahan has been his co-author on articles for much of the last decade. Their latest essay is “Feedback: Information as a Basis for Improvement,” which fits neatly with The Investment Ecosystem’s goal of “continuous improvement for investment professionals and organizations.”
That piece will be used to look at investment practice from two points of view. First, here, in regard to the world of asset management — and subsequently in a Due Diligence category posting that looks at the issues from the other side of the table.
Disconnects
Despite the widespread popularity of Mauboussin’s work among investment professionals, many of the ideas and principles presented therein don’t result in actual implementation within organizations.
Furthermore, it’s perplexing that investment professionals — who are so open to new companies, strategies, and vehicles from an investing standpoint — can be anything but innovative when it comes to organizational design, investment process, and the like. That sort of bounded creativity stands in stark contrast to Mauboussin’s approach, which seeks to apply insights from other disciplines in order to improve investment decision making.
Those disconnects are on full display in “Feedback.” The authors aggregate a dozen “facets of process improvement” in three categories: people, organizations, and feedback during process execution.
People
This section opens with a fairly innocuous summary:
If you want to build an excellent, repeatable process you need to start with having the right people. You then need to train them to be effective, figure out how you should deploy their skills, and institute a policy of practice.
If, as the authors then suggest, the “first step is to identify the skills that can lead to success,” do asset management firms routinely examine them in detail, and do they analyze potential hires on that basis? In large part, no. Specifically regarding those with some years of experience, previous performance is more heavily weighted than it should be (given the inherent imprecision in performance as a measure of skill, as well as the often overlooked importance of the candidate’s previous environment in producing it).
The authors use the characteristics of good forecasters found in Superforecasting, the book by Philip Tetlock and Dan Gardner, to illustrate the kind of skills inventory that firms should be using — a list that is particularly appropriate given the importance of forecasting in so many investment roles and the lack of effort put forth to try to improve the results from that activity.
They also delve into the differences between intelligence quotient (IQ) and rationality quotient (RQ), noting that “how people think is more important than raw measures of intelligence.” That leads to an examination of the differences between “fluid rationality” and “crystalized rationality” — including what can be tested for (and what can’t), as well as in what categories improvement is most likely (and where it isn’t).
Can training provide that improvement? If so, how? The report addresses ways to improve forecasting skills; to identify and institutionalize standards for analytical concepts that are prone to sloppy definition and application; and to assess “which aspects of investment process can be addressed systematically and which must be in the realm of judgment.” Also worth noting are the leverage effects that come from training on effective communication skills and data visualization techniques.
Given the nature of the investment realm, “it is easy to get drawn into a pattern of all playing and no practicing.” But how can one “practice” this craft? With the computing power and gamification techniques of the day, you’d think that advanced simulations could be developed to both hone desired skills and provide exposure to a range of possible scenarios to experience. In the meantime, the authors offer some more prosaic options.
The section closes with this (an apt segue into the next topic):
There is a lot of opportunity to improve coaching in the investment business . . . it starts by professionals acknowledging that there is room to improve.
Organizations
Many of the shortcomings in the areas noted above can be traced to the culture of the industry as a whole and asset management in particular. “Values and norms are influenced by governing objectives, incentives, teamwork, and the overall organizational environment,” so examinations of those come next.
Asset management firms “have tension between two potential objectives,” one related to the goal of producing long-term returns for their clients and the other focused on the financial success of the firm and its employees. Most often, that tug of war leads to playing the game as it is currently being played; the industry is marked by similarity rather than differentiation and inertia instead of innovation.
Incentives can play an important role, but “effective incentives are very difficult to design, and incentives by themselves rarely promote the proper behaviors.” In fact, many firms cling to incentive structures that reflect the noise of the market — and most equate the performance of an individual to the “numbers” that appear to represent their worth. That leads to a not-so-merry-go-round of hopes that the results will be better next time, instead of an orientation for improvement grounded in the need for feedback on the fundamental drivers of long-term value-added.
Over time, there has been a massive increase in the proportion of assets that are managed by teams versus individuals — at least superficially, since it’s unclear how many are in fact what could be called real teams. Usually lacking in their construction are careful consideration of the importance of collective (and multifaceted) intelligence on a team, the elements that lead to team success, and the kinds of training that are helpful. Yet those factors likely are more important than traditional drivers of team composition (previous areas of experience and track records).
The “environment,” or culture if you will, of an organization is the soil in which everything grows. It should be “committed to ongoing learning,” should “welcome different points of view,” should audit the decision-making process in ways that lead to specific and ongoing improvement, and should engender an atmosphere of equanimity:
All organizations go through ups and downs, so removing peaks and valleys is essential. A focus on proper process with a long-term view contributes to this attitude.
Feedback during process execution
The final section includes some specific feedback techniques to improve investment processes:
Providing feedback to fundamental investors is inherently difficult because the process to make decisions is often poorly defined and the outcome, the rise and fall of asset prices, is noisy in the short run. One way to solve this problem is to break down decisions into measurable components. But it all starts with documenting decisions and how they are made. Without a record of what you expect to happen, it is difficult to measure the accuracy of your predictions.
That means identifying in advance the “variant perception, a well-founded view that the market has not priced properly” which is motivating a particular decision. Using a “linchpin analysis” to flesh out that variant perception can help to reveal the weaknesses of the proposition, and it also serves as a roadmap for the decision process that can be evaluated after the fact.
One specific way to track the accuracy of predictions about probabilistic events is called the Brier score. Is it widely used? Not at all:
Most investment organizations have the raw material to keep Brier scores but fail to do so. Part of it is that keeping track of decisions requires some discipline. But the bigger issue is that when we are wrong, we generally cope by crafting a story to explain what happened in a way that avoids making us look bad. We are poor at predicting but great at explaining the past.
The same can be said for other pieces of what could be called a process attribution. The information is there to analyze whether the individual components of that process diagram in the pitch book actually add value, but very few managers perform such analyses and fewer still reveal what they have found.
Many aspects of process feedback have gotten easier; there are more opportunities for “a machine giving feedback in order to improve a human’s game.” What’s lacking, though, is the commitment to identify the feedback loops that are important, to communicate how they can help to move the organization forward over time, and to clear away the cultural impediments that inhibit progress.
Your scorecard
If you work at an asset management firm — and especially if you are in a leadership role — take the time to walk through the items mentioned above in some detail and grade your organization. What are you doing and what are you not doing to build the kind of feedback culture that yields the ongoing improvements that you will need to compete in the future?
For those of you that evaluate asset managers, stay tuned for the next posting, which deals with the implications of all of this for your efforts to separate the wheat from the chaff.
Published: April 22, 2022
To comment, please send an email to editor@investmentecosystem.com. Comments are for the editor and are not viewable by readers.