Four for Friday ~ Benchmarking

At each step along the investment chain, providers are judged by performance.  Absolute measures of performance can come into play (since very strong or very weak returns are emotional triggers), but, for the most part, relative comparisons dictate who gets hired and who gets fired.

Benchmarks serve as the measuring sticks by which those judgments are made.  All too often, they are used haphazardly, with comparisons being made for periods that are far too short.  The infrastructure of the industry is built to spit out numbers and whether they arrive daily or monthly or quarterly or annually, performance against a benchmark captures a larger mindshare than any other consideration.

Principles

Almost twenty years ago, Laurence Siegel authored a monograph for (what is now called) the CFA Institute Research Foundation, Benchmarks and Investment Management.  While some references in it may be out of date, the principles aren’t.

The section on “Critiques of Benchmarking and a Way Forward” includes some thoughtful commentary and a framework for performance measurement from Peter Bernstein, providing an interesting comparison to Siegel’s own views.  One of Siegel’s conclusions is a good point from which to start a benchmark debate:

You can’t design a simple, rule-based, judgment-free portfolio that is demonstrably more efficient than the cap-weighted benchmark.

The most commonly-cited criteria for benchmarks comes from another Research Foundation book, Controlling Misfit Risk in Multiple-Manager Investment Programs, by Jeffery Bailey and David Tierney.  Along the way, their “properties of a valid benchmark” were reordered for use in the CFA curriculum via the acronym SAMURAI.  Here’s Ben Carlson’s summary of them:

Specified in advance (preferably at the start of the investment period)
Appropriate (for the asset class or style of investing)
Measurable (easy to calculate on an ongoing basis)
Unambiguous (clearly defined)
Reflective of the current investment opinions (investor knows what’s in the index)
Accountable (investor accepts the benchmark framework)
Investable (possible to invest in it directly)

That serves as a good list to go back to when considering what benchmark is used in a particular situation, whether it meets those criteria, and the implications of choices that deviate from them.

Customized benchmarks

The results of large institutional asset owners are covered in the financial press, usually by comparison to customized portfolio and asset class benchmarks (see, for example, “CalSTRS outperforms in every asset class”).  In his essay, “Lies, Damn Lies and Performance Benchmarks: An Injunction for Trustees,” Richard Ennis questions that practice:

As a result of benchmark bias, the majority of funds give the impression they are performing favorably compared to passive management when, in fact, they are underperforming by a wide margin.

Ennis bemoans the move away from clear and simple benchmarks to ones that are complex, potentially misleading, and full of questionable methods.  The bottom line for him:

[Trustees] must take control of performance reporting and see that it is done right.  This means adopting a passive benchmark of the type described here and living with it — no tampering or tweaking!

Moving the goalposts

Surveying another part of the ecosystem, Kevin Mullally and Andrea Rossi published a paper, “Moving the Goalposts? Mutual Fund Benchmark Changes and Performance Manipulation.”  From the conclusion:

In this paper, we document that mutual funds take advantage of a loophole in the SEC’s disclosure requirements to provide misleading information about their past performance.  Specifically, we find that mutual funds systematically and strategically change their self-designated benchmark indexes to embellish their benchmark-adjusted performance.  Simply put, funds add indexes with low past returns and drop indexes with high past returns.  Investors respond to these changes by allocating more capital to these funds and subsequently experience persistently low returns.

Furthermore, “High-fee funds, broker-sold funds, and funds experiencing poor performance and outflows are more likely to engage in this behavior.”

Provider examples

Consultants and asset managers sometimes produce pieces where they lay out how they think certain strategies should be benchmarked.  Here are some examples:

“Benchmarking Alternative Risk Premia,” bfinance.  This covers quite a bit of ground, and includes an exhibit showing the pros and cons of the different benchmarking approaches.  It was written in September 2020 and the first of its “key takeaways” demonstrates how market events can prompt a reassessment:

As is the case with all absolute return strategies, Alternative Risk Premia strategies present a benchmarking challenge.  There is now a pressing need for investors to understand recent performance and either re-underwrite or rethink existing allocations to the space.

“Determining an appropriate benchmark for low volatility equity strategies,” RBC.  The SAMURAI elements are used to compare four ways of benchmarking the strategies:  versus a minimum volatility index; a blended cash and broad cap-weighted index; or a broad cap-weighted index, using a) returns or b) risk-adjusted returns.

“No Stone Unturned,” GMO.  The firm’s emerging markets debt team argues for a “cash-plus” approach to evaluating its results, citing that there are “two major flaws in being overly wedded to an off-the-shelf benchmark”:

There are embedded characteristics, or betas, that may not necessarily be desirable.

If a manager’s opportunity set is constrained by their benchmark, potential alpha opportunities may be de-emphasized, or even ignored completely.