Executive Summary

This paper introduces Impact Cubed – a quantitative data-driven model to measure the sustainability of a portfolio of listed assets against a benchmark. We describe the concepts that lay the foundations of this risk factor model based tool, its design principles, and the data it currently uses.

Impact Cubed’s purpose is to bring greater transparency to the Responsible Investment industry. We demonstrate how it can do this by running the model on 25 European ESG funds whose holdings are public, against a global developed markets benchmark.

We found that there are some consensus strengths and weaknesses among European ESG funds. They clearly prioritize the provision of environmentally and socially positive products and services by their portfolio companies, but in turn have lower board independence, gender equality, and social impacts than their benchmark.

In terms of how much European managers are prepared to deviate from the benchmark to achieve better sustainability, our results show it isn’t much. The highest performer in our sample used roughly 11% of their risk budget on sustainability, with figures around the 4-5% mark being the most common.

The most striking finding from this exercise however is that these is a large dispersion of results among this group of funds marketed as sustainable. There are funds in the sample that do deliver higher sustainability than a market portfolio, but there are also several either overall less sustainable than their mainstream benchmark, or not substantially better. Comparing the holdings with benchmark constituents on a set of broadly accepted ESG indicators makes this clearly visible, however the same distinction would be impossible to make based on only the information issued by the funds themselves.



Over the last decade the responsible investment industry has boomed, with assets growing at more than double the rate of the asset management market overall. Many clients have asked their managers the question ‘Do you integrate ESG?’, and managers with the ESG (environmental, social and governance) edge have been more successful in growing their assets under management (AUM)1. One can understand the temptation among market participants to slightly oversell their ESG capabilities to keep up with this demand. But as ESG investing grows more sophisticated and enters the next stage in its development, the next big question is more specific: ‘What impact does your ESG fund have and how much exactly?’

But quantifying impact has traditionally been elusive. Some progress has been made for private investments in specific projects (e.g. the Pacific Community Ventures framework for measuring job quality impact). But it is the large public companies rather than these smaller private investments that consume vast amounts of our finite natural resources. They ‘optimise’ their labour force to lower their cost base, and lobby governments to endorse their environmentally and socially destructive practices. And it is predominantly in the listed companies where large proportions of our assets are invested via equity and fixed income funds.


So how do we quantify how impactful these funds are?


In this paper we present a model to estimate the Active ESG Shares in any portfolio of listed assets, which allows us to make statements such as ‘Fund A has twice the impact of Fund B’. We developed the model in response to a large Nordic investor’s demand for a single consistent metric for the amount of ‘ESG’ in a fund. Since the publication of the UN Sustainable Development Goals (UN SDGs), we have adopted them as guidance for how we capture the sustainability of listed companies. Our framework therefore measures the amount of ‘ESG’ in a fund by establishing how aligned the portfolio is with the UN SDGs. Our intention is that the model will allow investors to separate green investing from green marketing!

When thinking about quantifying impact in the context of portfolios of publicly listed companies, we focused on the cost of capital – lowering it for firms with positive externalities and raising it for firms with negative externalities. In a previous paper, The Collective Impact of Responsible Investors on the Cost of Capital2, we introduced the notion of Active ESG Shares and the link to a firm’s cost of capital. Our finding is that underweighting (not investing in) firms which are antagonistic to the SDGs (weapons of mass destruction, coal-based electricity generation, pornography, etc.) and instead investing in firms providing for human flourishing (healthcare, sanitation, education, renewable energy, plant-based protein sources, etc.) shifts the cost of capital in favour of a sustainable future.

This paper consists of two sections. The first part 1. Introducing the model describes the model’s framework and articulates the chosen factors. In the second, 2. Testing the model on 25 ESG funds the model is applied to 25 well-known ESG funds to measure their impact, and the results are presented.


1.         Introducing the model



 We developed a linear factor model framework that defines fourteen characteristics of a firm to capture how well it aligns with the SDGs. We obtain or estimate each characteristic for every company held by the fund and the fund’s benchmark. We can estimate Active ESG Shares from this, and a covariance matrix of returns for all these firms.

The underlying thinking is that holding the benchmark represents ‘business as usual’. So it is the manager’s deviations from the benchmark weights that are of interest. For example: is the manager underweight on coal? Is the manager overweight on weapons? Thus we first establish the fund’s active weights (these are the differences between the fund’s holdings and the holdings of the benchmark portfolio). We calculate the exposures of these active weights to each of the fourteen characteristics (these are the (active) weighted average of each firm’s characteristics.) These exposures are referred to as the fund’s active factor exposures.

We then use the covariance matrix to find the minimum variance portfolio that achieves these active factor exposures. Active ESG Shares are the sum of the absolute values of the active weights of this minimum variance portfolio.

Active ESG Shares are exactly like the ‘normal’ concept of Active Shares, but they represent the active part of the portfolio that is expressed through the fourteen factors used in the model. Our research shows that the Active ESG Shares average about 5% to 10% of total Active Shares for the ESG funds that we have examined.

Active ESG Shares are by definition less than (or at most equal to) the total Active Shares of the portfolio. Active ESG Shares together with AUM and a demand elasticity assumption provide an estimate of the shift in cost of capital created by the fund. You can read one of our previous papers, titled The Collective Impact of Responsible Investors on the Cost of Capital, for more detailed background.


Model specification:

Using this notion, if we define:

Wp: 1 x N vector of portfolio weights Wb: 1 x N vector of benchmark weights

X: N x K matrix of K factor values (in our case K = fourteen impact factors) V: N x N covariance matrix of stock returns


Active Weights are then:

Wa = Wp – Wb

Total Active Shares = sum(abs(Wa))

The active exposures to the K factors are calculated as: AFE = Wa * X


Next, we use a quadratic optimisation to find the minimum variance portfolio that achieves the same or better factor exposures:

Choose Wm to minimise: (Wm * V * transpose(Wm)) subject to Wm * X >= AFE

Active ESG Shares are then: sum(abs(Wm))


Lastly, we can calculate the risk of the total Active Shares (known as tracking error, or TE) and the risk in the Active ESG Shares, which is what we use as our single impact metric to compare funds:

Impact = sqrt(Wm * V * transpose(Wm))


Description of impact measures chosen

In choosing the impact measures to include in the model, we were guided by several considerations:


To span the sustainability space

We considered all the major ways in which public companies affect people and the planet. It is not only through the products and services they provide or do not provide, but also how they operate, and even where geographically they operate or provide their services. With this in mind, we selected an optimal number of measures that give a composite picture of how sustainable a company is (given our current collective understanding of sustainability, and as validated by the UN SDGs). Before we add any new company characteristic into the model, we undertake a relevance test.


To be based on publicly available data, and ability to estimate non-reporters

There are many impact frameworks in development currently, and a multitude of ways to think about and accurately measure sustainability. However, we place more value on a framework that is practical over one that is conceptually perfect but impossible to apply (because of data availability issues). We have based our measures around publicly available data – but, in the instances where data is not available, our measures are built from estimates we are able to make confidently.


To use well-accepted indicators in existing ESG frameworks

 We were determined not to add more confusion to the already very noisy conversation on sustainability metrics. So, instead of inventing new metrics, we employed the most broadly accepted and understood ones. For instance, a carbon footprint may not be a perfect measure of environmental sustainability, but most of us accept and use it without further explanation needed.


To avoid subjectivity as best as possible; we avoid weighted averages of the policy, strategy, management or stated intentions

The ESG research and responsible industries have been plagued by subjectivity. Asset managers hire their own in-house ESG analyst teams to avoid research provider analyst subjectivity. Research providers devise more and more complex and multi-layered methodologies in the hope of reining in individual analyst bias in their research products. By using publicly available data and broadly accepted measures in our model, we intentionally manage subjectivity. We do not make judgement calls in the calculation of our measures, and the process is data driven and fully automated.


Preference for outcome indicators

 Another method that we select to prevent subjectivity is to focus on outcome indicators. For example, when we endeavour to measure gender equality, we do not look at the declarations or the company’s intent on what it is trying to accomplish through empowerment and mentoring programmes – we focus purely on outcomes and measure how gender diverse their top management actually is.

Over the last eighteen months, the model has been refined through a series of workshops in more than ten cities in Europe and North America to ensure a wide variety of opinions from a diverse group of investors. Participants ranged from impact-oriented family offices and small sustainability-dedicated asset managers, to large generalist asset managers and public pension funds. We gained feedback across the investment value chain from service providers and investment consultants before we developed the fourteen measures described below.


Carbon efficiency

Carbon equivalent, or greenhouse gas equivalent, efficiency measures how company operations link to climate change by indicating how much greenhouse gases a company emitted to earn one unit of revenue. This measure is traditionally called ‘carbon footprint’ in the responsible investment industry. This measure (including underlying data) is sourced from several commercial providers and covers carbon scope 1 and scope 2 emissions.


Waste efficiency

Waste efficiency, also known as the ‘waste footprint’, is a closely related cousin of carbon footprint, but, instead of carbon emissions, it describes how much waste a company generated to earn one unit of revenue. Waste accounting frameworks are developing, and we support any initiatives that refine and improve waste accounting by companies.


Water efficiency

Water efficiency is the ‘water footprint’ and measures how many litres of water a company used in generating one unit of revenue. We treat every litre of water uniquely – determining whether the litre was collected from rainwater, groundwater or cleaned waste water. We do not count any brackish/saltwater used, because it is not typically a resource also needed by local communities.

Some water utility companies report the litres of water that they have delivered as the litres of water they have used to create a unit of revenue. This inflates the water number for water utility companies, as it double-counts the same water.


Gender equality

Gender equality has a distinct dedicated SDG goal. In relation to companies’ impact, we measure the gender balance of their top management. As a starting point, we use the board-level data (often publicly available globally). To broaden the metric, we amend it with the percentage of women in the C-suite and other top management.


Executive pay

Executive pay measures the ratio of companies’ top management compensation compared with average employee compensation. Initially the CEO’s total compensation is used, and if possible we add other top management compensation, like board or C-suite, where available. All compensation includes short-term and long-term variable pay components, such as performance bonuses and stock options. The average employee compensation is the personnel expense divided by the number of employees. The personnel expense typically includes pension expense, for example. In cases where total personnel expense is not available, we use alternative means to estimate average employee compensation, such as country-level average salaries for employees in a particular industry combined with employee count.


Board independence

The quality of powerful institutions is mentioned in the SDGs. While this typically refers to government organisations, the governance quality for (large) companies is relevant given their systemic importance. The governance quality of any company can be measured in many ways; we chose board independence as the most representative metric.

We use the local stock listing requirements to determine whether a particular board member is independent or not. The majority owners of companies, and their appointed board members, may or may not be independent, depending on the jurisdiction that they are in. Some board members might serve on corporate boards for decades, while other jurisdictions have term limits as to how long a board member can be considered independent. In keeping with the goal of not introducing our own subjective judgement into the metrics, we do not override the local stock listing requirements with our own standard of board independence.


Business model indicators 

Our four business model indicators – environmental good, avoiding environmental harm, social good and avoiding social harm – are constructed in the same way. Each product and service is categorised into one of the four buckets or tagged as a neutral product or service. Broadly speaking, a fund avoids, for example, social harm by omitting classic sin stocks such as alcohol, tobacco, firearms and adult entertainment. Our departure point was an existing industry classification with around 2,500 product and service categories. From a sustainability point of view, many of them could be collapsed into broader categories, where added granularity does not change the sustainable impact of a product of service. As a result, our classification differentiates between 800+ products and services.

We have erred on the side of caution and neutrality in establishing the classification. There are many products and services that can be considered positive or negative depending on one’s point of view. Nuclear power, for example, can be considered positive because it creates a lot of electricity with relatively low carbon emissions. On the other hand, it can be considered negative because it creates hazardous waste as a side product. As a general rule, whenever a controversy like this existed, we classified that particular product or service as neutral.

As a result, around 15% of the products/services were categorised in the four positive or negative categories, with approximately 85% of them being neutral.

Subsequently each company’s revenue was classified into the 800 types of products and services. For example, a US$10m investment in one company that earns 70% of its revenue in oil exploration and 30% in solar panels is the same as a separate US$7m investment in oil exploration and US$3m investment in solar panels.

Geographic mapping (‘Geo’ in Table 1) is applied to the business models that are scored in only a limited number of countries. Access to telecommunications is a specific sub-goal in the SDGs, but it applies only in a limited number of countries that have a lack of access to telecommunications, and so only revenues from those markets will be counted as positive. On the other hand, access to healthcare facilities is universally SDG supportive, so all healthcare facilities in every country are positive.


Examples of products and services from each category are shown in Table 1.


Table 1

Economic development

Much of the SDG text revolves indirectly around economic development. Foreign direct investment and resulting GDP growth have a positive impact in a large number of SDGs; prime examples are poverty reduction and bringing an end to hunger.

Companies operate, sell their goods and source raw materials internationally, and most look outside their domicile for growth opportunities. The more companies operate in the least- developed countries, the greater indirect positive impact they have.

This measure employs companies’ geographical spread of operations and matches it against GDP per capita. The outcome quantifies how their operations indirectly support economic growth in places where there is a lack of economic activity (as signified by low GDP per capita).


Avoiding water scarcity  

Water is a local natural resource. Its availability varies seasonally, and it is difficult to transport in large quantities over long distances. The availability of water depends greatly on where in the world it is used. Therefore, to accurately reflect a company’s sustainability impact with respect to water as a natural resource, we need to map its use geographically.

Water usage is typically categorised into three groups: domestic/municipal use, agricultural use and industrial use. By measuring the industrial use, we can establish how much competition the industrial use introduces against domestic use, and how much it contributes to the water scarcity in that particular area.

We factor in companies’ geographical spread of operations and align them to a water scarcity map. The water scarcity maps are sourced from World Resources Institute Aqueduct maps to establish where the water is abundant and where it is scarce.



Employment is one of the SDGs. We map companies’ geographical spread of operations against unemployment rates provided by the International Labour Organisation (ILO).


Tax gap

The tax gap measures the difference between the total amount of taxes owed and the total amount paid. Taxes are a form of wealth transfer, and thus the amount corporations pay (or do not pay) has ramifications on social services.

We analyse companies’ income statements to determine their pre-tax earnings and taxes paid. We compare that with their geographical spread of revenue sources and OECD-reported corporate tax rates by jurisdiction.

We use a five-year moving average to smooth the volatility over corporate pre-tax earnings (which fluctuate annually) over taxes paid (which also fluctuate annually but at a different pace than companies’ earnings) and corporate tax rates (which tend to vary over election cycles).



Data collection and sources

We primarily use data points that are provided by companies in their annual and quarterly filings, which we access through data aggregators. The raw data points include:

  • Company basic information
  • Domicile
  • Breakdown of products and services
  • Geographical breakdown of revenues and assets
  • Number of employees
  • Composition of top management and board
  • Company financial and sustainability information
  • Income statement, balance sheet
  • Top management compensation
  • Company reporting on externalities including greenhouse gases, water use and waste generation


The company-provided data is complemented by macroeconomic information from various sources such as:

  • World Resources Institute Aqueduct model for water scarcity
  • WORLD Bank / ILO / OECD for
  • Unemployment rates
  • GDP per capita
  • Statutory corporate tax rates


Most of the data points the model uses are part of listing requirements on different stock markets around the world, and are therefore available for all investors to use by default. In cases where the companies do not provide a specific data point, we use in-house machine learning methods to estimate missing data.

In this paper, the 25 sample funds are evaluated against an MSCI World benchmark. This benchmark consists of around 1,600 of the largest companies in 23 developed markets. In this universe, we estimate around 10% of data, which is normally distributed. Based on this, we approximate the error inevitably introduced by estimating missing data points to be negligible.



2.         Testing the model on 25 ESG funds



To test how the model compares different funds, we downloaded a list of ESG funds from the yourSRI.com website and then researched a publicly available source for each fund’s holdings.

  • We decided to focus on equity funds for this exercise, excluding fixed income, multi-asset and alternatives funds (based on yourSRI.com categorisation).
  • All funds are registered to be available in the UK (according to com).
  • Their investment universe is global developed markets, indicated by the use of MSCI World as their performance measurement benchmark (according to yourSRI.com).
  • They use Best-in-Class or ESG Integration as their ESG strategy. We did not include funds that were clearly thematic (investing only in clean technologies) or are simply excluding a small number of companies and/or industries. The categorisation of ESG themes follows the categorisation used by Eurosif in their European SRI Market


Below are listed the 25 funds of which we were able to obtain holdings:

  1. Aberdeen Global – Responsible World
  2. Acadian Sustainable Global Equity
  3. BSF Impact World Equity
  4. Candriam SRI Equity World
  5. Comgest Growth World
  6. DB Platinum CROCI World
  7. EdenTree Amity Internationa
  8. EdenTree Amity Global Equity Inc. Fd for Charities
  9. F&C Responsible Global Equity
  10. FP WHEB Sustainability
  11. Goldman Sachs GI Eq Partners ESG
  12. JPM Global Socially Responsible
  13. JSS Sustainable Equity – Global
  14. JSS OekoStar Equity – Global
  15. LGT Sustainable Equity Fund Global
  16. Liontrust Sustainable Future Global Growth
  17. LO Funds – Generation Global
  18. Mirova Global Sustainable Equity
  19. Morgan Stanley Global Opportunity
  20. Nordea Global Stars
  21. RobecoSAM Sustainable Global Equities
  22. Sarasin Responsible Global Equity
  23. Stewart Investors Wldwd Sustainability
  24. SWC PF Green Invest Equity
  25. Triodos Sustainable Equity Fund


We ran our model on the holdings of each of these funds, using capitalisation-weighted global developed markets as the benchmark. The summarised results are shown in Table 3 on the next page.


Table 3

It is worth noting which of our chosen factors are the most popular with these funds. In Table 3, one can see that all factors have a positive mean across the managers with the exception of board independence, gender equality and employment. It would appear that as managers look for firms with positive impact in their product mix, they sacrifice governance metrics – independent boards and a balanced management gender mix. The underweight exposure to employment impact appears to be the result of investing in companies who provide environmental solutions but do so in countries with relatively low rates of unemployment.

The most popular factors are economic development, avoiding social harm and investing in environmental good. In other words, the average ESG manager invests in companies that have more exposure to emerging markets and in companies that are providing environmental solutions (solar, LED lights, efficiency systems, etc.), while avoiding companies that are engaged in socially harmful activities (classic sin stocks – alcohol, gambling, adult entertainment, weapons, etc.). One pleasing discovery was that the average manager holds companies that use water in less drought-prone areas than the benchmark portfolio. This is a factor that we weren’t sure ESG managers were paying attention to.

You may recall from the model framework section that we measure impact as the risk of a minimum variance portfolio that achieves factor exposures that are at least as good as the exposures of the fund in question. By its nature, the impact number cannot be larger than the fund’s tracking error. To arrive at this one number, we calculate the fund’s positive and negative impacts and net them out. For instance, a fund may have holdings that are extremely carbon efficient but with no women in top management. In this case, the positive carbon efficiency impact will be partly or fully offset by the negative gender equality impact.

The positive and negative impact summary numbers, together with the net impact and an estimate of each fund’s tracking error, are reported in Figure 1. The first thing to note is the large spread in results – there are some very good and some very bad results. Secondly, two funds actually have an estimated negative impact – their negative factor exposures outweigh the positive factor exposures. We suspect that these managers focused on only one or two impact areas without managing their other externalities.

One expects to see more impact for higher tracking error funds. Just as return scales with risk along the security market line, impact scales with tracking error in active holdings. To make this point clearer, we have plotted the funds’ estimated impact and tracking error in a scatter diagram. The funds closest to the ‘efficient impact frontier’ are the more efficient funds in the risk-impact dimension. As mentioned, the construction of our impact measure is bound by the tracking error. In other words, you can’t have more impact than tracking error. Thus the efficient impact-risk frontier is at most the 45-degree line going through zero. In reality, no manager is using anywhere near all their tracking error on impact. The best manager in our sample set has an impact which is 11% of their tracking error (30 bps for 3% TE). The top ten managers have an impact above 5% of their tracking error. Unfortunately, six have an impact of less than 2% of their tracking error.



Figure 1

The maximum slope any fund achieves from the origin sets ‘the empirical efficient frontier’, and the further other funds are below this line, the worse its performance in impact-risk dimension. In our case, the best fund has 11% conversion of tracking error to impact. Take a fund’s TE and multiply by 11%, then subtract its net impact, and you get the distance below the efficient line – a measure of how inefficient each fund is relative to the best fund.


For our next paper, we will bring in the third dimension, expected return, by asking ‘what information ratio would we have to assume for each manager to prevent them being ‘dominated’ by other funds?’ Remember that information ratio links tracking error to excess return: so, together with fee, the information ratio will provide us with the third dimension – excess return – as well as tracking error and impact.



3.         Conclusion

The demand for responsible investment products by various institutional and private asset owners has grown exponentially. As an understandable response, a multitude of asset managers have brought a wide variety of different ESG funds to the market. By developing a unified impact measurement framework model and running it against a large enough selection of comparable ESG funds, we demonstrate two conclusions.


The total amount of ESG in any given ESG fund is low.

Most asset managers tout their ESG credentials loudly and market their specific product aggressively, which is understandable considering the explosion in demand for ESG. ESG claims have traditionally been easy to make and impossible for a client to verify. The introduced framework makes it clear that at a modest 11% of tracking error at most, the true measurable impact of any given ESG fund in a representative sample is far below what their messaging and positioning would imply. From running well over 100 funds through the model since its launch, we have found that funds with twice this percentage exist, and it is possible for sustainability to account for 30% to 50% more of an ESG fund’s tracking error.


There are big differences in impact between ESG funds.

ESG funds, like most investment funds, are sold based on few quantifiable metrics, the reputation of the management company, and the marketing message tied to the fund itself. Based on these, the ESG funds are indistinguishable from each other. The introduced impact measurement framework uncovers vast differences between ESG funds. In the worst cases, the impact is negative, and the fund investor would have been better off by investing in an index fund rather than investing in the ESG fund in question (this probably would have been cheaper too!). Excluding the few worst examples, it seems that there is about a ten-fold difference between the best and worst quartile ESG funds in terms of impact. All other things being equal, this translates into very meaningful differences between ESG funds.

There are a number of possible reasons for this. For example, different attributes of the management company (size and domicile for example), or differences in the investment process and the used data. But we leave this more specific research for future papers.

The purpose of this paper was not only to point out the potential for greenwashing among ESG asset managers and to alert ESG fund buyers to potential differences between ESG funds. We also wanted to raise a very sincere question about the responsible investment industry and the sustainable financial system. What you measure is what you get. What we need is a sustainable financial system that aggressively allocates ESG-aligned capital, not ESG funds that are misleading clients by making unfounded claims and are sustainable in name only.




1   http://www.pionline.com/article/20180108/PRINT/180109891/msci-links-esg-with-stronger-asset-growth

2 http://www.aurielequities.com/client-area/wp-content/uploads/2017/08/Auriel-Collective-Impact-in-Listed- Equity.pdf