Modern Portfolio Theory from Scratch: The Efficient Frontier

Nobody can reliably predict the market. Be wary of anyone who claims otherwise. What is quantifiable is risk, and managing a portfolio is really about managing risk. The surprising part is that combining two volatile assets can produce a portfolio calmer than either one alone. A US equity fund running at 15% annual volatility pairs with a commodity ETF at 20%, yet the 50/50 mix comes in below 13%. They do not move in lockstep; when one zigs, the other tends to zag.

It turns out Harry Markowitz formalised this intuition in 1952, and it won him a Nobel Prize. The core insight is deceptively simple: what matters is not how risky each asset is in isolation, but how they move together. Two volatile assets that zig and zag at different times can combine into a portfolio that's calmer than either one alone. By the end of this post, you'll compute optimal portfolios using nothing but matrix algebra, trace the efficient frontier with Monte Carlo simulation, and understand the capital market line that underpins modern asset allocation.

The Power of Diversification

Before we dive into the code, watch what happens as we add assets to a portfolio one at a time. Each new, imperfectly correlated asset pushes the frontier further to the left, unlocking risk-return combinations that were previously impossible:

With just US Equity, you're stuck at a single point (15% vol, 11% return). Adding International Equity creates a curve of possible portfolios. Adding Real Estate and Commodities pushes the frontier left and up, reaching lower volatility than any individual asset. This is the Markowitz free lunch: diversification reduces risk without sacrificing return.

Let's Build It

Click the badge to run this yourself:

import numpy as np
import matplotlib.pyplot as plt

np.random.seed(42)

# Expected annual returns and covariance matrix for 4 asset classes
asset_names = ["US Equity", "Intl Equity", "Real Estate", "Commodities"]
mu = np.array([0.11, 0.16, 0.06, 0.04])  # expected returns

# Covariance matrix: vol = 15%, 20%, 17%, 16%
# Correlations: US-Intl 0.6, US-RE 0.3, US-Com 0.0, Intl-RE 0.35, Intl-Com 0.1, RE-Com 0.15
cov_matrix = np.array([
    [0.0225, 0.0180, 0.0077, 0.0000],   # US Equity (15%)
    [0.0180, 0.0400, 0.0119, 0.0032],   # Intl Equity (20%)
    [0.0077, 0.0119, 0.0289, 0.0043],   # Real Estate (17%)
    [0.0000, 0.0032, 0.0043, 0.0256],   # Commodities (16%)
])

n_portfolios = 50_000
n_assets = len(mu)

# Generate random portfolio weights (long-only)
all_weights = np.random.dirichlet(np.ones(n_assets), size=n_portfolios)

# Compute return and volatility for each portfolio
port_returns = all_weights @ mu
port_volatilities = np.sqrt(
    np.array([w @ cov_matrix @ w for w in all_weights])
)
sharpe_ratios = port_returns / port_volatilities

# Find special portfolios
min_vol_idx = np.argmin(port_volatilities)
max_sharpe_idx = np.argmax(sharpe_ratios)

print(f"Min-variance portfolio:  return={port_returns[min_vol_idx]:.2%}, "
      f"vol={port_volatilities[min_vol_idx]:.2%}")
print(f"Max-Sharpe portfolio:    return={port_returns[max_sharpe_idx]:.2%}, "
      f"vol={port_volatilities[max_sharpe_idx]:.2%}, "
      f"Sharpe={sharpe_ratios[max_sharpe_idx]:.2f}")

Min-variance portfolio:  return=7.36%, vol=10.38%
Max-Sharpe portfolio:    return=11.65%, vol=13.21%, Sharpe=0.88

Each dot is a portfolio with different asset weights. The cloud has a distinctive "bullet" shape: you can increase expected return, but only by accepting more volatility. The left edge of the cloud, the efficient frontier, represents portfolios where no further risk reduction is possible for a given return level.

Notice that the minimum-variance portfolio has a volatility of 10.4%, which is lower than any individual asset (the least volatile asset, US Equity, has 15% volatility). This is diversification at work.

What Just Happened?

Portfolio Return: A Weighted Average

The expected return of a portfolio is simply the weighted sum of individual asset returns:

Where $\mathbf{w}$ is the vector of portfolio weights and $\boldsymbol{\mu}$ is the vector of expected asset returns. With equal weights $[0.25, 0.25, 0.25, 0.25]$ across our four assets, the portfolio return is $(0.25)(0.11) + (0.25)(0.16) + (0.25)(0.06) + (0.25)(0.04) = 9.25\%$. Nothing surprising here.

This is exactly what the original R code computed with xABC %*% muABC, a single matrix multiplication of weights times returns.

Portfolio Variance: Where the Magic Happens

The portfolio variance is where diversification shows its power:

Where $\boldsymbol{\Sigma}$ is the covariance matrix. This quadratic form means that off-diagonal terms (covariances between assets) matter as much as the diagonal terms (individual variances). When covariances are low or negative, the cross-terms shrink the total variance below what you'd get from a simple weighted average of individual variances.

# The original R code's calculation, translated to Python
w = np.array([1/3, 1/3, 1/3])
mu_3 = np.array([0.01, 0.04, 0.02])
cov_3 = np.array([
    [0.10,  0.30,  0.10],
    [0.30,  0.15, -0.20],
    [0.10, -0.20,  0.08],
])

port_return = w @ mu_3
port_variance = w @ cov_3 @ w
print(f"Portfolio return:   {port_return:.4f}")
print(f"Portfolio variance: {port_variance:.4f}")
print(f"Portfolio std dev:  {np.sqrt(port_variance):.4f}")

Portfolio return:   0.0233
Portfolio variance: 0.0811
Portfolio std dev:  0.2848

Notice the covariance between assets B and C is $-0.20$. This negative covariance is doing the heavy lifting: when B goes up, C tends to go down, and vice versa. Their movements partially cancel out in the portfolio.

The Dirichlet Trick for Random Weights

To explore the space of all possible portfolios, we need random weight vectors that sum to 1 and are non-negative (no short selling). The Dirichlet distribution is perfect for this. With np.random.dirichlet(np.ones(4)), each draw is a point on the 3-simplex: four non-negative numbers that sum to exactly 1. This gives us uniform coverage of the weight space without rejection sampling.

Finding the Efficient Frontier

The upper-left boundary of the scatter cloud is the efficient frontier. Every portfolio on this curve is Pareto optimal: you cannot improve return without increasing risk, or reduce risk without sacrificing return. Portfolios below the frontier are suboptimal because you can always find a portfolio with the same risk but higher return (or the same return but lower risk).

# Extract the efficient frontier by binning volatilities
n_bins = 100
vol_bins = np.linspace(port_volatilities.min(), port_volatilities.max(), n_bins)
frontier_returns = []
frontier_vols = []

for i in range(len(vol_bins) - 1):
    mask = (port_volatilities >= vol_bins[i]) & (port_volatilities < vol_bins[i + 1])
    if mask.sum() > 0:
        best_idx = np.argmax(port_returns[mask])
        frontier_returns.append(port_returns[mask][best_idx])
        frontier_vols.append(port_volatilities[mask][best_idx])

print(f"Frontier has {len(frontier_vols)} points")

Frontier has 99 points

Going Deeper

The Minimum-Variance Portfolio

The portfolio with the lowest possible volatility is found analytically. For the unconstrained case (weights can be negative, i.e., short selling allowed):

Where $\mathbf{1}$ is a vector of ones. This is a system of linear equations: invert the covariance matrix, multiply by ones, and normalise so weights sum to 1.

ones = np.ones(n_assets)
inv_cov = np.linalg.inv(cov_matrix)
w_mv = inv_cov @ ones / (ones @ inv_cov @ ones)

print("Minimum-variance weights (analytical):")
for name, weight in zip(asset_names, w_mv):
    print(f"  {name}: {weight:.1%}")
print(f"Return: {w_mv @ mu:.2%}, Vol: {np.sqrt(w_mv @ cov_matrix @ w_mv):.2%}")

Minimum-variance weights (analytical):
  US Equity: 41.6%
  Intl Equity: -1.1%
  Real Estate: 20.9%
  Commodities: 38.7%
Return: 7.19%, Vol: 10.37%

The analytical solution loads heavily into US Equity (42%) and Commodities (39%), both of which have zero correlation with each other. The small negative weight on International Equity means the unconstrained solution shorts a tiny amount of the highest-variance asset. In practice, most investors add a long-only constraint (all weights $\geq 0$), which is what our Monte Carlo enforces through the Dirichlet distribution.

The Sharpe Ratio and Tangency Portfolio

The Sharpe ratio measures risk-adjusted return:

Where $R_f$ is the risk-free rate. For simplicity, we set $R_f = 0$. The tangency portfolio maximises the Sharpe ratio. Analytically:

w_tan = inv_cov @ mu / (ones @ inv_cov @ mu)

print("Max-Sharpe (tangency) weights:")
for name, weight in zip(asset_names, w_tan):
    print(f"  {name}: {weight:.1%}")
sharpe = (w_tan @ mu) / np.sqrt(w_tan @ cov_matrix @ w_tan)
print(f"Return: {w_tan @ mu:.2%}, Vol: {np.sqrt(w_tan @ cov_matrix @ w_tan):.2%}, "
      f"Sharpe: {sharpe:.2f}")

Max-Sharpe (tangency) weights:
  US Equity: 41.1%
  Intl Equity: 39.5%
  Real Estate: 1.1%
  Commodities: 18.2%
Return: 11.64%, Vol: 13.20%, Sharpe: 0.88

The tangency portfolio is dramatically different from the minimum-variance portfolio. International Equity jumps from essentially 0% to 40% of the portfolio, because it has the highest individual Sharpe ratio (16%/20% = 0.80). The optimizer is willing to accept higher volatility (13.2% vs 10.4%) because the extra return more than compensates. Real Estate, with a weak Sharpe ratio (6%/17% = 0.35), nearly vanishes.

The Capital Market Line

If you can borrow and lend at the risk-free rate, the best strategy is to combine the risk-free asset with the tangency portfolio. This creates a straight line in risk-return space called the capital market line (CML):

Every point on this line dominates the efficient frontier (except at the tangency point where they touch). Want less risk? Hold some cash and some tangency portfolio. Want more return? Leverage up the tangency portfolio.

When Does MPT Break Down?

Modern Portfolio Theory relies on several assumptions that rarely hold perfectly in practice.

The most dangerous assumption is that the covariance matrix is stable. The chart above illustrates the problem: during calm markets, correlations between assets stay low (US-Intl correlation around 0.6), providing the diversification benefits we computed. But during crises, correlations spike towards 1.0. In the 2008 financial crisis and the March 2020 COVID crash, virtually all risky assets fell simultaneously. Diversification evaporates precisely when you need it most.

Other key limitations:

1. Returns are not normally distributed. Real asset returns have fat tails. The 2008 crash was a "25-sigma event" under normality, meaning it should essentially never happen.
2. Investors care about more than mean and variance. Real investors also worry about drawdowns, liquidity, and the possibility of total loss.
3. Expected returns are impossible to estimate accurately. Small errors in expected return estimates can produce wildly different optimal portfolios. This is the "error maximisation" problem identified by Michaud (1989).

Despite these limitations, the framework remains the starting point for virtually all quantitative asset allocation. Extensions like the Black-Litterman model address the estimation problem by blending market equilibrium with investor views.

Try It Yourself

1. Change the correlations. Set all off-diagonal covariances to zero and re-run the Monte Carlo. How does the frontier shape change? (Hint: the minimum-variance portfolio gets even less volatile.)
2. Add a constraint. Require each asset to have at least 5% weight. How does this affect the minimum-variance portfolio?
3. Use real data. Download returns for SPY, EFA, VNQ, and DJP from Yahoo Finance using yfinance, compute the sample mean and covariance, and plot the frontier with real numbers.

Where This Comes From

Harry Markowitz published "Portfolio Selection" in the Journal of Finance in March 1952, while a graduate student at the University of Chicago. The paper was just 14 pages long. His doctoral advisor, Milton Friedman, reportedly questioned whether it was even economics. Decades later, Markowitz shared the 1990 Nobel Prize in Economics for this work.

The key insight was treating portfolio construction as an optimisation problem. Before Markowitz, investors selected securities based on expected return alone. Markowitz showed that a rational investor should consider the entire distribution of portfolio returns, and that the variance (or standard deviation) is a natural measure of risk.

"The portfolio with maximum expected return is not necessarily the one with minimum variance. There is a rate at which the investor can gain expected return by taking on variance, or reduce variance by giving up expected return."

Markowitz, H. (1952). Portfolio Selection. The Journal of Finance, 7(1), 77-91.

The Mathematical Framework

Markowitz framed the problem as a quadratic programme: minimise portfolio variance subject to a target expected return and the constraint that weights sum to 1.

Using Lagrange multipliers, the solution traces out a parabola in $(\sigma, \mu)$ space. This parabola is the efficient frontier.

James Tobin extended the framework in 1958 by introducing a risk-free asset, giving us the capital market line and the separation theorem: the optimal risky portfolio (the tangency portfolio) is the same for all investors, regardless of risk preference. Conservative investors hold more cash; aggressive investors leverage up, but they all hold the same risky portfolio.

William Sharpe built on this in the 1960s to derive the Capital Asset Pricing Model (CAPM), which says that in equilibrium, the tangency portfolio is the market portfolio. The Sharpe ratio we computed above is named after him.

Further Reading

• The original paper: Markowitz, H. (1952). Portfolio Selection. The Journal of Finance, 7(1), 77-91. Read Section II for the efficient frontier derivation.
• The full treatment: Markowitz, H. (1959). Portfolio Selection: Efficient Diversification of Investments. John Wiley & Sons. The book-length expansion with detailed numerical examples.
• Accessible introduction: Elton, E. J. et al. (2014). Modern Portfolio Theory and Investment Analysis (9th ed.). Chapters 4-6 cover mean-variance optimisation clearly.
• Next algorithm to learn: If you're interested in time series applications of covariance estimation, see our post on cointegration and pairs trading, which uses similar linear algebra on financial time series.

Related Posts

• Cointegration and Pairs Trading — Another quant finance application using linear algebra on financial time series, testing for long-run equilibrium between asset pairs.
• Hyperparameter Optimisation: Grid vs Random vs Bayesian — The Sharpe ratio is an objective function; Bayesian optimisation offers a smarter search over portfolio weights than Monte Carlo.
• Linear Regression Five Ways — The matrix algebra here (least squares, covariance matrices, inverses) is the same toolkit used in portfolio optimisation.
• Changepoint Detection: Finding Regime Shifts — Portfolio covariance matrices change over time; detecting when a regime shift has occurred helps with dynamic allocation.
• Gaussian Process Regression — Another application of covariance matrices (kernel functions), showing how the same linear algebra creates flexible probabilistic predictions.

Frequently Asked Questions

What is the efficient frontier and why does it matter?

The efficient frontier is the set of portfolios that offer the highest expected return for each level of risk (volatility). Any portfolio below this curve is suboptimal because you could achieve higher returns without taking on additional risk. It provides a principled framework for deciding how to allocate capital across assets.

How does diversification reduce portfolio risk?

When assets are imperfectly correlated, their price movements partially cancel each other out. The portfolio variance formula includes cross-terms (covariances) that can shrink the total variance below the weighted average of individual variances. In our example, the minimum-variance portfolio had lower volatility than any single asset.

What is the Sharpe ratio and how is it used in portfolio construction?

The Sharpe ratio measures risk-adjusted return: expected return divided by volatility (assuming a zero risk-free rate). The tangency portfolio maximises this ratio, representing the best trade-off between return and risk. Investors can then combine the tangency portfolio with cash to match their personal risk tolerance.

Why does Modern Portfolio Theory break down during market crises?

MPT assumes that correlations between assets remain stable over time. During crises, correlations tend to spike towards 1.0, meaning all risky assets fall together. Diversification benefits evaporate precisely when they are needed most, which is the most dangerous limitation of the framework.

Can I use MPT with real market data instead of assumed returns?

Yes. You can download historical returns from a data provider, compute the sample mean return vector and covariance matrix, and run the same optimisation. However, expected return estimates from historical data are notoriously noisy, and small estimation errors can produce wildly different optimal portfolios.

What is the capital market line?

The capital market line (CML) is the straight line from the risk-free rate through the tangency portfolio on the efficient frontier. It represents portfolios that combine the risk-free asset with the optimal risky portfolio. Every point on this line dominates the curved efficient frontier except at the tangency point.