Transformers Beat LSTMs 62% of the Time — But Not How You'd Expect
Ran both architectures on 252 trading days of S&P 500 data. Transformer won directional accuracy 62% vs LSTM's 58%. But here's the catch: LSTM's losses were smaller. When Transformer was wrong, it was really wrong — average error 2.3% vs LSTM's 1.1% on missed days.
This isn't another "Transformers are the future" post. It's a side-by-side implementation where I tracked every metric that matters for actual trading: directional accuracy, mean absolute error, Sharpe ratio of a simulated strategy, and training time. The results don't fit the narrative you'd expect from reading ML Twitter.
The Setup: Same Data, Fair Fight
Pulled SPY daily data from 2023-01-03 to 2024-12-31 using yfinance. 252 trading days, split 80/20 train/test. Features: close price, volume, 5-day MA, 20-day MA, RSI(14), normalized to [0,1] with MinMaxScaler.
Continue reading the full article on TildAlice
