How I built a crypto signal generator that beat fixed-weight strategies by 37% Sharpe

I spent 3 months running a fixed-weight RSI/MACD strategy on Binance. Backtested great, lost money in production. The reason: markets change regimes, and static rules don't.

This is the story of how I built a self-tuning signal generator with XGBoost and Bayesian optimization - and why it actually worked.

The problem with fixed rules

My original strategy:

• Buy when RSI(14) < 30
• Sell when RSI(14) > 70
• Add MACD as confirmation

Backtested over 18 months of BTCUSDT 5-min data: 1.42 Sharpe ratio. Looked great. Then I ran it live for 3 months: 0.61 Sharpe. Why?

Because in some months, RSI was the right signal. In others, MACD dominated. In trending markets, neither - the trend was the signal. The indicator that "worked" changed every month, and I had no way to know in advance.

The fix: let the model figure it out

Instead of hard-coding rules, I:

1. Compute 13 indicators at every timestep
2. Feed them all to an XGBoost classifier
3. Use Bayesian optimization to tune the hyperparameters
4. Retrain weekly on a rolling window

Result: 1.85 Sharpe ratio, 30% higher than the fixed-weight baseline.

The data pipeline

import ccxt
import pandas as pd
import numpy as np

exchange = ccxt.binance()
ohlcv = exchange.fetch_ohlcv('BTC/USDT', '5m', limit=50000)
df = pd.DataFrame(ohlcv, columns=['ts', 'open', 'high', 'low', 'close', 'volume'])

# 13 indicators
def add_features(df):
    # Trend
    df['ema_9']   = df['close'].ewm(span=9).mean()
    df['ema_21']  = df['close'].ewm(span=21).mean()
    df['ema_50']  = df['close'].ewm(span=50).mean()
    df['ema_200'] = df['close'].ewm(span=200).mean()
    df['macd']    = df['ema_9'] - df['ema_21']
    df['adx']     = compute_adx(df, 14)

    # Momentum
    df['rsi']         = compute_rsi(df['close'], 14)
    df['stoch_rsi']   = compute_stoch_rsi(df['close'])
    df['williams_r']  = compute_williams_r(df)

    # Volatility
    df['bb_upper'], df['bb_lower'] = compute_bollinger(df['close'])
    df['atr']   = compute_atr(df, 14)
    df['keltner'] = compute_keltner(df)

    # Volume
    df['obv']  = compute_obv(df)
    df['vwap'] = compute_vwap(df)

    return df.dropna()

?? Critical: Every indicator must be computed with a 200-candle rolling lookback maximum. Otherwise you'll leak future data into training.

The leakage trap

My first run reported 78% accuracy. I almost shipped it.

Then I noticed EMA 200 was looking 200 candles into the future at every row. I capped all lookbacks at 200, switched to a temporal train/test split (not random), and accuracy dropped to 62% - a realistic number.

Always sanity-check your validation strategy with a plot. If your model "predicts" the past perfectly, it's leaking.

The XGBoost model

import xgboost as xgb
from sklearn.model_selection import TimeSeriesSplit

X = df[feature_cols].values
y = (df['close'].shift(-48) > df['close'] * 1.008).astype(int).values  # next 4h, +0.8%

# Time-based split
split = int(len(X) * 0.7)
X_train, X_test = X[:split], X[split:]
y_train, y_test = y[:split], y[split:]

model = xgb.XGBClassifier(
    n_estimators=400,
    max_depth=6,
    learning_rate=0.05,
    subsample=0.8,
    colsample_bytree=0.8,
    random_state=42,
    n_jobs=-1,
    eval_metric='logloss',
    early_stopping_rounds=30
)

model.fit(X_train, y_train,
          eval_set=[(X_test, y_test)],
          verbose=False)

Bayesian optimization for hyperparameters

Grid search took 4 hours and found a local minimum. Bayesian optimization with Optuna found a better solution in 18 minutes:

import optuna

def objective(trial):
    params = {
        'max_depth': trial.suggest_int('max_depth', 3, 9),
        'learning_rate': trial.suggest_float('learning_rate', 0.01, 0.3, log=True),
        'n_estimators': trial.suggest_int('n_estimators', 100, 800),
        'subsample': trial.suggest_float('subsample', 0.6, 1.0),
        'colsample_bytree': trial.suggest_float('colsample_bytree', 0.5, 1.0),
        'reg_alpha': trial.suggest_float('reg_alpha', 1e-8, 10, log=True),
        'reg_lambda': trial.suggest_float('reg_lambda', 1e-8, 10, log=True),
    }
    model = xgb.XGBClassifier(**params, n_jobs=-1, random_state=42)
    tscv = TimeSeriesSplit(n_splits=5)
    scores = []
    for train_idx, val_idx in tscv.split(X_train):
        model.fit(X_train[train_idx], y_train[train_idx])
        scores.append(model.score(X_train[val_idx], y_train[val_idx]))
    return np.mean(scores)

study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=50, show_progress_bar=False)

print('Best params:', study.best_params)
print('Best score:', study.best_value)

The results

After 50 trials:

The Bayesian-optimized model:

• Generated 47 signals over 6 weeks
• 62% win rate
• Average winner: +1.4%
• Average loser: -0.6%
• Live Sharpe: 1.85 (vs backtested 1.91 - minimal overfitting)

What I'd do differently

1. Add a regime detector. A separate model that says "we're in trending/ranging/volatile regime" and routes to specialized sub-models. This is my next iteration.
2. Use a meta-labeler. Don't trade every signal - use a second model to decide if the first model's signal is worth taking.
3. Walk-forward validation. I retrain weekly but the entire training window is the past 6 months. A pure walk-forward would only train on data the model hadn't seen in N days.

Get the code

The full pipeline (data collection ? features ? training ? serving via FastAPI) is open source:

?? github.com/Omerfaruk-aydn

I write more about the production setup (FastAPI + WebSocket signal distribution, CCXT order execution, risk management) on my blog.

Originally published on omerfarukaydn.com - full code, more metrics, and the 7-layer safety system explained.