[Oct 21] Rechecking Live Performance — A Cold-Blooded Verdict Against the Index, and a Design That Actually Makes Money

Asked an AI advisor for a hard-nosed verdict on whether live returns are beating the index, then pushed it further for a structure that actually pays off after costs.

This is the English version of a post originally written in Korean for my algorithmic trading system devlog(new tab).

Is it actually worse than the index?

There are stretches where looking at the live account's cumulative return just feels bad.

If it's doing worse than simply buying and holding the index, there's no reason to keep pouring resources into this system. So today I asked an AI advisor to give me a verdict with no sugarcoating — spell out whether the right call is to stop, scale down, or keep going, and be explicit about what the evidence can and can't tell us.

The answer that came back didn't quite match the gut feeling.

Against the benchmark set in advance, the account wasn't actually behind. But that didn't mean there was any evidence it was picking stocks well, either. Most of the loss traced back to the market being bad overall — the contribution from stock selection itself was close to zero.

Meanwhile, the costs from turnover were coming out for certain, every single day. So the advisor's conclusion was: no reason to stop right now, but going forward the decision criteria need to pin down evidence of real skill and a hard limit on relative drawdown more explicitly.

"Don't give me monitoring rules — give me a structure that actually makes money"

The first verdict wasn't enough.

I asked again — not for rules that just decide when to stop, but for an actual structure that can beat the index after costs. The premise was that with more than three months of track record behind it, the system's strengths and weaknesses should be distinguishable by now.

What came back this time split the capital into two layers.

A large share of the capital gets parked in something that simply replicates the index, which structurally removes the risk of underperforming it at all. Only a small remaining slice gets concentrated into the areas where a real signal had actually shown up in past observations.

And before that small slice gets any bigger, the proposal calls for pre-registering a handful of hypotheses and testing them first. It's a safeguard against fitting the criteria to the results after the fact.

Neither piece of advice has touched any code yet — both are sitting there waiting on approval.

Also today — closed out a small loose end

A ticket tracking whether a nightly batch job's data-correction tail stayed within expected bounds got closed today, after three straight days of normal readings.

Why this matters

The core of today was separating "this feels bad" from "what the evidence actually says."

Looking at cumulative return alone, it looks rough — but whether that's because the whole market was bad or because the system's choices were bad are two different questions. Mixing them up means fixing the wrong thing.

The other piece was stepping outside the stop-or-continue binary.

A structure that caps the downside with index replication and only makes small bets where there's validated signal turned out to be a third option between the two. Though it's still just a proposal — whether it actually pays off depends on approval and testing still to come.