I build financial models.
Then try to
break them.

A row in the log looked like a win that should have been a loss. The row turned out to be correct, but chasing it exposed a bigger problem. The system decided whether a trade had won by comparing its own copy of the price candles, while the market settles on its own published result. The two disagreed often enough to matter. My code also scored an exact tie as a loss, where the market’s rules resolve a tie as a win.

I switched settlement to the market’s official winner, kept the candle comparison only as a fallback, and wrote a backfill that re-checked every historical row. Of 4,923 rows, 551 had the wrong outcome, roughly one in nine. Twenty-one belonged to longer-horizon markets that hadn’t resolved yet and were left open.

LessonA backtest is only as good as its definition of a win. Every P&L figure I had produced before this was suspect until it was recomputed against the source of truth.

The strategy log said I was up by several times what was actually in the wallet. The log recorded orders when they were placed, and some were never filled. It also knew nothing about earlier losses. I stopped treating the log as the record of what happened.

Real P&L is now the live wallet balance minus a stored baseline, shown next to the log’s figure so any gap is visible at once. Underneath sits a daily ledger built from an append-only transaction file: every buy, payout and fee, with a running balance. Building it turned up two things. The exchange’s events arrive slightly out of order, so balances have to be computed in file order, not timestamp order. And my own first version anchored to a start timestamp that was off by a year, which counted every historical fill as new and sent the running balance deeply negative. I caught it because the running balance was plainly impossible.

LessonReconcile to the one number you can’t argue with, which is the balance held by the custodian. When the model and the balance disagree, the model is wrong.

Two live strategies won 90% or more of their trades and still finished negative. Their paper-trading twins, running the same logic, looked solidly profitable. The gap was about three to four percent of notional per trade, and it broke into three parts: the taker fee (around 0.8% at those prices), a slippage buffer on fill-or-kill orders (around 2.3%), and adverse selection, because the orders that fill are disproportionately the ones the market was about to move against.

When contracts are bought near 0.90, the break-even win rate is about 90%, so a few points of friction consume the whole edge. Paper trading had been filling at the quoted ask with none of those costs. I added baseline strategies, which buy a fixed side on every bar, to measure what it costs just to play. That toll came to one to three cents per trade at the ask, and every taker strategy has to clear it before it earns anything.

LessonA simulated result isn’t evidence until the fill model is at least as pessimistic as the real order book.

In the same audit, a control strategy that bought “down” on every bitcoin bar made money over the two-week window. It had no signal at all. Bitcoin had simply drifted down.

That changed how I read every other winner. A strategy whose gains resemble a short position in the asset may be carrying market exposure and not producing anything of its own. The audit’s rule is that before a winner is promoted, its returns get regressed against the market move using the factor-exposure module I built into the backend, and only what’s left over counts.

LessonEvery result needs a control that shows what doing nothing clever would have earned.