I fired QuantDinger's two grid optima at the same instant — and failed to build the comparison

Feed measured costs back into the grid and the optimum slides from 60/180 to 120/360. Here is what happened when I tried to put both on the same quote, and what

You are done with this article when you can take two settings from your own tool, fire them at the same instant on the same quote with exactly one field changed, and decide up front whether that comparison can stand at all.

The human lines are real commands rewritten as conversation. The outputs are measured and I have not touched a single number.

1. The blank this tool kept leaving

In part 14 I ran QuantDinger's backtest grid and fed it the measured costs pulled from our own fill records (commission 0.0010, slippage 0.0001). The answer changed.

Raise the cost and the optimum slides toward the slower end. That was the single most valuable line in part 14.

So the plan this time was simple. Put both optima on the same quote at the same instant and see whether they really split on a live feed.

I could not. Two walls

① The grid rows contain no trades. Here is everything in the rows of the grid results (40-qd-grid-cp0.json, 40-qd-grid-cp1.json):

fast · slow · totalReturn · totalTrades · totalCommission · profitFactor · maxDrawdown · finalEquity

Not one per-trade entry or exit level. So "how do the two optima diverge on the same bar" cannot be answered from this record. I know there were 68 trades and 33 trades. I do not know where any of them happened.

② A period pair is not a field a trial order accepts. The trial-order request body (TrialBuyReq) has no fast or slow. And even if it did, our rule grammar caps periods at 200, so it cannot take 360. Part 14 already collected that rejection:

422 {"code":"invalid_rules","detail":"period_limit"}

So what would make this comparison stand — trades[] on every row

A trade list attached to each grid row. Like this:

{"fast": 60, "slow": 180, "totalReturn": -5.344…,
 "trades": [{"entry_time": "…", "entry_price": "…", "exit_time": "…",
             "exit_price": "…", "direction": "long", "exit_reason": "…"}, …]}

With that, you can put the two optima's entry levels side by side at the same instant. It is not an exotic ask — part 15's Moss actually emits trades[] in its own backtest output (21-moss-own-backtest-30d.json). QuantDinger's /backtest result does not. That one field decides whether a counterfactual comparison is possible with a given tool at all.

Backtest center — six grid runs stacked in the UI

Look at Recent backtest runs on the right. One row is one grid cell, and what you can see is a single return figure. Click anywhere in that list and you still will not get "this setting entered at what price, at what minute." The wall this article hit is on this screen.

2. Why they have to go out at the same instant

The first use of trial orders is in part 2 — scoring one judgment from a tool all the way to the end. This article is the second use.

Fire arms at the same instant on the same quote with one field changed, and the difference can only come from that field. That is the whole idea. Run them minutes apart and the quote moves, and you can never separate "the setting did it" from "the market did it."

This works because the balance is never touched. If the account moved, you would have no capital for the second arm, and even if you did, the first arm would tie up capital and change the second arm's conditions. The act of measuring would damage what you are measuring. The trial ledger sits outside all of that, so you can fire as many arms as you like.

How many actually go through

I fired six variants at once on one symbol and one quote.

The only thing that sets a ceiling is the monthly quota. And that quota is plain division.

Retries are free, so mistakes do not drain the budget. On pro's 1,000 a month, a two-arm comparison runs 500 times; three arms, 333; four arms, 250.

3. The recipe — it runs as-is

The whole thing is in 11-trial-order-quantdinger-grid-ab.sh. It goes around in four steps.

① Plant — two arms, one field apart. Both arms carry the same client_tracking_id, and metadata.arm tells them apart.

BODY=$(jq -n --arg px "$PX" --arg sl "$STOP_PX" --arg a "$ARM" \
  '{venue:"binance-spot", symbol:"BTCUSDT", side:"long",
    type:"limit", limit_price:$px,                    # ← the only field that differs
    client_tracking_id:"qd-grid-ab-1",                # ← identical on both arms
    exit_plan:{stop_loss:$sl, close_after_seconds:45},
    metadata:{arm:$a, changed_field:"limit_price"}}')  # ← base / variant

Both arms share one stop. The protective 5% stop is the same in both settings, but the entry prices differ, so the stop levels would split. To leave exactly one field free I pinned both to 85900 × 0.95 = 81605.00 off the phase quote.

Intent belongs to the tool. Grid rank, PF and trade counts never go to the server. The server holds execution records; the only thing joining the two is the tracking ID.

client_order_id,tracking,arm,grid,profit_factor,total_trades,cost_basis
qd-grid-ab-1-60-180,qd-grid-ab-1,base,60/180,0.5123854981235674,68,assumed
qd-grid-ab-1-120-360,qd-grid-ab-1,variant,120/360,0.678789397136766,33,measured

② Collect — ?tracking=. An exact-match filter on the tracking ID, scoped to your own account.

curl -sS "$BASE/v1/forward-tests?tracking=$RUN_ID&limit=200"

③ The whole CSV in one call. The account-wide orders endpoint takes the same filter.

curl -sS "$BASE/v1/forward-tests/orders?tracking=$RUN_ID&format=csv"

Collection used to cost "one list call plus one export per test." In a month where you used the full quota, that is 1,001 calls. The columns are identical to a single-test export, so concatenating two CSVs does not break.

④ Join — do not use awk. The metadata, exit_plan and tags columns contain quoted commas. Split on -F, and the columns shift — silently, with no error, which is worse. Join with an inline python3 and the csv module.

Three things pinned into the recipe as comments

Tick rounding. Our recorded entry fills are 85979.70711050 and 85795.14865650. Put either straight into limit_price and you get:

422 invalid_limit_price "px 85979.70711050 is not a multiple of tick 0.01"

The server does not quietly round for you. A condition price has to sit on the same grid as a real order, and a server silently rewriting a price you supplied is more dangerous than a rejection. So the caller truncates — and for a long limit, downward is the conservative direction, so it truncates rather than rounds up: 85979.70 / 85795.14.

Do not put floats in metadata. Determinism blocks them. Carrying probabilities there is a common thing to want, so the guidance comes with the rejection:

422 invalid_metadata "metadata 에 float 를 쓸 수 없습니다 (결정론) — 숫자는 문자열로 보내세요 예: {"p_up": "0.55"}"

float_paths tells you which key. Send it as a string and it passes.

Auto-exits inherit the tracking ID. Realized P&L attaches to the exit row. That row used to carry an empty tracking ID, so collecting with ?tracking= brought back the entry only, with the P&L missing. The exit now inherits the entry's tracking ID and tags.

4. What came back

Environment note. What follows came off an isolated temporary server with synthetic quotes. This is not production, and the P&L here is not a market fact — it is arithmetic on fixed quotes I supplied. The only thing being measured is whether the comparison design stands.

Both arms went out onto the phase quote 85900.00 / 85900.01 at 13:26:45.882 and .889 — seven milliseconds apart.

They split. On whether they filled at all

On the same quote (ask 85900.01), 85979.70 filled immediately and 85795.14 sat resting on the book.

And here is the most valuable thing in this article.

An unfilled arm never starts its close_after_seconds clock.

The relative deadline is anchored to the publication time of the first entry fill, so with no fill the clock never runs. The arm stays put until max_duration_hours. Inside the 45-second observation window the variant produced nothing at all — no P&L, no fees, no close reason, no elapsed time.

An arm that never fills drops out of the comparison. That is not a bug on our side, it is the contract (inventing fills that never happened would be far worse). But if you design a comparison without knowing this, you will never work out why one row of your table is empty. If you are going to run limit arms, check first that the price band will fill all of them. Otherwise send them all as market orders and vary a different field.

Our detail view — two arms of one comparison, stacked under the same tracking ID

Compare the title lines of the two cards. Only the memo differs — "base, stop −0.05%" versus "variant, stop −0.20%" — while 추적 ID: tool-a-run-42 and the ab-compare tag are identical on both. That is what a one-field comparison looks like on screen. (Both arms in this shot did fill. When one does not, as with the variant in the table above, its entry, exit and net P&L cells are blank outright.)

The evidence that the quote was the same

The reference price at acceptance was 85900.005 for both arms (qty_basis.reference_px). Other comparisons produced stronger evidence — in a three-arm run the entry fills' published_at were all 13:25:57.507, identical to the millisecond, and a four-arm run landed inside 13:29:40.456–.457. One published quote filled every one of them.

This installment cannot use that evidence. One arm never filled, so there is no published_at to pair.

The trial-order screen — the arms of one experiment stand next to each other

8 / 1000 at the top is this month's usage — one arm is one unit. Below it the cards stack up, each carrying the ab-compare tag and the same tracking ID. Pulling test_id from here and downloading the CSV is steps ② and ③ of the recipe.

The quota behaved

Two arms = 2 uses. The account ledger snapshot was identical before and after — not a single row moved across all eight account and trade tables.

5. The limits of this combination

① The grid-parameter comparison did not stand. That is the conclusion. What I actually compared is "the entry prices those two settings produced on our side," not the grid itself. How 60/180 and 120/360 diverge on a live feed is not answered by this method. The tool does not emit per-trade levels, and our rule grammar caps periods at 200.

② An unfilled arm leaves no record of the quote it saw. The acceptance response carries no separate bid/ask; reference_px is a single midpoint. So you cannot read anywhere "this arm saw which quote and therefore did not fill." That is a gap we should open.

③ There is no quote snapshot identifier. A BBO has no id, so you cannot assert from the responses alone that two fills came from one and the same quote. Matching published_at down to the millisecond is the best available, and that is circumstantial, not identification.

④ Synthetic quotes. The P&L above is not a market fact. What was confirmed is the wiring and the contract, not exchange performance or real cost.

⑤ One correction on the collection path. When this run happened, GET /v1/forward-tests?q=<tracking id> returned 0 results — at the time q looked only at test_id, symbol, memo and tags, not client_tracking_id. What actually recovered the experiment was the tag (?tag=a2-04-04 → exactly 3 results).

That field opened immediately after this run — an exact-match ?tracking= filter plus q extended to the tracking ID. The recipe in section 3 uses that new contract. But I did not run that code. The "0 results" above is a fact about the state before it opened, and this article makes no measured claim about the state after. When you first run the recipe, check for yourself how many results ?tracking= returns.

6. When to use this and when not to

Use it when

• Your tool emits absolute price levels. If an entry, target or stop can serve as the one varied field, the comparison stands.
• The field you want to vary actually exists in the trial-order body — limit_price, exit_plan.take_profit, exit_plan.stop_loss, side, venue.
• Both settings sit in a price band that will fill them both. Otherwise go market and vary the exit conditions instead.

Do not use it when

• The tool's output is a parameter, a ratio or a period. There is no such field in a trial order. The moment you translate it into an absolute level, it stops being the tool's answer and becomes your translation.
• The grid rows carry no trade list. That is this article — you end up unable to say what you compared.
• You want to compare absolute P&L size. The server sets the quantity, and it is barely above the minimum notional.

And do not force a success story. My two arms really did split, but they split on the limit price, not on the grid. Writing that up as "the grid optima diverged" would be a lie. "The comparison does not stand" is a result too, and knowing what would make it stand is worth more to a reader. That one field is trades[] on every grid row.

Evidence for this article: measurement log · raw data · recipe. Every number above came from that record, and so did the failed comparison and the limits in section 5.