Why your trading tool only answers half the question — measuring a judgment with an order that never touches your balance

A procedure for registering a tool's judgment together with its exit condition and letting it run to completion on a separate ledger. I pushed all thirty-one to

You've finished this article successfully if you can take a single judgment from your own tool, register it with the condition that closes it, and get a scorecard back without your account moving at all.

1. Why this procedure is needed

Run thirty-odd open-source investment tools and an odd pattern shows up.

These tools produce judgments. They can't measure whether those judgments were right.

That isn't laziness. To measure, you need closed trades; to close a trade you need to place an order; to place an order you need an account. Most tools stop there.

Three places where it actually broke

Straight from the runs in this series.

① Balance. One tool told me to buy 0.116 BTC. At the price at that moment, 86,026.32, that blows well past the 10,000 in the account. The result was insufficient_balance — that round vanished for a reason that has nothing to do with the quality of the judgment. Raising the capital doesn't fix it either: every tool sizes on a different basis, so tuning capital for one bounces the next.

② Shelf life. Plenty of tools give a judgment with no "valid until". I once carried levels straight over from six months earlier: 30 days, 3,889 trades, ending at −225.29. The tool wasn't wrong — I kept feeding it a judgment past its expiry date.

③ Quantities that can't be compared. One tool says 43 shares, another 0.1 BTC, another 6.44 shares. That 0.1 BTC was 8.6% of that tool's own starting capital — a size that makes "risk 2% of capital" unusable from the start. Line those three up and you're comparing sizing taste, not judgment.

④ And there's no "until when". That's ② turned around, but it deserves its own line: as you'll see, the deadline is the one field a trial buy insists on, and across thirty-one tools not one produced it in a usable form. It's the problem I spent the longest on in this article.

And the most wasteful place of all — a requirement the tool computed for itself

One review tool writes the required sample size into its own report: 384 trades to pin win rate within ±5%, 1,705 to pin expectancy.

The signals it actually had: 43.

Another confidence-scoring tool states it needs at least 30 observations to calibrate, and at n=1 refused itself.

Nobody was giving these tools the thing they had written down and asked for.

2. The whole picture

A trial buy registers one trade together with the condition that closes it and makes the server responsible for its entire lifetime. The name covers both long and short, and fills land only on a trial ledger, separate from simulated and live accounts.

Three lines carry it.

1. It doesn't touch your balance. Cash, holdings, P&L, loss limits and performance aggregates don't move. ① above disappears.
2. An exit condition is mandatory. You must supply at least one deadline (close_at or close_after_seconds). ② above is blocked at the contract level.
3. The server picks the quantity. Send qty and you get rejected. ③ disappears — at a price, which section 6 covers.

The procedure runs in four boxes.

The third box is the point of this article. That's where the material the tools were asking for comes from.

3. What you need

• A weltiq pro account. Trial buys are pro-only, 1,000 per month.
• The run-forward-tests scope on your API key. operate-sim will not substitute.
• I assume your tool is installed. This article installs nothing new — every tool-side number is from runs I did in earlier parts.

WELTIQ_BASE_URL=https://weltiq.ai
curl -sS "$WELTIQ_BASE_URL/v1/forward-tests/usage" -H "Authorization: Bearer $WELTIQ_API_KEY"

On my isolated server: limit 1000 · used 0 · remaining 1000 · plan pro · can_enter true, resetting at 2026-10-01T00:00:00Z. The quota is a UTC calendar month with no carry-over.

That's 33 a day. Enough to run thirty-one tools once a day. Keep the number in mind and section 4's shape makes sense.

4. Let's go around once

4.1 One long — handing sizing to the server

curl -sS -X POST "$WELTIQ_BASE_URL/v1/trial-buys" \
  -H "Authorization: Bearer $WELTIQ_API_KEY" -H 'Content-Type: application/json' \
  -d '{
    "venue": "binance-spot", "symbol": "BTCUSDT",
    "client_order_id": "my-tool-run-1",
    "side": "long",
    "exit_plan": {"close_after_seconds": 60, "take_profit": "51000", "stop_loss": "49000"},
    "max_duration_hours": 24,
    "memo": "one more bar of confirmation after the volume condition",
    "client_tracking_id": "my-experiment-17",
    "tags": ["confirm-next-bar"],
    "metadata": {"tool": "29B-review.py", "external_run_id": "local-run-1"}
  }'

You get 201, with fields filled in that I never sent.

"qty": "0.00013",
"qty_basis": {
  "rule": "smallest_lot_multiple_meeting_min_notional_within_price_headroom",
  "lot": "0.00001", "min_notional": "5", "reference_px": "50000.5", "price_headroom_pct": 20
},
"auto_close": {"managed": true, "state": "waiting_entry",
               "effective_close_at": "…", "effective_close_basis": "max_duration",
               "relative_clock_started": false},
"affects_account": false,
"usage": {"used": 1, "remaining": 999, "counted": 1}

Three things worth reading.

qty_basis publishes the rule. Why 6.5 USDT worth when the minimum is 5? Because it picks the smallest unit that still clears the minimum after a 20% drop from the reference price. I checked by hand: 0.00012 × 40000.4 = 4.80 fails the 5 minimum, 0.00013 × 40000.4 = 5.20 passes. It isn't quietly rounding up — it hands back the rule, the headroom and the reference price.

effective_close_basis tells you what's currently winning. Before the first fill the close_after_seconds clock hasn't started (relative_clock_started: false), so what sets the end right now is max_duration.

affects_account: false. That one line is the answer to ① in section 1.

Sixty seconds later:

exit_trigger = close_reason = "close_after"
relative_from     22:22:32.285872   ← publication time of the first fill
relative_close_at 22:23:32.285872   ← exactly +60s
exit_trigger_px   null

Note that exit_trigger_px is null. A trade closed by a deadline has no "trigger quote", and the field isn't padded to pretend otherwise. In systems where this arrives as 0, statistics break quietly later.

4.2 A short — and the condition price is not the fill price

  -d '{"venue": "binance-usdm", "symbol": "BTCUSDT",
       "client_order_id": "my-tool-run-2", "side": "short",
       "exit_plan": {"close_after_seconds": 3600, "take_profit": "49000", "stop_loss": "51000"}}'

For a short, the target sits below the stop. Entry filled at 0.002 @ 49994.49995 — a short is a sell, so it slips toward the bid.

What happened when the target came in is the most important moment in this article.

1.01 seconds apart, 5.39 wide. The server judges that the condition was met and submits the exit order; the fill happens afterwards, on the first quote published after that, under the ordinary fill rules. Slippage and fees apply.

That single row is why you shouldn't trust a backtest that fills you at the condition price. Realized P&L was +2.07731891, of which 0.09890089 was fees.

And this isn't only true of my synthetic quotes

A separate verification run the same day used live Binance quotes. One of its auto-exits went like this:

2.63 seconds from trigger to fill, with one retry along the way. The 1.01-second gap I measured on synthetic quotes shows the same property on live ones — the condition only starts the exit; the fill happens on the first quote published after it.

That run did four trades and all four lost money: spot long (−0.0157), USD-M short (−0.0990), slot long (−0.0144), post-deploy long (−0.0160), all USDT. Price didn't move enough in 30 seconds to clear the fees. That's normal. What a 30-second trial is for is the record, not the return.

4.3 Learning the edges by getting rejected

Deliberately collecting rejections is the fastest way to understand this thing. The point is that the codes differ.

The list that rides along with the first row is useful.

"short_venues": ["binance-usdm", "bybit-linear", "cme", "us-equity"]

Don't memorize this list; read it from the response. On the day I wrote this I watched us-equity enter that list. The decision comes from one place (the market profile), so when a market's settings change, the list changes with it.

The third row is worth a second look. Direction passed and the session blocked it — which means "shorts aren't supported" and "the market is closed right now" are distinguishable by code. Not collapsing both into one 400 Bad Request matters quite a lot if you're automating.

4.4 Sending the same thing twice

Run an agent and the same judgment goes out twice — on restart, on reprocessing the same bar, on a retry after a timeout.

The second row is right to be strict. A different body is a different judgment, and letting it overwrite under the same name makes the record untrustworthy later.

From a slot (a rule running inside our engine) you don't even build the id: it's derived as sha256(deployment | slot version | bar time | rule id | action), so replaying the same bar produces one order.

4.5 Export — where outside tools read

This is the destination of the whole procedure.

curl -sS "$WELTIQ_BASE_URL/v1/forward-tests/$TEST_ID/export.csv" \
  -H "Authorization: Bearer $WELTIQ_API_KEY"

The CSV header has 30 columns, and the six I wanted are at the end.

… quote_source,published_at,side,exit_plan,exit_trigger,exit_trigger_at,exit_trigger_px,close_reason

Without those six, a tool reading the spreadsheet doesn't know why a trade closed. A record with returns but no reasons can't be reviewed. close_reason takes eight values: take_profit · stop_loss · close_at · close_after · max_duration · manual_exit · user_stop · window_expired.

And the linking fields I sent come back intact.

external_run_id is the coupling for this procedure. Put your tool's own run number in there and its review script can later join "how did my run #3 actually end" against its own records.

There's a JSON export too, and its comparison is identical to the one from the detail API (I compared the dictionaries for equality). If two paths gave different numbers you couldn't trust either.

The server's own auto-exit order appears in the CSV as well — client_order_id is exit-plan:<round_id>:1 and memo reads "automatic exit per the registered exit plan". Your orders and the server's are distinguishable in the record.

4.6 Checking for yourself that the account didn't move

Rather than trust affects_account: false, I counted. Before and after, I compared nine account-side tables in full: accounts, portfolios, sleeves, ledger, direction leases, reduce reservations, order intents, fills, state transitions.

Identical. Cash still 100,000 USDT, holdings and P&L zero, intents and fills zero rows.

While three trial buys were accepted, filled and closed, not one row was written on the main account side.

5. What was different after running it

5.1 Tools can finally be lined up side by side

Here's what the server picked, per market.

Stocks and futures get one share or one contract; crypto gets the smallest unit clearing the minimum order value. That's the answer to ③ in section 1. Instead of comparing 43 shares against 0.1 BTC against 6.44 shares, you compare judgment on a fixed quantity.

5.2 What each requirement gets filled with

Re-reading the tools from this series against that standard:

One more note on the second row. Having the option to close on time alone matters more for scoring than it looks. Attach a target and a stop and you bake "winners close first" into the sample. Close purely on a deadline and winners and losers close by the same rule, so a calibration sample stays clean. That comes from the contract, not from any trick of mine.

5.3 What surprised me — the quota was 2, not 3

Three entries were accepted and I watched used climb to 3. When it was over it read 2.

The quota ledger said why, plainly:

+1  entry_accepted                     (#1)
+1  entry_accepted                     (#2)
+1  entry_accepted                     (#3)
-1  placement_rejected:market_closed    (refund for #3)

An entry that was accepted but then rejected when the server placed it with the broker gets one back. There was never a fill. The contract document I'd read before writing this described the refund mechanism but not what triggers it — I reported that, and it's been filled in.

For the record, an accepted entry that is cancelled unfilled or expires still costs one. That one did reach the market, so it's different.

5.4 So I pushed all thirty-one through it

Everything above was my own call. But "the tool's judgment fits this field" is a claim you only settle by actually sending it, so I took the recorded output of all thirty-one tools from this series and pushed it into trial buys as-is. I did not reinstall anything — what's under test is the transfer, and its input already sits in each part's run log. 127 calls.

One rule: invent nothing. Where a tool's record had no deadline I did not make one up — I sent it without a deadline first, took the rejection, and kept that as the result. Same for floating-point levels: send as-is, take the rejection, then round.

Exactly one tool out of thirty-one went in with nothing changed.

That one supplied both levels and a 48-hour deadline itself. What set it apart wasn't cleverness — it was writing down "until when" as a number.

Three rules fell out

① No tool produces a usable deadline. Eight recorded one, and none of the eight works — the four absolute timestamps are all in the past (they're records, so of course), and the four durations exceed the 7-day cap (3 months, 63 days, 3–6 months, and one more).

② The 7-day cap truncates the tool's judgment. When a tool says "3–6 months" and I put 7 days in that slot, what I measure is not that tool's judgment. This is a different order of problem from rounding: rounding adjusts a value; truncation changes a decision. It's now item ⑧ in section 6.

③ Which tools hit the tick is predictable. Every tool emitting a percentage or a fractional fill price hit it (seven); every tool emitting whole cents did not (four). What I wrote in 4.5 wasn't a coincidence, it was a pattern.

And it turned up a defect of ours

A trial buy was judging direction from the latest market profile and fill rules from version 1. Because of that, the warning that "the borrow fee on US equity shorts is charged as zero" (section 6 ④) was not reaching the response. Fixed, with a test that checks the warning actually arrives. Reading didn't surface it; sending did.

6. What still doesn't work

This is the most important section here. It's where you find out how far to trust the numbers above.

① I verified with synthetic quotes. I ran against an isolated temporary database with fixed quotes I inserted myself (BTCUSDT 50000/50001, AAPL 230.00/230.02), and I lowered the quote to 48900/48901 once during the run to make the short's target fire. That timestamp is in the evidence file.

So what's confirmed is the wiring and the contract, not exchange performance or real fee levels. The 5.39 of slippage above means "our fill model behaves this way", not "Binance slips this much".

Live-quote records do exist, though — the four trades after section 4.2 ran on real Binance quotes. Those are still on the trial ledger, so they aren't costs an exchange charged a real account. The fees and the fill-price gap remain model output.

② The feature is in production, but the numbers here came from an isolated server. Trial buys themselves are deployed and running. Every call above, though, went to an isolated server of mine with synthetic quotes — that does not mean production produces the same numbers. The contract (field names, rejection codes, CSV columns) is the same code and therefore identical; fill prices, slippage and fees are not.

③ Shorts opened only halfway. us-equity is newly open; binance-usdm, bybit-linear and cme already were. Spot (binance-spot, bybit-spot) and Korean equities still aren't. Spot has no such thing as naked selling, and retail short selling in Korea is a separate matter.

This bites in practice. In an earlier part I met a tool that essentially only produced shorts (0 of 9 buys, 6 sells) — it targets spot, so it gains nothing from shorts opening. It has to move to futures to come alive.

④ The borrow fee on US equity shorts is charged as zero. In reality it runs from about 0.25% a year to tens of percent, depending on the symbol and the day. At zero, the longer you hold a short, the more favourable our record is versus the market.

The market profile says so about itself, calling it the one non-conservative bias in that profile. Every other bias overstates cost; this one alone runs the other way. And a trial buy can hold a short for up to 168 hours, which makes this the path most exposed to it.

So don't read "shorts are open" as "short P&L is measurable". The timing of a direction flip is measurable. The P&L of a long-held short isn't, yet.

⑤ US futures (CME) are blocked. Quotes and charts work; the order purpose isn't open yet. Combinations aimed at futures stop here.

⑥ Floating-point levels don't go in as-is. A value like 756.2436 from some tool isn't a multiple of the US equity tick (0.01), so it's rejected. It is not quietly rounded — a condition price has to sit on the same grid as a real order, and a server silently altering a price you gave it is the more dangerous behaviour. Round to the tick on your side. Every tool that emits floating-point levels meets this.

⑦ Allocation disappears. Because the server sizes the order, "10% of capital in this name" folds into one share. Tools using scale-free metrics (R multiples, expectancy %, Brier) lose nothing; tools whose output is the allocation lose half of themselves on this path.

⑧ The lifetime cap is 7 days — and this weighs more than ⑥. max_duration_hours runs 1–168. In the run, four tools hit it (3 months, 63 days, 3–6 months, one more). Truncating to 7 days is accepted, but you must not read the truncated round's scorecard as that tool's scorecard. Right now nothing in the record marks it as truncated — I've filed that the truncation should be recorded when it happens.

⑨ The small size creates one specific problem: dust remainders. The reference quantity in 5.1 only just clears the minimum order value. So a partial fill can leave a remainder below that minimum.

The production verification has a case. An entry filled at 0.00013 @ 50006.50005, leaving 0.00013 outstanding; the exit order went out six times and filled once. Rejection code liquidation_rejected:min_notional, five retries. The screen says it plainly: "this will not resolve even when quotes return. The server will not fabricate a fill or round the quantity up to fit."

That is the right design — fabricating a fill would be far worse. But that round stays in "awaiting exit" and its P&L never settles — when you score, exclude it or count it separately.

So the conclusion isn't "do everything with trial buys". Keep allocation-driven tools on the existing path and put judgment-accuracy questions on trial buys — the two are parallel, not substitutes.

7. Coming next

Section 5.4 covered the transfer for all thirty-one, so next time goes the other direction — actually feeding an exported record into a tool's own review step: a tool that generates candidates in volume at the front, and a review tool that was asking for sample size at the back. I'll also write down how the 33-a-day budget gets split.

And there's a visible way to narrow ④. The broker already reports, per symbol, whether it's easy to borrow, and that value is already being stored — just not read. Once it is, the zero-borrow bias can be cut down symbol by symbol.

Evidence for this article: the run log · raw request/response per case · the reproduction script. Every number above comes from that record — including the surprise in 5.3 and everything still blocked in section 6.