Scoring AI Trading Analyst's wait-for-pullback — what you measure when the entry sits below the close

This tool's real refusal is not AVOID, it is the entry price. Here is the procedure for turning a pullback-wait plan into a counterfactual — including the part

Done when you can take a "bullish, but not here" plan, turn it into a counterfactual, and ask whether the gap the tool was waiting for was worth waiting for. This cycle the answer came back unresolved, and why is in the article.

1. The blank this tool kept leaving

1.1 A correction first — there is no real AVOID here

AI Trading Analyst (github.com/zubair-trabzada/ai-trading-claude · MIT) is a bundle of Claude Code skills. Sixteen markdown instruction files and one PDF script, and it needs no API keys at all.

Its output format takes a direction of LONG / SHORT / AVOID, so I went looking for an AVOID record to build a counterfactual from. This tool never actually issued one. AVOID appears in exactly two places in the raw material.

1. A format placeholder — DIRECTION: <LONG / SHORT / AVOID> in the instruction file. That is the shape of the blank, not a judgment.
2. An artefact of a schema bug — the record contains AAPL · Trade Score 0/100 · Grade F · Signal AVOID, which is what happens when the PDF script silently substitutes defaults for keys it cannot find (data.get("ticker","AAPL"), data.get("overall_score", 0)). Neither the ticker nor the score is the tool's judgment; both are defaults.

The PDF page one you get from the payload the instructions ask for

I fed it an NVDA analysis and the cover says AAPL, the score is 0, and the signal is AVOID. Nothing I passed in was read at all — everything you need in order not to read this AVOID as a judgment is on this one page. The JSON keys the instructions ask you to build are not the keys the script reads.

So I did not build a counterfactual out of AVOID. I used the decision not to order that this tool actually did make.

1.2 This tool's refusal lives in the entry price, not the direction

I asked it about the same symbol (NVDA) three times in independent contexts. All three came back LONG. And yet —

All three entries sit below the last close. r2 spells it out (the tool's output):

"Entry is not live. Entry Trigger 1 (RSI below 65) is not met at RSI 80.3, and price at $222.27 is above the $217.65 limit. This is a wait-for-pullback plan, not a buy-now plan."

A wait-for-pullback order is a decision not to order right now. Read the direction alone and that refusal is invisible.

The ten lines /trade thesis NVDA produces

This is the tool at work. The top two lines are the as-of time and the reference price the document settled on; the green block is the last section. Look at ENTRY: $222.60 (limit order) sitting below the Current Price above it — that is "buy on the way down", not "buy now". That one line is the subject of this article.

Two numbers not to confuse. The current price this tool read in r1 was $227.38 (the 2026-09-21 close, via web search); the last one-minute bar close that weltiq's bar API measured that day was 227.30. The "waiting gap" below uses 227.30.

2. Why this has been empty until now

Scorecards have only ever existed for things you bought. A limit order that never fills leaves no record at all — not even the fact that you waited.

But a pullback wait is not a blank. It is a testable claim: waiting for 222.60 beats buying at 227.30 right now.

Here is why it stayed empty. To test that claim, you have to buy now — and the moment you do, your account moves. Capital gets tied up, the loss limit gets eaten into, the performance rollup gets contaminated. The act of measuring destroys the thing being measured. So nobody measured it.

A trial order steps around that. It carries the judgment through to the close in a separate ledger that touches none of your balance, P&L, limits, or performance totals.

The decision not to buy gets a scorecard too. Without touching your account.

This is not a claim that nobody else could do it. A backtest could approximate it in principle. But a backtest is a replay of the past, so it cannot catch the moment and the quote the call was actually made at — and a live account brings back the destruction above.

3. The recipe — the thing that actually runs

3.1 First, get a judgment out of the tool

There is no dashboard. No web UI, no TUI, no address to open. Claude Code's terminal output and a markdown file in your working directory are the whole of it.

git clone https://github.com/zubair-trabzada/ai-trading-claude.git
cd ai-trading-claude && ./install.sh

Install: 0.129 seconds. Sixteen skills, five agents and one script land in your home directory.

The finished install

Everything one ./install.sh does. If the bottom three lines — 16 skills, 5 agents, 1 script, and where each went — appear, the install is done. The amber line in the middle is a reportlab warning you can ignore unless you want PDFs.

Invocation has to be pinned to the slash command.

/trade thesis NVDA

Natural language does not reach it. In my own testing, "Build me an investment thesis on NVDA with entry, stop loss and price targets" invoked a different plugin entirely, 3 times out of 3, while this tool got 0. The mission-style sentence ("Trade NVDA and make me a profit") matched nothing at all, 0 out of 3. /trade thesis NVDA pins it immediately, 1 for 1.

Four to five minutes later TRADE-THESIS-NVDA.md appears, and the only section you need is the last one, ## 10. Action Plan Summary.

3.2 Place the version that does not wait

The full recipe is in 05-trial-order-ai-trading-analyst-pullback.sh. Two cases go out.

# Long — the opposite of "wait for 222.60". Buy at the close, now
curl -sS -X POST "$WELTIQ_BASE_URL/v1/trial-buys" \
  -H "Authorization: Bearer $WELTIQ_API_KEY" -H 'Content-Type: application/json' \
  -d '{"venue": "us-equity", "symbol": "NVDA",
       "client_order_id": "ai-trading-analyst-1-long", "side": "long",
       "exit_plan": {"close_after_seconds": 600,
                     "stop_loss": "211.00", "take_profit": "235.75"},
       "client_tracking_id": "ai-trading-analyst-1",
       "tags": ["counterfactual", "part-27"]}'

# Short — buy the very dip the tool was waiting for. Roles swapped, numbers verbatim:
#   take_profit 222.60 = the tool's entry · stop_loss 235.75 = the tool's T1

Three things are decided before you send anything.

① Why long: the opposite of "wait" is "buy now". The stop and target are r1's, verbatim.

② Why short: all three runs issued an entry below the close, which is an explicit expectation of a near-term decline. But this tool's medium-term direction is LONG, so this short measures its timing call, not its direction call. That distinction is recorded on the case.

③ What the 600-second deadline costs you: this tool's horizon is 3–6 months and the contract cannot hold that. In an earlier transfer check, three months (7,776,000 seconds) came back 422 invalid_close_after_seconds and 2160 hours came back 422 invalid_max_duration_hours (1..168 allowed). Truncate to seven days and it is no longer this tool's judgment. So this does not measure the horizon; it measures the waiting gap of "not right now".

A note on ticks. 211.00, 235.75 and 222.60 are all multiples of the US equity tick (0.01), so they pass as they are. That is because this tool issues whole cents, not because I tidied anything. Tools that issue floats are different — elsewhere in the same cycle stop 82034.9994 came back 422, and only went through as 201 once rounded to 82035.00. The server does not round for you — a condition price has to sit on the same grid a real order does, and silently rewriting a user's price would be worse.

3.3 Pull it back — two calls

curl -sS "$WELTIQ_BASE_URL/v1/forward-tests?tracking=ai-trading-analyst-1&limit=200" \
  -H "Authorization: Bearer $WELTIQ_API_KEY"

curl -sS "$WELTIQ_BASE_URL/v1/forward-tests/orders?tracking=ai-trading-analyst-1&format=csv" \
  -H "Authorization: Bearer $WELTIQ_API_KEY"

The second line is the account-wide endpoint added on 2026-09-28. Server-generated close orders inherit the tracking ID and the tags.

When you join, do not use awk. The metadata and exit_plan columns contain quoted commas, and cutting on the field separator silently shifts your columns. The recipe joins with an inline python3 csv module and says why in a comment. And numbers in metadata go in as strings — floats return 500.

Environment notice: the numbers below came off an isolated server with synthetic quotes. This is not production, and the P&L is not a market fact — what is being measured is the wiring and the scoring frame, not whether the tool was right.

4. What came back

Both cases were accepted with 201. Then placement returned rejected:market_closed.

The session measurement at run time:

venue=us-equity  local_time=2026-09-28T00:27-04:00  is_open=False  state=closed
next_open=2026-09-28T09:30-04:00   (= 22:30 KST)

So "was the wait right" is unresolved for this part. I did not fake the clock — a fill obtained that way is not a fill that passed the real rules. To measure it again, run the same recipe between 22:30 and 05:00 KST.

Accepted, then bounced at placement

This is our screen. Look at rejected:market_closed in the close_reason column — acceptance (201) and fill are different stages, and something that was only accepted has no P&L. The quota comes back too, since there was never a fill. This screen is what separates "blank" from "failed".**

4.1 One frame did get built here

Level-issuing tools say "not right now" with the entry price, not only with the direction. The distance between that entry and the current price is the tool's waiting gap, and the counterfactual asks whether that gap was worth opening.

This tool's waiting gap is −2.07% (227.30 → 222.60). And the round-trip cost on US equities is zero commission plus a 0.00206% sales fee — two orders of magnitude smaller.

That is the substance of this part. Since cost is far smaller than the waiting gap, this tool's waiting is not justified by cost. It is justified only by whether the pullback actually arrived.

Crypto was the exact opposite. The binance-spot round-trip cost measured in the same cycle is 0.2223~0.2224%, which reads as "move 0.222% within ten minutes and that wait was wrong." What sitting out is worth is decided by the exchange, not by the judgment.

4.2 The surprise — confidence could not have made a difference

In the same cycle I placed four waits from another tool the same way. Confidence 0.88, 0.90, 0.89 and 0.20 produced net P&Ls identical to the last decimal.

They were placed at the same moment against the same quote. In a frozen market, P&L is decided by the round-trip cost, not by confidence.

To measure any correlation between confidence and outcome, the counterfactual has to be placed at the moment and the price the call was made. Batch it up and replay it later, and the confidence disappears while the cost remains.

The reason this part ended unresolved has the same root — it should have gone out the instant the call was made; batched up later, it hit a closed market.

5. What this combination cannot tell you

① One sample says nothing. This part's sample is zero — neither case filled. Even one would not have been enough. A wait that ends in a loss is not a wait that was right; you need a distribution.

② The counterfactual measures one side only. "What if I had bought" is measurable. "What did I save by not buying" is still a judgment call.

③ We do not say the tool was right or wrong. We measure and record. This cycle did not even manage to measure, and saying so plainly is what this section is for.

④ This cycle is unresolved. The market was closed, so there were no fills. The −2.07% gap and the cost comparison in 4.1 are arithmetic over recorded values, not something a counterfactual trade confirmed. Do not blur that line.

⑤ This tool's market data is web search. Entries and stops are computed on top of sources where the same day's price reads $222.27 in one place and $270.34 in another. The $222.27 that r2 and r3 used is five dollars below the actual close. A counterfactual inherits that contamination whole.

⑥ The horizon does not fit in the window. Measuring a 3–6 month judgment in a ten-minute window measures "right now", not that horizon. Truncating to seven days is the same problem — rounding adjusts a value, truncation changes the judgment.

⑦ The exchange decides what sitting out is worth.

⑧ Synthetic quotes. What this procedure confirmed is the wiring and the contract (acceptance codes, rejection reasons, CSV columns), not exchange performance or real fee levels.

6. When to use this and when not to

Use it for level-issuing tools. That is this part's lesson. Read the direction alone and this tool's refusal is invisible. Look at the distance between the entry and the current price — that is how it says "not right now".

Use it to compare the waiting gap against round-trip cost. The arithmetic is trivial (gap ÷ cost). The value is in which way it tips, and by how much. Two orders of magnitude means waiting is not explained by cost logic.

Do not run it while the market is closed. That is exactly why this cycle is unresolved. Place US equity counterfactuals between 22:30 and 05:00 KST. Acceptance works any time, but acceptance and fill are different stages.

Do not use it to score the tool's horizon. The 168-hour cap bites. Write down, every time, whether you measured "right now" or the horizon.

Do not go hunting for AVOID records. As section 1.1 shows, those values are either format placeholders or artefacts of a schema bug. This tool's real refusal lives in the entry price. Separate judgment from default before you read anything.

Evidence for this article: the counterfactual measurement record · per-case request and response text · this part's recipe · the tool's own run log. Every number above came out of those files, including the fact that it ended unresolved.