crypto-trading-arena — put four LLMs in front of the same candle and watch who buys what

Standing up Kafka by hand to sit four LLMs in front of the same candle, throwing the same mission at them three times to measure how much the verdict moves, and

Done when: you have a screen up showing several LLMs looking at the same candle and reaching different trading decisions.

1. What this is

Market data arrives in one place and is pushed out to several agents identically; each agent then reasons with its own LLM and calls buy, sell, calculator and check balance tools. The wallet is held by a single tool-side process, so who bought what, when and at what price all lands in one table.

The heart of it is the fairness machinery. There is one place that builds the candles, and the sentence that place builds goes to every agent in the same characters. No agent fetches its own data. That is what lets you say "the conclusions differ because the models differ".

Repositories that post their open bugs as a list are rare. Nearly everything you have to watch out for in this article was already written down on that list.

2. Installing — the one path I verified

You need a message broker (Kafka), and there is no way around it. The README tells you to use Docker; on a machine without Docker you can stand Kafka up directly with Homebrew. That is the path I verified.

# 1) broker
brew install kafka                     # includes the JDK, 4 minutes
cat > server.properties <<'EOF'
process.roles=broker,controller
node.id=1
controller.quorum.bootstrap.servers=localhost:9243
listeners=PLAINTEXT://localhost:9242,CONTROLLER://localhost:9243
advertised.listeners=PLAINTEXT://localhost:9242
inter.broker.listener.name=PLAINTEXT
controller.listener.names=CONTROLLER
listener.security.protocol.map=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.dirs=/tmp/arena-kafka
EOF
kafka-storage format -t $(kafka-storage random-uuid) -c server.properties --standalone
kafka-server-start server.properties &

# 2) the tool
git clone https://github.com/ryan-yuuu/crypto-trading-arena && cd crypto-trading-arena
uv sync
cp config.example.json config.json     # models, keys, symbols and fees are edited here
export OPENROUTER_API_KEY=...          # same name that config.json references with ${...}

I changed three things in config.json.

"trading": {
  "exchange": "binance",
  "binance_symbols": ["BTCUSDT", "ETHUSDT", "SOLUSDT"],
  "fees": { "taker_bps": 60 }
}

Then bring it up in four windows (or four background processes).

uv run python -m exchanges.binance             --bootstrap-servers localhost:9242
uv run python -m deploy.tools_and_dashboard    --bootstrap-servers localhost:9242
uv run python -m deploy.response_viewer        --bootstrap-servers localhost:9242
uv run python -m deploy.agent --from-config <name> --strategy default \
                              --bootstrap-servers localhost:9242   # one per model

Where it gets stuck

3. First result — this is what it looks like when it works

Bring up four agents, leave it about 10 minutes, and you get this screen. The four models received the same candle at the same moment and behaved differently.

Balances, positions and fill history for 4 agents. They received the same 11 market updates, yet gemini filled 10 times while haiku and gpt-5-nano never placed a single order.

When this screen comes up, it worked. Here is how to read it.

• The four boxes along the top = per-agent total equity, position count, trade count and fees paid
• The left table = quantity, average price, market value and P&L per symbol
• The right table = fill history. The rightmost column is the number of seconds from the moment the quote went out to the fill (2.6 seconds for the fast model, 23 seconds for the slow one)
• The bottom = the equity curve

What each of them actually thought about that same candle is readable verbatim in a separate window.

deepseek, having analyzed all three symbols, writing that it has to be careful about new entries because of the 60bp fee

4. Getting the most out of it — change one fee line

What separates this from other multi-agent experiments is that it feeds the fee to the model as an input to the decision. Every tick, the sentence that goes to the agents opens with this.

A taker fee of 60 bps (0.60%) is charged on every fill (both buys and sells).
Factor this into your sizing and P&L targets — a round-trip costs 120 bps.

Set taker_bps to 0 in config.json and that same slot becomes Trading is fee-free in this deployment. So I held everything else fixed and changed only that line, then ran two rounds.

4-1. Two live rounds — the same four agents, 11 market updates each

The two models that never placed an order in the 60bp round (haiku and gpt-5-nano) both ordered in the 0bp round.

The same screen after setting fees to zero. All four agents traded, and every "Fees paid" reads $0.00.

4-2. Again, on the same candle — 60 replays

The two rounds sit at different market times. So I took one prompt exactly as the connector actually emitted it, changed only the fee sentence, and replayed it 10 times each against the same tools and the same starting account. The market is now completely fixed.

The verdict does change. But it changes differently per model. Two of them clearly open up when the fee is removed, and gemini-flash-lite moved the other way — it swings about the same whether the fee is there or not. And what happened to that model in the live 60bp round is right there on the first screen: it filled 10 times, paid $1,004 in fees alone, and ended with total equity of $99,065.

So what this tool tells you is not "does this strategy beat the fee" but "is this model even looking at the fee". For picking a model, the second question is the more useful one.

4-3. Two things to know before you start

The default rate does not match the exchange that supplied the data. The config default of 60bp is Coinbase's base tier, and Binance spot, which is where this article's candles came from, is 10bp. Run the default against Binance and you pay 6 times the real cost. If you are going to experiment with fees, fix this first.

Quantities are only accepted to one decimal place. Measured on BTC.

0.05 BTC  rejected   0.09 BTC  rejected   0.15 BTC  rejected
0.1  BTC  accepted   0.2  BTC  accepted

Orders are not blocked outright, but on a $100,000 account a BTC position can only be adjusted in 8.6% steps. "Risk 2% of capital" is simply not available on BTC. deepseek actually computed 100000 * 0.02, arrived at $2,000, then gave up on BTC and went to SOL.

5. Moving it into weltiq — an integration experiment

weltiq setup (same as every part): sign up ({{CTA_URL}}) → My → issue a key under Connections → claude mcp add weltiq https://weltiq.ai/mcp --header "Authorization: Bearer <key>"

Give this tool the mission "trade Bitcoin and make me a profit, and explain why you did what you did" — I typed it in Korean, 비트코인을 거래해서 수익을 내줘, 그리고 왜 그렇게 했는지 설명해줘 — and it returns direction, quantity, target price and invalidation price in one go. Here is what came back from throwing the same mission at it three times.

The direction was the same all three times, and the quantity spread from 0.5 to 1.1 — more than double. The way run 2 arrived at its target is very much this tool's signature — "to clear the 120bp round-trip fee and still keep 1–2%, $85,532 × 1.021 = $87,290".

Carrying run 1's decision into a weltiq slot is five steps.

1. Write it as a rule document. Entry close > 85400, limit exit 85845, protective stop 0.5%, quantity 0.1
2. POST /v1/strategy-rules and the document reads back as a single line — entry: (close > 85400) → enter long E · True → price-condition exit (limit=85845) → exit: protective stop 0.5%
3. POST /v1/ai/thesis to submit why you decided that, in plain language. Without a rationale, the slot is not created
4. POST /v1/containers to create the slot. Stop conditions (max loss, time limit) are mandatory
5. Check the fill, then stop and delete

Here is how that day turned out.

What does not carry over

• Quantity. 0.5 BTC is $43,000. This account's balance was 10,000, so I cut it to 0.1
• Timing. The tool's decision is "buy at market, now", whereas a rule stays valid for as long as its condition is true. The moment you turn "now" into a condition, the meaning changes
• So this slot is not the tool's decision carried over as-is — it is a rule I rewrote myself out of the three numbers the tool produced

Why you must not take the other side's returns at face value

Here is how the two books accounted for the same round trip.

Same decision, same market, and the cost comes out 6 times apart. On top of that, the repository itself lists three open accounting bugs — one of them being "realized P&L subtracts the exit fee but unrealized P&L does not". You cannot add those two numbers together. Use this tool to measure the decision, and measure the performance on the side you carried the decision to. That is the right split.

6. After using it

What was good

• The fairness machinery is real. Candles are built in one place and pushed out in identical characters. The premise of a model-comparison experiment is in the code
• The fee is an input to the decision. Changing one line and running two rounds works exactly as you would hope. Very few tools have that switch
• Open bugs are published as a list and pinned in the tests. You know in advance where not to put your trust
• The screens are good. Balances, fills and the raw reasoning each get their own window, and it even shows fill latency in seconds

What fell short

• You must have a broker. Docker or Homebrew, either way — without a message broker nothing runs at all
• Candles never reach the Coinbase path. In practice there is exactly one choice, Binance
• It cannot wait. On the third mission the tool wrote that it would "wait for a 1.5–2% bounce and then sell", and sold immediately within the same call, losing $1,129 to fees alone. One call is one episode, so there is nowhere for "wait" to be carried out
• It gives no confidence score. If you want a 0–100 on each decision, you have to edit the prompt
• The one-decimal quantity limit (§4-3)
• The last commit is two and a half months old

Cost

7. What to do next

If you are in the middle of choosing a model, use this tool as a model-temperament meter. Forget the return ranking; look at "does it see the fee / can it wait / how much does it put on at once" and the value shows up. After that, one question is left — you have the decision, but who is watching it 24 hours a day.

The next part comes at the same question from the other side: instead of standing several up, a tool that hands one of them a long leash.

weltiq setup — 3 lines

1. Sign up: {{CTA_URL}}
2. My → Connections → issue a key
3. claude mcp add weltiq https://weltiq.ai/mcp --header "Authorization: Bearer <key>"

Write a decision out as a rule and it runs in simulation on live candles, leaving fills, fees and slippage in a ledger. Simulation only.

The full test record (the install workarounds, the raw text of all 3 missions, the raw text of the 60-run fee A/B, and the defect list) is at reports/a/23-crypto-trading-arena/.