CoinRithm — a paper sandbox for AI trading agents, with a public P&L leaderboard

One key from sign-up, used read-only across CoinRithm, and the same things measured on weltiq alongside it. Plus the one question the two answered in opposite d

Part 8. CoinRithm — a company saying the same thing weltiq does has hung up a P&L leaderboard

This is the first part in the series to cover a web service. There is nothing to install. And the sentence this thing sells is "a paper sandbox for AI trading agents" — very nearly the sentence weltiq, the company I write this series for, sells. There is one question the two answered in opposite directions. CoinRithm publishes a P&L leaderboard of its agents; weltiq has a ban on P&L leaderboards written into its business documents. I am not going to rule on which side is right — I will set out both sides' reasoning with real numbers and leave the judgment to you.

What counts as finishing this article: you read the account, the ledger and a quote with one key from sign-up, pull the calibration scores for twelve prediction-market venues, and finally judge for yourself, on your own numbers, what hanging up a leaderboard gains and what it costs.

What it does Hands an AI agent 50,000 mUSD of fake money and runs it on real prices. It can place orders in spot, futures and prediction markets, and its record goes onto a public leaderboard
Who it fits Anyone who wants to look across several prediction-market venues · anyone who wants to build an agent and line it up against others · anyone comfortable reading and writing against an API contract
Who it does not fit Anyone who wants a loss limit enforced on a strategy · anyone who wants to swap exchange fee schedules and cost it out · anyone who wants Korean equities (there are none)
What you end up with An account, ledger and quote read out with one key, a calibration scorecard for five prediction-market venues, and five comparison tables measuring both sides with the same ruler
Cost to get started Sign-up free (there is no pricing page at all) · model supplied by them if you use the Studio · key login needed only to issue one · weltiq only if you work through the section 5 comparison
Time & difficulty 0 seconds to install. The leaderboard and the prediction-market aggregation are instant, no login. Difficulty low (reading) / medium (API)
Relationship to weltiq It does exactly the same thing. Section 5 is not "how to connect" but a pick-by-purpose table and comparison tables
Test environment & date 2026-09-22, 02:15–02:40 KST · API contract v1.7.0 · hosted MCP server 0.7.12
Disclosure The author is paid by weltiq for this series. Tool assessments use only results the author ran personally

On the CoinRithm side I only read. I have an account, but I placed no orders in their paper account, created no agent, and did not register for the leaderboard. Partly that is just following their own guidance, which says "Start read-only", and partly it is because I do not think writing into someone else's service is a reviewer's job. So "here is what happened when I placed an order over there" is not in this article. I did not measure it. The instruments and agent names appearing here are examples used to confirm functionality and are not trading opinions of any kind; the leaderboard numbers are that service's output and I do not vouch for them. Raw responses and the list of what I could not measure are in the verification record.


1. What this is

CoinRithm is a web service that hands an AI agent fake money and runs it on real prices. The agent runs once a minute, must take a quote before ordering, and every decision is recorded. There are two ways to attach: pick a ready-made character in the Studio and deploy it (a free hosted model comes attached), or connect your own agent over MCP or a ChatGPT Action. The latter is the way this series covers.

Item Value How it was checked
URL coinrithm.com · API api.coinrithm.com
Entity Bees-x Limited · England and Wales · company no. 13308136 site footer
API contract v1.7.0 (openapi: 3.1.0) · 54 endpoints https://www.coinrithm.com/openapi.yaml
Hosted MCP https://mcp.coinrithm.com/mcp · server coinrithm-trading 0.7.12 · 38 tools MCP initialize · tools/list
Paper capital 50,000 mUSD · the cash coin is USDT (coinId 825) GET /api/agent/portfolio
Pricing There is no pricing page (/en/pricing → 404). The front page says "$0 · 100% free" live call
Live trading None. Paper only API docs

The CoinRithm front page ↑ The CoinRithm front page. The sentence on the left is what this company sells — "you need a sandbox before you touch real capital". The black terminal on the right is a mock-up, and they label it themselves in the top right: "illustrative session, not live data". Not many companies attach that label

The same: real prices · fake money · unattended 24-hour execution · attaching over MCP · pricing the fills · keeping records · no live trading. Three things differ.

CoinRithm weltiq
Prediction markets Aggregates and compares across 12 venues. Places orders there too None
Hosts the agent Attaches a free model and runs it for you No. Judgment stays on the user's side
P&L leaderboard Publishes one (Agent Arena) Banned (a clause in the business documents)

The third is the body of this article, in section 6.


2. Getting in — how far without a login, and where the key is

With nothing to install, this slot is the access path.

Screen URL Login
Agent trading guide (the starting point) /en/agentic-trading Not required
Agent Arena leaderboard · top-agent detail /en/arena · /en/arena/<handle> Not required
Studio (building an agent) /en/my-agents/studio Not required up to picking a character
Paper trading screen · prediction-market hub /en/mock-trade · /en/prediction-markets Not required
API key issuance /en/settings/api-keys Required

There is exactly one login wall — issuing a key. The leaderboard, the top agent's scorecard and the prediction-market aggregation are all just visible. Most services of this kind push a sign-up form at you first. I could not find a paywall — /en/pricing is a 404 and there is no pricing link in the footer. (Where the money comes from, I could not determine.)

The CoinRithm paper trading demo ↑ The CoinRithm paper trading screen (logged-out demo; the numbers are samples). The green "Personalized Feedback" button in the middle and the yellow "Golden Rules" are this service's coaching layer — a box weltiq does not have. Counting wins and losses out loud, as in Wins 14 / Losses 19 · 42.4%, is not on weltiq's screen either. The table on the left even splits out the frozen quantity (Frozen) per coin

The moment you go in, a "Golden Rules of Trading" popup covers the screen — "don't average down into a loser", "always set a stop", "never stake more than 1–2% of capital". And yet the order API has no spot stop-loss field. The education layer and the execution layer say different things (confirmed against the schema in 5-3).

Issuing a key — and attaching over MCP

The CoinRithm API key screen ↑ The CoinRithm API key screen (logged out). Look at the yellow box at the top, "Staying in control" — "we recommend starting read-only", "a key acts as you", "revoke it immediately if it leaks". The four Permissions boxes below are the whole of this service's permission model: Read (always on) · Spot trading · Futures trading · Prediction markets

The key is 59 characters beginning with crk_.

$ curl -H "Authorization: Bearer $KEY" https://api.coinrithm.com/api/agent/me
{"userId":119,"keyId":104,"agentName":null,"agentModel":null,
 "scopes":["read","trade:spot","trade:pm","trade:futures"],
 "usage":{"period":"2026-09","requests":1}}

This one key is everything — the same key reads the account, takes quotes and places paper orders. All four scopes are on, so they are all switched on by default at issuance (even though the guidance recommends starting read-only). To follow that guidance, keep Read and turn the other three off.

The line that attaches is identical on both sides — one line, one header, the same transport.

claude mcp add --transport http coinrithm https://mcp.coinrithm.com/mcp \
  --header "Authorization: Bearer $COINRITHM_API_KEY"

claude mcp add --transport http weltiq https://weltiq.ai/mcp \
  --header "Authorization: Bearer $WELTIQ_API_KEY"

One difference is interesting: CoinRithm attaches without a key. Call tools/list with no key and all 38 tools come back (the same 38 with a key), and weltiq is the same design — 68 come back with no key. Both are "the tool surface is public, permission is at call time". But on their side 11 of those actually work without a key — the ten pm_data_* tools and get_crypto_movers.

# pm_data_overview without a key → HTTP 200
  {"stats":{"totalMarkets":1587752,"totalOpenMarkets":48961,
            "totalVolume":7382344484.474279, …}}

# whoami without a key → HTTP 401
  {"error":"missing_api_key","message":"No API key for this request. On the hosted
   MCP, send `Authorization: Bearer crk_live_…` with your own key."}

# weltiq MCP without a key → {"text":"Error executing tool markets","isError":true}

The prediction-market data is public and free in its entirety. weltiq has no such box — all 68 are blocked without a key. And the way they block differs: theirs returns a 401 that tells you the next action, while this side returns one line with no indication of how to fix it.


3. First result — what one key reads out

All reads. The account first.

$ curl -H "Authorization: Bearer $KEY" https://api.coinrithm.com/api/agent/portfolio

{"walletId":278,"bookScope":"api_key",
 "equity":{"totalUsd":50000,"available":50000,"frozen":0,
           "valuationBasis":"wallet_assets_spot_marked_futures_pm_at_collateral"},
 "pnl":{"24hUsd":0,"allTimeUsd":0, …}, "openOrders":[],
 "progression":{"league":"iron","xpPoints":0,"rankInLeague":31, …}}

This is what success looks like. Three things to point at. bookScope: "api_key" — the book is per key, so issuing another key gives you another account (weltiq splits capital per strategy container — the same problem solved differently, and theirs is simpler). valuationBasis — the valuation basis is pinned into the response as a string, so "how was this number computed?" never comes up. progression — league, XP, rank, daily tasks: a gamification layer sitting inside the account response. I had done nothing and was already 31st in the iron league, and that is the first clue for section 6.

The ledger (/api/agent/ledger) records not just orders but reads, one row each, with a latencyMs attached — a field weltiq's order path does not have. And in auditStats on the performance summary (/api/agent/performance) sits quoteBeforeTradeRate — "the share of orders that took a quote first" as a metric (mine is null, no orders). I have not seen that made into a metric before.

The cost of a fill is shown up front in the quote

I placed no orders, but a quote is a read endpoint, so I took one.

$ curl -X POST … -d '{"coinId":"1","side":"buy","quantity":0.001}' \
    https://api.coinrithm.com/api/agent/spot/quote

{"eligible":true,"blockReasons":[],"side":"buy","quantity":0.001,"orderType":"market",
 "executionPrice":85920.01571707883,"estimatedCostMusd":85.92001571707883,
 "estimatedFeeMusd":0.04296,
 "executionModel":{"version":"paper_execution_v1",
                   "feeBps":5,"spreadBps":4,"slippageBps":2,
                   "fundingMode":"not_modeled",
                   "assumptions":["paper execution",
                     "deterministic half-spread + slippage + taker fee",
                     "funding, order-book depth, latency and market impact not modeled",
                     "not an exchange fill guarantee"]},
 "freshness":{"asOf":"2026-09-21T17:30:14.563Z","ageSeconds":32,"status":"fresh"},
 "observation":{"schema":"coinrithm.agentObservation.v1",
                "observedAt":"2026-09-21T17:30:46.154Z",
                "sourceAsOf":"2026-09-21T17:30:14.563Z",
                "rowCount":2,"hash":"94e8810b1b47d42408b89537"}}

There is a lot in that one blob — a 5bp fee + 4bp spread + 2bp slippage, 11bp one way, handed over as raw numbers; what is not modeled written down (funding, order-book depth, latency, market impact); ageSeconds: 32 telling you the price is 32 seconds old; and observation recording when this read saw what, with a hash. The API docs also carry this line — "a flat round-trip is a small loss, not a free breakeven".

Where you might get stuck

Symptom Cause Fix
https://coinrithm.com/en/docs/api is a 404 There is no API docs page Fetch the schema itself: https://www.coinrithm.com/openapi.yaml
Cannot GET /api/agent/spot/quote A quote is a POST (a read, but a POST) -X POST with a JSON body
{"error":"q (symbol, slug, or name) is required"} /api/agent/resolve takes q, not query ?q=BTC
Orders demand a coinId It is a UCID, not a ticker (BTC = "1") Convert with resolve first
429 120/min per key · 20/min for trading writes Watch the RateLimit-* headers

4. Getting the most out of it — crossing twelve prediction-market venues to see who forecasts best

This is what CoinRithm does best, and what weltiq does not have at all. No key needed.

Can you tell which prediction-market venues forecast best?

$ curl https://api.coinrithm.com/api/prediction-markets/calibration
Venue Sample Calibration error Mean probability on winners
Polymarket 952 0.0216 0.5242
Manifold Markets 281 0.0422 0.7160
Kalshi 3,313 0.0569 0.4749
Limitless 110 0.1030 0.5059
Gemini 46 0.1157 0.6038

The methodology rides along in the response — it compares the price 24 hours before resolution against the actual outcome (leadHours: 24) and scores nothing below a sample of 30 (minSample: 30). Venues it could not score go into pending with a reason (Futuur: "only 3 scorable events, 30 required").

The CoinRithm prediction-market hub ↑ The CoinRithm prediction-market hub. The tab row at the top is the list of what this tool does — Compare · Whale Trades · Where venues disagree · Venue calibration · Sources. The 12 venues on the SOURCE COMPARISON line below, plus 48,899 active markets · $7,377,246,408 in total volume, are the scale of the aggregation

One call, no key, twelve venues tied together, and who forecasts how well, with the sample size attached. weltiq does not have this axis at all. That is not evidence of superiority but a question of position — weltiq does bar-level strategy execution; they have layered cross-venue prediction-market research on top of it.


5. Alongside weltiq — with the same ruler

Because the two overlap head-on, this slot holds comparison tables instead of a connection procedure. Every value was confirmed by a live call today, and nothing here is an estimate. First, which side for which purpose.

What you want to do Pick Why
Look across several prediction-market venues and compare CoinRithm 12 venues, 48,899 markets · per-venue calibration error (section 4). weltiq has none of it
Get an account on sign-up and run an agent immediately CoinRithm 50,000 mUSD · one key · a hosted model thrown in
Read the contract and wire it yourself CoinRithm OpenAPI 3.1 published (233KB · 54 endpoints) · listed in 6 MCP registries
Have a loss limit enforced on an order weltiq Without stop conditions a strategy container is rejected with a 422 before it is created. Their spot schema has no stop field at all
Swap exchange fee schedules and re-measure the same fill weltiq 6 fee schedules with dates and verification flags. Theirs is a fixed 5bp
Build a strategy as a document and leave it resident weltiq 8 of the 22 state-changing tools build and edit strategy documents. They have no such layer
Show your records to other people CoinRithm Scorecards and decision records are public with no login. weltiq publishes nothing (6-3)

5-1. What it costs to attach

CoinRithm weltiq
Install · the line that attaches None. Web + key · one claude mcp add --transport http … Identical
Auth Authorization: Bearer crk_… (or X-API-Key) Authorization: Bearer wq_…
Public API docs OpenAPI 3.1 published (233KB, 54 endpoints) Docs exist, but no published OpenAPI spec
Registry listings MCP Registry · Glama · Smithery · mcp.so · PulseMCP · MCP Market None

This is where they win. I hunted for the API docs for a long time, and the schema file itself was the documentation. That is better — the document humans read and the contract machines read can never drift apart.

5-2. Tool surface and the permission line

CoinRithm weltiq
MCP tools 38 68
Of those, state-changing 7 22
Work without a key 11 (10 pm_data_* + get_crypto_movers) 0
Per-minute limit 120 (+20 for trading writes) 120 (Pro 600)

CoinRithm's 7: place_spot_order · cancel_spot_order · open_futures_position · set_futures_sl_tp · close_futures_position · open_pm_position · report_pm_opportunity.

weltiq's 22: instantiate · backtest · backtest_submit · backtest_cancel · pine_import · rules_import · strategy_thesis · my_template_save · my_template_apply · slot_run · slot_update · slot_update_cancel · slot_stop · slot_start · slot_entry_control · slot_trace · promotion_evaluate · order_oneoff · order_amend · order_cancel · paper_order_submit · paper_order_cancel. (The three backtest tools do not place orders, but they leave an execution record on the server, so they go on this side. pine_preview and my_template_resolve are POSTs but save nothing, so they are left out.)

Having more is not better — 8 of weltiq's 22 (instantiate·pine_import·rules_import·strategy_thesis·my_template_save·my_template_apply·slot_update·slot_update_cancel) are tools for building and editing a strategy document, and CoinRithm has no such layer. Same phrase, "paper trading", but the unit is different — theirs is the order, this side's is the strategy document.

Where the permission line is drawn is the real point. Both sides have defined "things a key cannot do", and what they defined differs.

What a key or MCP cannot do CoinRithm weltiq
Widen scope or permissions No — dashboard only No — browser session only
Cap on order notional No such concept No — limit approval is session-only
Connecting live trading There is no live trading No — registering a connector or resuming is session-only
$ POST /v1/envelopes {"symbols":["BTCUSDT"],"max_order_notional":"500", …}
HTTP 403 {"detail":{"code":"session_required",
          "detail":"/v1/envelopes 은(는) 로그인 세션에서만 호출할 수 있습니다 (API 키 불가)"}}

(The Korean detail reads: "/v1/envelopes can only be called from a logged-in session (API keys not permitted)".) The delegate cannot widen its own limits — "an agent cannot widen its own limits." CoinRithm has no such layer: if the scope is trade:futures it opens positions up to 20x leverage with no notional cap (FuturesOpenRequest: leverage max 20, marginMusd min 10, no cap). In exchange it splits scopes finely (spot / futures / prediction markets separately) and its guidance repeatedly recommends starting read-only.

In one line: CoinRithm divides on "what you can do", weltiq on "how much you can do".

5-3. What each demands for a single order — this is where they split

Same "paper spot market buy", different demands.

Field CoinRithm weltiq
Instrument coinId (UCID, not a ticker) — required venue + symbol — required
Quantity quantity — required qty or size — required
Idempotency key idempotencyKey — required client_order_id — optional
Why you are buying agentTrace.rationaleSummary — optional thesis.why — required (422 without it)
Where you fold No field on spot · futures stopLossPrice optional invalidation.px_below/px_above — required (422 without it)

I actually took both 422s on the weltiq side.

$ POST /v1/orders/oneoff {… without "thesis" …}
422 {"detail":{"code":"thesis_required","detail":"가설(thesis)이 필요합니다"}}

$ POST /v1/orders/oneoff {… without "invalidation" …}
422 {"detail":{"code":"invalidation_required","detail":"청산 조건(px_below/px_above)이 필요합니다"}}

And here is the box weltiq loses. Place an order with no idempotency key and this side warns and accepts it anyway ("warning":"no_client_order_id" — "this order was sent without a client_order_id; send the same request again and a second order goes in"). They pin idempotencyKey as required and write in the docs — "a timeout does not establish whether the server accepted a write." In a world where agents place orders, the most common accident is a double order from a retry after a timeout; they blocked it in the schema, this side blocks it with a warning string. A warning string is something an LLM can read and ignore. weltiq loses this one.

5-4. The fill-cost model — and the measured round trip on weltiq

Since I decided not to write on their side, the boxes that need an order ran on weltiq only. I bought 0.001 BTCUSDT and sold it back 25.4 seconds later.

Buy Sell
Time 02:25:07.850550 02:25:33.263264
Fill price 85860.2751685 85843.0948315
Fee 0.08586028 USDT 0.08584309 USDT
Liquidity · price source taker · quote_source: binance · quote_scope: binance-spot 〃
Fee rule fee_profile: {rule: venue, profile: binance-spot, maker: 0.001, taker: 0.001} 〃

Over 25 seconds the price fell 2.00bp and the round-trip fee was 19.998bp. Net P&L −0.18888371 USDT — $85.86 held for 25 seconds and $0.19 gone. That is their docs' "flat round-trip is a small loss". Order acceptance took 58ms and the first read already said filled. But the fill response does not say how many bp were spread and how many were slippage.

Then I swapped exchange fee schedules at the same size.

$ GET /v1/fees/simulate?asset_class=crypto-spot&notional=85.86&trades_per_month=2&maker_share=0

Bithumb KRW Market      taker=0.0004  eff=0.00040  as_of=2026-09-16  verified=true
Upbit KRW Market        taker=0.0005  eff=0.00050  as_of=2026-09-16  verified=true
Binance Spot            taker=0.001   eff=0.00100  as_of=2026-09-16  verified=true
Bybit Spot              taker=0.001   eff=0.00100  as_of=2026-09-16  verified=false
OKX Spot                taker=0.001   eff=0.00100  as_of=2026-09-16  verified=true
Bitget Spot             taker=0.001   eff=0.00100  as_of=2026-09-16  verified=true

The same round trip runs 4bp to 10bp depending on the exchange. CoinRithm has no such box — just paper_execution_v1's fixed 5bp.

CoinRithm weltiq
Model name paper_execution_v1 RealityProfile (venue+version)
BTC spot, one way 5bp fee + 4bp spread + 2bp slippage = 11bp 10bp fee + (½×spread + 1bp) ≈ 11bp
Swapping fee schedules None (fixed) 6 exchanges, with dates and verification flags
Cost breakdown in the response Published as executionModel on every quote and fill None — only fee on the fill response
What is not modeled Stated in the response (funding, book depth, latency, market impact) In code comments (private repository)
Price freshness Published as ageSeconds (32 s on my call) quote_delay_s: 0 · quote_source·quote_scope on the fill

The total cost is nearly identical, and the difference is whether the user can see it. weltiq has a slippage model too, but it does not ride in the response. Then again, the actual bp numbers in their model live only in the response as well, and the published package code only says "the backend is the source of truth" — so both come down to "trust whoever built it".

5-5. Stop conditions and records

On the weltiq side I confirmed the gate but never saw it trip. Try to create a strategy container with no stop conditions and it is blocked at the schema.

$ POST /v1/slots {…, "symbol":"BTCUSDT","tf":"1m","balance":"200"}   ← no stop conditions
HTTP 422 {"detail":[{"type":"missing","loc":["body","stop_max_loss"],"msg":"Field required"},
                    {"type":"missing","loc":["body","stop_after_hours"],"msg":"Field required"}]}

Without a loss limit and a deadline the container cannot exist. So I set them very tight and launched — 0.30 USDT of loss, 1 hour.

The list of resident weltiq strategy containers ↑ The resident weltiq slot list. The top row is the container I launched — Status running but Position is —. On the right, Auto-stop −0.30 · until 9/22, 03:25 AM is the boundary that was mandatory. The screen says it in as many words: "A running slot holds no position until its entry condition is met"

But I never saw it trip. It ran for 12 minutes taking 1-minute bars and the entry condition never came up (stage: conditions · reason: pine_hold). With no position there is nothing for a loss limit to catch — so the scene of a loss limit actually cutting in is not in this article. On the way down, the stop was not instant but went through stopping, and the response said it holds protection until the position is confirmed flattened.

And on their side? I only read, so I could not trip anything. But the contract tells you something: the spot order schema (SpotOrderRequest) has no stop field at all, and stopLossPrice on futures (FuturesOpenRequest) is optional. Which means orders go out over there with no stop conditions.

CoinRithm weltiq
Record granularity Every call (reads included) Orders, fills, state transitions, journal
Latency Recorded as latencyMs None (only backtest duration)
Trace metadata runId·decisionId·strategyLabel·confidence·rationaleSummary The rationale sentence · strategy-document hash · lineage hash · experiment hash
Audit metrics runTraceCoverage·decisionTraceCoverage·quoteBeforeTradeRate None
Integrity contentHash per decision · contentHash per scorecard Strategy-document hash · experiment hash
Export export_agent_ledger (1,000 rows max) · export_run_evidence slot_export (20,000 rows by default, fee reconciliation included)
Public? Scorecards and decision records are public with no login Nothing is public

Boxes weltiq wins — export volume (20,000 against 1,000), fee reconciliation, and the lineage that threads backtest and paper together through the strategy-document hash. It loses three. ① There is no latency on the order path. ② It does not evidence "the data this decision could see" on every read — it blocks lookahead structurally with confirmed bars, but does not record it per decision. ③ There is no public surface. That is the biggest one, and it is the next section.


6. After using it — and why one side hangs up a leaderboard and the other does not

This section is why I wrote the article. Agent Arena on CoinRithm's front page is a public leaderboard ranking agents by P&L, and first place right now is Leverage Leo, +1,527 mUSD. weltiq's business documents say "honest simulation is the brand: no P&L leaderboards, no optimistic fills, no copy trading." Same material, opposite conclusions.

The Agent Arena leaderboard ↑ The Agent Arena leaderboard. The table in the middle is ordered by realized P&L, and LIVE REASONING on the right streams what the agents just decided, with probabilities — "BTC is in a mild uptrend but shows no fresh high and RSI is only 53.5 · 55% · 30 minutes ago". Look at RECENTLY ACTIVE at the bottom right: all four recent entries are red negatives**

6-1. Five reasons they may be right

I went in dismissive of leaderboards and changed my mind.

One — this is not a highlight reel. I pulled all four pages of the leaderboard and counted.

Agents listed Positive Negative Median Worst Sum of all 38 Mean win rate
38 10 28 −152.245 −19,187.16 −72,018.64 36.2%

Three quarters of the listed agents are losing money, and it shows exactly as it is. The −19,187 one is right there on the leaderboard. They could hide it easily enough.

The top of Agent Arena ↑ The top of Agent Arena. Of the five boxes in the band, look at AVG WIN RATE 36% — the people who built the leaderboard have written "the agents in here are wrong more than six times out of ten on average" onto the front screen. Of the nine lines streaming through LIVE FLOOR below, seven read lost with a red number. Each line carries why the agent did what it did, in its own words — "the market priced it at 46% but I see 55%"

Two — there really are devices to discount luck. The ranking contract rides along in the response.

"contract": {"version": "arena-ranking-v1",
  "ranking": {"listingMinimumDecidedTrades": 0,
              "qualificationDecidedTrades": 5,
              "positiveScore": "wilson_95_lower_bound_x_realized_pnl",
              "nonPositiveScore": "realized_pnl",
              "unrealizedPnlAffectsRank": false},
  "presentation": {"smallSampleBelowDecidedTrades": 20},
  "evidence": {"provesCoinrithmPaperExecutionRecords": true,
               "modelIdentity": "self_reported",
               "hiddenModelReasoningVerified": false}}

Positive P&L is multiplied by the Wilson 95% lower bound of the win rate — a small sample costs you score. Under 5 decided trades and you drop back as unqualified, under 20 you carry a small-sample warning, and unrealized P&L does not count toward rank. And it works — 3rd place a60-etuateautahdfgre (+342.70) sits above 4th place a99-x3-es (+346.88), and 29th place a30-mycelium-pm is positive at +48.94 yet sits below the −19,187 one because it has a single trade.

Three — open first place and you get numbers that argue against first place.

The first-place Leverage Leo detail ↑ The first-place Leverage Leo detail — it opens with no login. In the band at the top, REALIZED PNL +1,527 sits next to UNREALIZED −508 and MAX DRAWDOWN −7,057. Look at the label to the right of Verified Scorecard in the middle — "Reproducible · skill over raw PnL". And in the blue FORECAST SKILL box, SKILL VS MARKET −13% — meaning the top agent forecast worse than the market price it bought. The black window at the bottom left is the decision log including skips, and it records the rejections too, as in Rejects 2126

"metrics": {"realized_pnl_musd": 1527.160209, "trade_count": 315,
            "win_rate": 0.374603, "profit_factor": 1.07386,
            "sharpe": 0.008449, "deflated_sharpe": 0.561616,
            "max_drawdown_musd": 8533.321563, "calibration_error": 0.154098}
"forecastSkill": {"decidedCount": 61, "forecastedCount": 52,
                  "agentBrier": 0.277579, "marketBrier": 0.246012,
                  "brierSkillVsMarket": -0.128316}

The top agent's Sharpe is 0.0084 and its forecast skill against the market is −0.128. The company that built the leaderboard publishes these numbers about its own number one — numbers you would never ship if bragging about returns were the point.

Four — every individual decision is public. /api/arena/decisions returns resolved prediction-market trades like this.

{"agent":"a39-my-contrarian-carlbgjbb","agentModel":"Llama 3.1 8B",
 "question":"Bitcoin price on September 21?","chosenOutcome":"86,000-88,000",
 "predictedProbability":38,"agentForecastProbability":65,"edgePoints":27,
 "stakeMusd":25,"result":"lost","pnlMusd":-25,"brier":0.1444,"agentBrier":0.4225,
 "entryContext":{"marketAsOf":"2026-09-21T15:26:40.142Z","chosenProbability":38},
 "contentHash":"8cee806914afe42a63eea3aeb483f564fb181faef194292b5a0ec2a162a9395c"}

What the agent said ("65% probability") and what the market priced (38%) are both preserved, and it lost. The market at entry (marketAsOf) is frozen, so nobody can do "well, at the time…" after the fact. That is lookahead prevention evidenced per decision, and weltiq has no exact counterpart. The dataset description even writes down its own trap — "multi-outcome Brier is not cross-comparable, so never rank agents by it."

Five — if nobody can see it, it is not evidence. What surface other than a leaderboard shows that unattended execution is genuinely running? Three of the 38 are alive right now, and 13,573 cumulative decisions stream across the screen. weltiq has no such screen.

6-2. Five reasons this side may be right

Granting all five above, something remains.

One — what stays on the front screen is one line. All that sophistication lives within two clicks, while what is pinned to the front screen is +1,527, and the −508 (unrealized) in the next box does not count toward rank. By total account value Leo is at 51,019 — he actually made +1,019 while the leaderboard number reads +1,527. Not dishonest, since the contract says so — but the structure remains that ranking on realized P&L alone favors whoever cuts winners and carries losers.

Two — a Wilson lower bound discounts the error in a win rate, not the tail in the P&L.

Realized P&L Trades Win rate
Spot +77.10 1 100%
Futures −3,971.32 253 37.2%
Prediction markets +5,421.38 104 37.7%
Total +1,527.16 358 37.5%

He lost 3,971 in futures and made 5,421 in prediction markets to take first place. The single best trade was +8,083.94 — more than five times the final +1,527. A Wilson bound asks "is this win rate a sampling artifact?", and at 37.5% over 315 trades the sample is plenty; what made the P&L was not the win rate but one tail. They know it, which is presumably why deflated_sharpe ships alongside — but the ranking is still by P&L.

Three — much of what this leaderboard measures is "the spread of running one strategy many times".

The CoinRithm Studio ↑ The CoinRithm Studio — visible this far with no login. Five starter characters are the whole of it: Momentum Mia · Contrarian Carl · Levelheaded Leo · Odds Olivia · Steady Sam. "Advanced / bring your own agent" at the bottom is the door to attaching your own model

Counting the handles on the leaderboard: 8 Mia derivatives (a3-momentum-mia·a73-your-mia·a49-m-mia·a42-mon-mia·a48-mimi·a88-mia·a93-my-mia·a101-my-mia757899) · 5 Contrarian Carl · 5 Olivia · 2 Leo · 2 Sam — by name, 22 of the 38 are derivatives of those five characters. (Counted by name only; I could not confirm whether the strategies are actually the same.)

An LLM answers differently to the same input, so running one strategy many times splits the results. a3-momentum-mia is at −6,596 and a73-your-mia is at +122. That is not two strategies ranked against each other; it is the spread of one strategy. Yet the leaderboard stands it up vertically, and a reader has no way not to read spread as skill. Paradoxically it is genuinely good data — the distribution over repeated runs is the right way to measure skill. But put that information in a leaderboard and it reads as a ranking.

Four — first place has not traded in nearly two weeks. The API says "lastTradeAt": "2026-09-09T00:03:31.323Z" — 13 days before the observation date. (The screen says "Last active 17 days ago", so the two disagree. I could not determine which is right; both say "about two weeks".) The ranking window is three months, so first place in a three-month window is not a current state. If someone reads the leaderboard and clones it (Deploy a copy - free is on the detail page), what they clone is something that stopped two weeks ago.

Five — rank by P&L and the high-variance side floats to the top. Of the 38, 30 use futures (5 use spot). Futures leverage goes to 20x with no notional cap, so on a board sorted by absolute P&L the fastest way up is to bet big. First place is named Leverage Leo. The original Studio character description reads "Levelheaded Leo — patient, picks clean setups, skips the chop"; somebody forked that character, renamed it Leverage Leo, and that is number one.

6-3. So here is what weltiq loses

Written out this far it looks as though this side wins. It does not. weltiq's business documents say "honest simulation is the brand." And yet there is no surface on which that honesty can be seen.

CoinRithm weltiq
The top agent's actual trade record Opens with no login N/A (not published)
The top agent's Sharpe, max drawdown, forecast skill Public N/A
Per-decision records + hashes Public API Private
The fill-cost model's numbers In the response Private
The fill engine's code Private Private
Methodology document /en/methodology, public Internal document

The right-hand column is nearly empty. This side's records may well be more accurate, but checking that means trusting weltiq, while theirs takes one URL. Not building a leaderboard and publishing nothing at all are different things. weltiq decided the former and is doing the latter. This was the most uncomfortable passage for me to write, and I am writing it down as it is.

The side that hangs a leaderboard The side that does not
Gains Evidence that execution really runs · discovery · motivation · comparability Does not let spread be misread as skill · does not reward risk-taking
Loses One line on the front screen covers the rest · P&L ranking invites leverage · a stopped agent stays at number one No place to show the evidence

My conclusion is not "one side is right". The problem with a leaderboard is not the data but the vessel. Most of the data CoinRithm puts in it is honest and some of it is better than this side's — but a table sorted vertically is a vessel that reads as one line, however honest the data. And the vessel can be changed: lay the same data out horizontally as "the distribution of one strategy over eight runs" and it becomes spread, not rank. Their leaderboard already holds all of that data.

6-4. What was good and what was not

Good — exactly one login wall (the leaderboard, the top scorecard and the prediction-market aggregation all just open) · the OpenAPI source is the documentation (the human document and the machine contract cannot drift apart) · it writes what it cannot do into the response (fundingMode: not_modeled, hiddenModelReasoningVerified: false, modelIdentity: self_reported) · I have not seen cross-venue prediction-market aggregation anywhere else in this field (12 venues, 48,899 markets, down to per-venue calibration error).

Not good — there is no boundary on an order (no stop field on spot at all, optional on futures, no notional cap, and leverage open to 20x) · the API docs page is a 404 (hard to learn from the front page that the schema source exists) · the default at issuance is every scope (the guidance recommends read-only, yet the key came back with all four on) · the ranking contract is careful but the front screen is one number (the five points in 6-2).

6-5. What this article does not tell you


7. What to do next

Next is Part 9, InvestSkill. If CoinRithm "gives you an account", InvestSkill is the side that "gives you a judgment", and it sits at the low end of this series for attachment cost — its output format is standardized, so carrying a conclusion into a rule is nearly mechanical. If this part compared something that overlaps head-on, the next one is about fitting into an empty box.

Getting weltiq ready — 3 lines 1. Sign up: {{CTA_URL}} 2. My page → Connections, issue an API key 3. claude mcp add weltiq --header "Authorization: Bearer <key>" https://weltiq.ai/mcp

This is where to look when you need the right-hand column of the section 5 table — enforced loss limits, swappable fee schedules, resident strategy documents.


Every call, output and number here was actually run on 2026-09-22. On the CoinRithm side I only read; every weltiq order is simulated. None of this is a trading opinion. Raw responses and the list of what I could not measure are in the verification record.

Tony Brown · 2026-09-24