Guide / AI in trading

Why LLMs lose at trading and the responsible ways to use AI in trading

Every public LLM-vs-market competition we have looked at has the same shape: the model produces fluent, confident, plausibly-reasoned trade ideas, and then loses money slowly. Retail-trading subreddits are full of variations on the same observation, often in the form of disappointed surprise. This is not a failure of any specific model; it is a structural mismatch between what language models optimize and what trading rewards. This guide is the honest version: why LLMs lose, where they actually add value, and the architecture that lets them help instead of hurt.

The pattern: LLMs trading like overconfident retail

Across multiple public trading competitions over the last 18 months, frontier LLMs have produced trading journals that read like a thoughtful retail trader: a thesis grounded in recent news, a position sized relative to a stop, a daily review. The PnL distributions cluster around "break-even before fees, losing after fees," with occasional outliers in both directions driven by single-stock news. The dominant failure mode is the one you would expect from an overconfident retail trader: holding losers too long, rationalizing the thesis after the price moved, and concentrating into recent winners. The LLM is not making different mistakes from humans; it is making the same mistakes more fluently.

The Reddit and X discussion around these results has converged on a few themes: ChatGPT "trades like a human and loses like one," all-LLM bracket competitions produce no clear winner against buy-and-hold, and naive prompt-driven agents underperform a simple 60/40 portfolio over enough trading days. We are not going to quote those threads verbatim. The pattern is real, the explanation is structural, and the explanation is the part this guide is for.

Why free-form market prediction is the wrong job for an LLM

A language model is trained to predict the next token in a sequence drawn from its training corpus. The loss function rewards calibrated linguistic plausibility, not calibrated financial accuracy. Three consequences fall out of that mismatch.

No calibration in probability space.When a model writes "I am 70% confident this breakout holds," the 70 is a token, not a calibrated probability. Re-prompt the same model on the same setup with slightly different framing and the number swings by 20-30 points. A probability you cannot reproduce is not a probability; it is a vibe.
Training distribution = the noise.A meaningful share of the LLM's "market views" training data is retail content: Reddit posts, finance Twitter, blog commentary. That is precisely the population a profitable strategy needs to beat. The model has absorbed the average opinion and is now repeating it back in clean prose. The average opinion is, by definition, already in the price.
Compounding error in multi-step reasoning. A model that is 80% accurate on each step of a five-step technical analysis is 33% accurate end-to-end. Add a sixth step and you are at 26%. Markets reward sustained accuracy across many small decisions; autoregressive generation is the wrong tool for sustained accuracy.

There is also a subtler problem: model drift across versions. A backtest run against GPT-4o in September will produce different trades than the same backtest run against the same prompts in March. Most trading strategies decay because the market adapts; an LLM-driven strategy decays because the model itself changes underneath you.

Where LLMs do add value

None of the above means "do not use AI for trading." It means do not use AI for the part that has no edge. There are at least four jobs LLMs are unambiguously good at:

Function-calling for data lookup."Show me TSLA's last four earnings surprises" should not require remembering an API. The LLM picks the right tool, fills the parameters, and renders the result. The numbers come from the tool, not from the model.
Query intent classification.A user typing "find me oversold names in the semiconductor space with positive analyst revisions" is describing a three-filter scan. An LLM is excellent at mapping that English to a screener spec; a screener is excellent at running it.
Summarization of news, transcripts, filings.A 10-K is 200 pages. Two paragraphs that answer "what changed since the last 10-K and which numbers are flagged for accounting adjustment" is genuinely useful work. The model is reading and condensing, not predicting.
Semantic search of a strategy library."Have I previously tested a long-only pairs trade on these two names?" against a corpus of saved backtests is an embeddings problem with an LLM summary on top. No prediction is involved.

A taxonomy: LLM as trader, analyst, orchestrator

We classify every "AI-in-trading" idea by which job the LLM is doing. Most of the failure stories are misclassifications: someone built an LLM-as-trader product and described it as LLM-as-orchestrator.

Pattern	What the LLM does	Where edge comes from	Verdict
LLM as trader	Predicts direction, sizes, sends orders	Hopes; no calibrated source	Bad
LLM as research analyst	Reads filings, summarizes news, suggests theses	Human still decides and trades	Decent
LLM as orchestrator	Routes intent to deterministic tools, summarizes structured output	From the tools (factor lib, scanner, backtester)	Great

The Copilot architecture pattern

The orchestrator pattern is what BlitzPulse Copilot implements. The shape of a request:

Intent classification. The user types a question. A small, fast LLM (Haiku-class) decides what kind of question it is: a quote lookup, a screen, a backtest, a portfolio risk query, a news summary, or an explanation of a previous result.
Tool dispatch.Each intent maps to a typed tool with a structured input schema. The LLM fills the schema (ticker, lookback, threshold) from the user's question; if any field is ambiguous, it asks a follow-up rather than guessing.
Deterministic execution. The tool runs in Python against real data. RSI is computed by a NumPy function, not predicted by an autoregressive generator. The output is a JSON object that the LLM never edits.
LLM summarization of the result.The LLM's job is to translate the structured JSON back into a paragraph the user can read. Every number in the paragraph traces back to the JSON; we render the underlying table alongside so the user can audit.

This is the same shape an OpenAI "function calling" API was designed for, and the same shape Anthropic's tool use API converges on. It is a translation layer, not an oracle. The LLM is the ergonomics; the edge is in the tools.

Honest critique: the costs we still pay

Token cost is real. A 4k-token Copilot turn against frontier models is a few cents. At scale that adds up; we route easy intents to cheaper models and reserve the frontier for hard reasoning (e.g. multi-step diagnostic flows on a portfolio anomaly).
Hallucination on numbers persists. Even with function-calling, a sufficiently distracted prompt can produce a summary that misreads the underlying JSON. We mitigate by rendering the table next to the prose and by running a numeric-consistency check that flags claims not supported by the table.
Arithmetic without tools is unreliable.The classic algotrading complaint ("the model said 10 plus 10 equals 22") is real, and the answer is never "train a better model." The answer is: do not ask the model to add numbers. Ask it to call a function that adds numbers.
Determinism for compliance. Some users need an auditable record of why the system said what it said. LLM outputs are non-deterministic by default; we store the input prompt, the chosen tool, the tool inputs, the tool output, and the model temperature for every Copilot turn so a turn can be replayed.

What this means for you

If you are building or buying an AI-in-trading product, the question to ask is not "how good is the model?" The question is "where does the edge come from?" If the answer is "the LLM predicts the market," the product will lose money in steady state. If the answer is "the LLM makes a research workflow 3x faster by routing to validated tools," the product is doing real work. Both are valid stories; only the second one is honest.

For the trader using these products: ask to see the tools the LLM is calling. Ask whether the numbers on screen came from the model or from a function call. Ask what happens when the model is wrong, and who is liable. If those questions do not have crisp answers, you are paying for prose, not for edge.

Frequently asked questions

Why do LLMs underperform in trading competitions?

Three structural reasons. First, language models are calibrated for token likelihood, not for the conditional probability of a financial outcome, so their confidence carries no edge. Second, their training distribution is dominated by retail content (Reddit, Twitter, finance blogs), which is the same noisy signal they would have to beat. Third, the autoregressive generation process compounds errors: a model that is 60% accurate on each of five sequential reasoning steps is 7.8% accurate end-to-end. Markets are competitive equilibria where the average opinion is already priced in; producing the average opinion in fluent English does not move you up the leaderboard.

Can an LLM ever beat the market?

Not as a free-form predictor of direction. As an orchestrator of deterministic tools (data lookup, news classification, structured query), an LLM can be the routing layer for a strategy that has edge for non-LLM reasons. The edge lives in the tools (a calibrated factor model, a low-latency news feed, a well-built backtester), and the LLM lives in the interface that makes those tools accessible to a human. This is not the same as the LLM beating the market; it is the LLM removing friction from a workflow that beats the market.

Why do LLMs make arithmetic errors on price math?

Token-level language models do not reason about numbers as numbers; they predict the next token. A model that has seen the sequence '10 + 10 = 20' a million times will reliably emit '20' for that exact prompt, but a sequence it has not seen ('compute the 14-day RSI of these 30 closes') is a chain of token predictions where the intermediate state has no enforced numerical consistency. The fix is not to make the LLM better at math; it is to make the LLM call a function that computes RSI in Python and then summarize the result. Function-calling is the right pattern; pure-LLM arithmetic is the wrong one.

Should I use an AI agent to auto-trade my account?

Not without a deterministic risk envelope around it. The pattern that loses money is: prompt the LLM, parse its JSON output, send the order. The pattern that does not lose money is: the LLM proposes a trade, a deterministic risk engine validates it against position-size, drawdown, sector, and concentration limits, and only then is the order sent. The LLM is the suggestion layer; the risk engine is the gate. If you cannot describe your risk envelope as code, you should not be running an autonomous agent on real capital.

What is the BlitzPulse Copilot architecture in one sentence?

Intent classification (an LLM decides what tool to call) routes to deterministic execution (price math, factor lookups, backtests, broker queries) and then the LLM summarizes the structured result in natural language. The LLM never invents numbers, never holds state across requests, and never sends orders without a human in the loop. It is a translator between English and our tool surface, not an oracle.

References

Efficient-market hypothesis - the priced-in-opinions framing.
Large language model - background on autoregressive next-token training.
Calibration in statistics - why "70% confident" only matters if it pays off 70% of the time.

Educational content, not investment advice. Information reflects our research as of publication and may change. Last updated 2026-05-31.