What if you ask an LLM Agent to make predictions instead of a Math function?

Imagine you're a great chef trying to make an ultimate hot sauce. You have different ingredients, and each one can be varied from 1 to, let's say, 50 drops. There are a billion combinations. The catch is that every time you brew a test batch, it takes 3 days to ferment and costs you $100. You can't just guess randomly, because you'll run out of money that way. You can't try every combination, because then it will waste a lot of time.

To make calculated decisions for such times, we use Bayesian optimization. It keeps a record of batches which were already cooked, mentioning the ingredients used, the amount they were used in, and how they turned out. It uses that record to make an educated guess about two things for any combination that you haven't tried yet:

how good it's going to taste
and how confident that guess actually is

This guessing engine is called a Gaussian Process.

A Gaussian process fitted through four data points: a solid line for the predicted function, with a shaded band showing how confident the prediction is between the points — Figure: Görtler, Kehlbeck & Deussen, A Visual Exploration of Gaussian Processes, Distill (2019). CC BY 4.0

And for the next batch, it has two options: either to lean on what's already scored well, or rather try exploring the untouched ingredients to find out if there is something which is better than the best we already have -- this is usually called the explore-exploit tradeoff.

I tried replacing this process with an LLM. It would make educated guesses by reading all the records, rather than a Gaussian process doing it. For testing and benchmarking this, I created four layers:

The classical Bayesian optimization
The LLM-based approach
The hybrid approach, in which both LLM and Bayesian optimization were combined
A human-in-the-loop implementation -- basically an agent with a reliability layer, so a human can catch it before it acts on a decision it has almost no information about

For this, I used a very famous benchmarking function called the Branin function for doing the comparison.

f(x_1, x_2) = a\left(x_2 - b x_1^2 + c x_1 - r\right)^2 + s(1 - t)\cos x_1 + s

a = 1,\quad b = \frac{5.1}{4\pi^2},\quad c = \frac{5}{\pi},\quad r = 6,\quad s = 10,\quad t = \frac{1}{8\pi}

x_1 \in [-5, 10],\quad x_2 \in [0, 15]

The following are the results.

1) Classical implementation of Bayesian optimization

In this layer, I built Expected Improvement by hand instead of importing a library, specifically so I'd understand the math with each iteration instead of treating it as a black box.

The true global minimum of the function was 0.3979, and the classical Bayesian optimization on 20 iterations found the best value to be 0.408 -- which seemed satisfactory, until pushed to 35 iterations, which brought it down to 0.3982. In a real setting, you'd never know if you had stopped 15 evaluations too early, because you don't have the true minimum to check against. That's a real practical problem, not a toy artifact. The algorithm has no way to know it's converged.

Classical Bayesian optimization convergence over 20 iterations, levelling off just above the true minimum of 0.398 — 20 Iterations

At 20 iterations the curve has gone flat and the search looks finished. Give it 15 more, and it keeps creeping down to the true minimum:

Classical Bayesian optimization convergence over 35 iterations, reaching the true minimum of 0.398 — 35 Iterations

2) What happens when you trust the LLM directly

This follows the same loop as before, but replaces the Expected Improvement function with an LLM to pick the next point given the full history as plain text. So the LLM is basically working as a consultant to the chef, proposing the next combination of ingredients to try.

As we knew, the true minimum was 0.3979, but the LLM converged to 0.4007.

Best value found per iteration by the LLM proposer compared with the classical Gaussian process — LLM vs Classical GP

For love of the game, I fed the LLM wrong points (a permuted call -- feeding wrong numbers), and it gave out the minimum to be 2.463, which is hilarious, because the LLM has no way to tell real feedback from fabricated feedback -- it trusts whatever it's shown, and predicts the next point accordingly.

Which actually diverges from a recent paper on LLMs for Bayesian optimization in scientific domains: that if a model is genuinely reasoning from the data, lying to it about the data should hurt the results. If it's not really using the feedback at all, scrambling it shouldn't hurt much.

Best value found per iteration by the LLM given permuted feedback, stuck near 2.46, compared with the Gaussian process — Permuted LLM vs GP

What actually happened was quite the opposite of "ignoring bad data." For 40 iterations, the LLM made zero progress. It never even beat its own starting guess, because it kept going back to the same two spots that the fake feedback told it were good -- and it ate up almost 82% of the search. Every point it tried landed in those two spots, and it kept searching.

Here's the part that actually surprised me: some of its answers came from real reasoning, and some came from random guesses, and I compared the two. The "reasoned" answers weren't any better than the "random" ones. If anything, they were slightly worse.

So the problem wasn't that the model ignored data it couldn't check -- instead, it trusted the data it couldn't check, and built a completely wrong picture of the world on top of it. If you're ever deciding whether to let something like this run on its own with real data, that's honestly the scarier version of the two failures.

3) Gate the LLM behind real math, not just a prompt

For fixing that problem, I introduced a hybrid layer.

What's happening here is that the LLM proposes five options, and the classical math from Layer 1 decides which one of them is the real winner and should actually be tried.

So now, the consultant's ideas don't skip the line anymore -- they go through the chef's actual notebook-based model first, and a good hunch can still win, while a bad one gets filtered out before you spend $100 proving it.

Looking at the numbers: the true minimum of the function is 0.3979. Classical BO on its own found 0.3982. Put through a hybrid loop, it came out to 0.4036 -- still not impressive, and it took a lot more tokens and iterations to get there. I'd guess a stronger model would close that gap further.

The more interesting number is the win rate. Out of every round, the LLM's own suggestion was the one actually picked 42% of the time (21 out of 50). That matters because it's not competing against nothing -- each round, its 5 suggestions were put up against 20 random guesses, which means pure luck alone would win about 20% of the time. So the LLM was winning more than double what random chance would predict. That's a real sign it was adding value, not just getting lucky occasionally.

Best value found per iteration by the hybrid LLM and Expected Improvement loop compared with classical Bayesian optimization — Hybrid vs Classical BO

4) Adding human in the loop for reliability

Now, what's happening is we pick up the same hybrid loop from Layer 3, but now it can genuinely pause mid-run and wait for a yes/no response from a human. It's LangGraph's actual interrupt() -- the program truly stops and waits on real input during an iteration.

This is like introducing a restaurant policy. Any batch in a flavor region with almost no data needs the restaurant owner's sign-off before the chef cooks it. Not second-guessing the chef's math from his notebook -- just acknowledging that, in the specific spots where nobody (chef or consultant) has enough information to be confident, the cost of being wrong is worth a second check.

So, what does "uncertainty" mean here, actually? It's the Gaussian Process's own uncertainty at the chosen point, calibrated against the model's real behavior. Points near existing data, which are well-explored, score low. Points far from anything tried score high. When the score is high enough, it pauses and asks the human for a decision; the rest gets handled automatically.

So, what happens when the human says no? Rejecting doesn't mean nothing gets tried that round -- it swaps in the best candidate the model already has genuine confidence about, instead of gambling on something totally untested.

Looking at the numbers: the true minimum was 0.3979, the classical best found was 0.3982, and the minimum found using hybrid + HITL was 0.4068. The interrupts fired 12 out of 25 times. The LLM's win rate was 48%, against a chance baseline of 20% -- since only 5 of the 25 candidates in each round came from the LLM.

Of those interrupts, 9 were approved by the human and 3 were rejected -- which didn't make a huge difference to the final minimum, but at least the human had the chance to supervise what was being tested.

Best value found per iteration by the hybrid loop with human-in-the-loop approvals, marking 9 approved and 3 rejected interrupts, compared with classical Bayesian optimization — Hybrid + HITL vs Classical BO

Conclusion

So, did I prove LLMs can replace a Gaussian process? Not exactly.

But did I prove there are decent sous chefs with genuinely good ideas who absolutely cannot be trusted alone in the kitchen with a $100 batch on the line? Yeah, I guess.

The consultant was confidently wrong for 40 batches straight when fed bad notes, without any hesitation. Give him a supervisor with a calculator, and suddenly he suggests things worth trying.

And then give the owner veto power over anything nobody's tasted before, and nobody's betting the week's budget on something no one's tasted either.

If you want to poke around the kitchen yourself, my implementation is on GitHub, with each of the four layers in its own folder.

Naman-GG/BayOAgentView the implementation on GitHub

References

Gupta, R., Hartford, J., & Liu, B. (2025). LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? Findings of the Association for Computational Linguistics: EMNLP 2025, 15482–15510. arxiv.org/abs/2509.21403