AI for science is about making problems verifiable

I can't remember the last time an LLM failed to solve a verifiable problem for me.¹ A verifiable problem is one where the model can get its own feedback: whether the code compiles, whether the tests pass, whether the number goes up, or whether a proof of a very hard problem is valid. One method of post-training LLMs is reinforcement learning with verifiable rewards, which tunes the models to be good at solving these closed-loop problems. These kinds of techniques have led to breakthroughs like solving a Millennium Prize problem by showing that the Navier-Stokes equations can develop a singularity in finite time, backed by an enormous Lean-verified proof.

It's no longer surprising to hand off a well-defined coding problem to an LLM, work on something else, and come back to the right answer.² Anyone who isn't using these tools and is still impressed by demos needs to sit down with them and recalibrate. A number of my colleagues have been using LLMs successfully to rescue hard-to-reproduce code or abandonware from published papers. Others have mentioned how quickly their undergraduate and graduate students have been able to produce meaningful results. People are also using them to reverse engineer just about any software, including Adobe's Creative Suite and video games, taking the verification loop all the way down to decompiling the binary with tools like Reverse Engineer Anything (REA).

Recently, I've been taking computational science papers that are underspecified in their methods, but well specified in the problems they solve, and handing them to an LLM as a challenge to reproduce the work. It writes all the code to do this, including some finite element method (FEM) codes, which I have virtually no experience writing myself. I can't judge whether the code is well structured or whether it would be easy to adapt to a second problem, but this was effectively throwaway code, and it's easy to see that it does get the right answer.

What does this mean for the future of science?

I think this has implications for all of science, work, and humanity, but materials science is the only field I feel qualified to write about. Current LLMs are bad at true novelty outside the bounds of what they are trained on. The Navier-Stokes result is groundbreaking, but it worked because humans had mathematized the question into a verifiable problem. In my experience, an LLM asked to predict something like a new composition can't come up with a credible one that isn't a trivial modification of something that already exists, if not directly laundered from the literature without attribution. Other people I've talked to in different fields seem to have similar experiences. The models might be able to reason using some world knowledge, but they will still have gaps and blind spots because they don't have the expert knowledge to really scope the entire problem.

Sean Goedecke recently wrote that we're in the centaur age for software, where humans and AI combined are better than either alone. LLMs can work on and solve verifiable problems faster than humans can review them, so taste (knowing which problems are worth spending human effort on) is increasingly important. Knowing the right question to ask was always valuable, but now its value is amplified. It seems to me that the way to get the most out of AI for science right now is to leverage human expertise to turn scientific hypotheses into verifiable problems, which has always been the core of the scientific method. It's the art of science, and it's a really exciting time to be able to exercise that creativity.

So many hypotheses have been limited by compute and by the ability of domain scientists to write code to test them computationally. AI is already so good at accelerating code and helping domain scientists translate their ideas into code that I think we're in the middle of a step change in the rate of progress and breakthroughs. There's so much to do that I'm not at all worried about AI completely solving computational materials science, but the nature of the work is changing rapidly. I don't think we're very far from individuals and small teams being able to dump solutions to hundreds of problems that will take humans a long time to understand, as the frontier labs have done for math. Understanding still matters because that's how we do our own compaction of ideas and use creativity to figure out the right questions to ask.

I think the next major frontier in materials science will be bringing this to the real world via experiments. There's a lot of investment in autonomous labs right now aimed at exactly that, but we're still far from being able to close the loop from synthesis through processing to measuring and characterizing the real properties of interest for things like new alloys. We're going to have to create new ways to measure the properties we care about via proxies (in the same way that we infer strength from microhardness), and to balance trade-offs between multiple properties by measuring them on the same material. Only then can we really let models go out and hill climb, combining their ability to brute-force problems with existing computational techniques for cutting down problem spaces, like Bayesian optimization.

While there are still a lot of ethical issues with using LLMs, and it's unclear whether the current trajectory is sustainable, it is clear that AI is here to stay. I still have concerns about how we train people to be creative and to understand things when so much understanding comes from putting in reps,³ learning from being wrong, and building intuition for fundamentals by getting your hands dirty, whether in code or in the lab. I think it is so important for mentors to hold their students, postdocs, and mentees accountable for developing understanding and not just producing results.⁴

It's hard to predict exactly how long the centaur age for science will last. I'm optimistic that the golden age of math and science is coming, as long as we prioritize developing expertise over producing results.