I’ve written before that a protein is really a sentence — binding pockets act like nouns, the folding rules act like syntax, and changing one amino acid can change the whole meaning the way changing one word can. If that idea is new to you, start there. I want to pick up exactly where it leaves off, with a question I gave a whole conference talk about: if proteins have a language, can we teach a machine to speak it?
It turns out we can. And the consequences are strange and wonderful.
From translating French to translating proteins
Here’s a thing about large language models that I find genuinely funny: the model doesn’t know what language is. When you feed a model English and Spanish and Chinese, it learns to translate between them — but before any of that text reaches the processor, it’s been chopped into tokens and turned into numbers. From the computer’s side, there’s no French. There’s just patterns in ones and zeros.
Which means the machinery doesn’t care whether the “language” is French or the sequence of amino acids in a protein. Feed the same kind of model a few hundred million protein sequences and it starts to learn their grammar — which letters tend to follow which, which arrangements fold into something stable, where a binding pocket is likely to sit. Models trained this way have a name: protein language models.
A model that can write proteins, not just read them
Reading is the first half. The part that stops me in my tracks is the second half.
Because a generative language model doesn’t just classify — it continues. You give it the start of a sentence and it writes what it thinks comes next. Do the same thing with a protein, and the model writes what it thinks the protein would evolve into. You hand it a protein you care about, and it hands you back a family of new proteins with slightly different properties — variations nature hasn’t gotten around to trying yet.
Scientists call this directed evolution, and models in the ESM family are one of the tools people use to do it. The first working version my collaborators and I built paired two models: one generating candidate proteins, one predicting how well each would fit a target molecule. Optimize the pair together and you get proteins that look like they’d do the job — which you then send to a wet lab to find out if they actually do. (We’ve since moved past that first design to a setup where several protein models generate in parallel and feed a docking-and-verification step, but the core idea is the same: read the language, then write in it, on purpose.)
Where do you find a textbook big enough?
A model is only as good as what you tune it on. So: if you wanted to teach a protein language model, where would you look for enough varied, hard-won biology to learn from?
The answer, I’ll admit I’m biased toward, is the ocean.
Picture it from the model’s side. If you’d never heard of an ocean and you were training protein models, you’d find yourself wishing for a gigantic body of water spread across wildly different conditions, full of microorganisms constantly adapting to survive. You’d wish, in other words, for exactly the thing already lapping at the beach. Down at the volcanic vents, life runs on chemistry from the seafloor instead of sunlight — a different biochemistry entirely, so a model given a taste of that place writes proteins you’d never see anywhere else. Up in the intertidal zone, where the temperature and salinity and acidity lurch around with every tide, you get proteins shaped by volatility — a more improvisational kind of solution. Different neighborhoods of the ocean speak in different dialects, and each one teaches the model something new.
This is how you start designing medicine
Two things I most want these models for. The first is medicine.
Take Ozempic — the peptide drug a lot of people are talking about. It’s a small one, and honestly we’re still mapping out everything it does across the body, which is the whole point of what biologists call the interactome: a molecule on its own tells you very little; what matters is how it interacts with every other protein and metabolite around it. A protein language model lets you take something like Ozempic and ask for an analog — a close cousin — then push that candidate into a discovery process to see if it’s interesting. Scale that up and you can imagine agent systems that model the entire path from genome to protein to effect.
I’ll say plainly that this is where I think it’s headed, not where it already is. As an AI scientist, my honest finish line — the moment I’d say a system had reached something like human-level — is when it can run a full drug-discovery process end to end: a model reading the literature for a hypothesis, handing it to a protein model that generates a candidate, mutating that into a space of variants, and running the whole batch through docking and verification. We’re not there. But you can see the shape of it from here.
The second use is quieter and, to me, just as lovely: proteins are far better chemists than we are. There’s a protein in shellfish that makes a brilliant, food-safe blue. It used to mean farming shellfish and crushing shells — expensive, and rough on anyone with an allergy. So a company copied the protein, edited it into yeast, and now they brew the color the way you’d brew beer. I’d bet that within a decade it’ll be more likely than not that some link in our supply chain has been touched by that kind of biomanufacturing.
The clock part
All of it — the medicine, the pigments, the dialects — comes from the ocean, which is no surprise, since that’s where our biochemistry came from in the first place. So let me end where the talk ended, on a note that isn’t really technical at all.
We remember the people who let the Library of Alexandria burn as a caution: stewards who lost something whose value they hadn’t finished measuring. I think within a few years we’ll understand that the ocean is a library of Alexandria lapping up on the beach — and the clock is ticking to read it before it’s gone.
Adapted from a talk I gave at the Ai4 conference (2024) on behalf of New Atlantis Labs. Watch the full 19 minutes.