All three models
Limitations
Everything here is measured, not estimated, and comes from the model reports.
Each model was evaluated on its own test set, so the numbers are not
interchangeable between tabs. A confidence of 0.85 does not mean the same
thing on any two of them.
None of the three predicts potency.
The two v2 models are classifiers: they answer a comparison and return a probability.
The v1 ranker is a regressor whose output is on a pIC50 scale but is
not calibrated, so it orders compounds and nothing more. None of them returns a
pIC50, a Kd or an affinity you can quote. A compound can be ranked first and still be
inactive on everything you asked about, the question was which is better, not
whether any of them works.
Accuracy by confidence band
Measured on the held-out ChEMBL test set. The right-hand column is what share of
all comparisons landed in that band, and it is the part most people miss.
| Confidence | Right this often | Share of comparisons |
|---|
| 0.90 – 1.00 | 87.2% | 3.0% |
| 0.80 – 0.90 | 85.0% | 5.3% |
| 0.70 – 0.80 | 83.2% | 12.2% |
| 0.60 – 0.70 | 74.6% | 26.9% |
| 0.50 – 0.60 | 58.4% | 52.7% |
Half of all comparisons land in the bottom band.
On the test set 52.7% of predictions came out between 0.5 and 0.6, where the model
is right 58.4% of the time. Expect a run to contain calls that are barely better
than a coin flip, and let the confidence decide which parts of the ordering you act
on. The report names 0.70 as the useful operating point: a fifth of
comparisons answered at 0.842 accuracy against a 0.680 baseline.
Confidence is not a calibrated probability and does not transfer between
distributions: 0.70 is right about 99% of the time on the training distribution and
about 84% of the time on held-out ChEMBL. Use it to rank and to threshold. Never
present it to someone as a percentage chance that a particular row is correct.
Pasted sequences
A pasted sequence is unvalidated.
This model was trained on 733 distinct targets. Of the 700 with more than one
ligand, 695 reached the model. The
published accuracy figures are measured across 477 targets on the ChEMBL test set.
A sequence outside the training set will still be encoded and scored, but nothing
in the report speaks to how well it does there, and results are flagged.
A pasted sequence that matches one the model already ships is resolved by lookup
rather than re-encoded, so it is exactly as in-distribution as choosing that kinase
from the list. Capitalisation and line wrapping do not matter. Paste the
full-length UniProt sequence, not a kinase-domain slice, every shipped
vector was built from the complete sequence, so a domain-only fragment is encoded
differently from anything the model has seen.
Where accuracy falls
| Situation | Accuracy |
|---|
| Neither ligand new to the Knowledgebase | 0.726 |
| One ligand new | 0.673 |
| Both ligands new | 0.582 |
| Overall, held-out ChEMBL | 0.690 |
Accuracy falls as ligand novelty rises, which is the opposite of what a screening
campaign wants: the compounds you most need an answer about are the ones the model
is weakest on. When both compounds are new to the Knowledgebase the model scores
0.582, over 265,128 comparisons, close enough to a coin flip that a
ranking of entirely novel chemistry should not be acted on without confirmation.
That is the screening case, and it is the weakest number on this page. Two compounds that are close analogues will usually come back near
0.50, and that is the model behaving correctly, similar structures often do
not differ much in potency, so it declines to call rather than guessing.
Scope
- Only the 500 kinases shipped with the model can be chosen from the list.
- Trained on the Eidogen-Sertanty Kinase Knowledgebase only, no ChEMBL, no
BindingDB. ChEMBL was used exclusively as an unseen test set.
- The ranking is relative to the compounds you supplied. Change the rivals
and the scores change; they are not a property of a compound on its own.
- Every figure quoted here is single-order scoring. The deployed service averages
both ligand orders and measures 0.6928 against the report's 0.690, so these
numbers are a conservative floor.
Accuracy by confidence band
Measured on 3.1 million unseen ChEMBL comparisons. These are this model's own
numbers and are markedly better than the potency model's. Do not carry
either set across.
| Confidence | Right this often |
|---|
| 0.90 – 1.00 | 99.3% |
| 0.80 – 0.90 | 96.4% |
| 0.70 – 0.80 | 88.4% |
| 0.60 – 0.70 | 74.4% |
| 0.50 – 0.60 | 57.9% |
Answering every comparison gives 75.3%. Acting only at 0.70 and above covers 37%
of comparisons at 92.3%; at 0.80 and above, 16% at 97.2%.
What this model does not tell you
A preferred side is not activity.
The two probabilities always sum to 1, so one kinase is always preferred: including for a compound that binds neither. This model says which of two proteins
binds a compound more tightly, never whether either binds it at all.
It also gives no selectivity ratio. There is no fold-selectivity here and
none can be derived: the output is a probability about an ordering, not a
difference between two affinities.
Pasted sequences, and mutants
Pasting a sequence works. Mutants are not a special case, the model
was trained on thousands of wild-type-against-mutant comparisons, and many mutant
sequences ship with it. Paste one of those, in any capitalisation or line wrapping,
and it is matched to the stored sequence rather than re-encoded, which makes it
every bit as in-distribution as picking a kinase from the list. Anything genuinely
new is encoded on the spot with the same ESM2 recipe the shipped vectors were built
with, and a single point mutation is a small move in that same space.
What changes for a sequence the model has not seen is only the accuracy claim.
The figures above were measured between sequences it trained on, and because no
comparison was withheld, wild-type-against-its-own-mutant has no measured
accuracy and none can be quoted. The run is still worth doing; it just comes
without a number attached to how often it is right.
Paste the full-length sequence, not just the kinase domain. Every shipped
vector was built from the complete UniProt sequence, so a domain-only slice is
encoded differently from everything the model has seen and is the one paste that
quietly moves the answer. Pasting into a side overrides the kinase picked above it.
Scope
- Covers 497 kinases. Three genes in the potency list are absent here:
DCK and GALK1 have no second sequence to compare against, and
TSSK1A is byte-identical to TSSK1B. The two lists must never be merged, a gene missing here cannot be scored by this model.
- Trained on 4,340,117 KKB comparisons with nothing held back, which is why the
mutant accuracy above cannot exist.
- Each row is an independent question about one compound. The ordering on the
results page is by confidence, for readability; it is not a ranking the model
produced across compounds.
This model is not built like the other two
The v1 ranker is a pointwise regressor, not a pairwise classifier. It is
fitted on individual pIC50 measurements and scores one compound against one kinase
at a time; a ranking is produced by scoring each compound separately and sorting.
Ranking a library of n compounds therefore costs n passes, not
n². It never sees two compounds together, and it never sees a
three-dimensional structure, the input is the amino-acid sequence and the
SMILES, nothing else.
The score looks like a pIC50 and must not be read as one.
It is on a pIC50 scale but is not calibrated. It orders compounds within one
target. Use the position, not the number, and never quote it as a predicted potency.
Accuracy by confidence
Measured on the held-out half of the Kinase Knowledgebase. Confidence here is the
margin, the gap between two compounds' scores. Acting only on the
widest margins is what makes the model useful.
| Calls you act on | Right this often | Pairs |
|---|
| Widest 10% of margins | 98.8% | 304,048 |
| Widest 25% | 96.5% | 760,120 |
| Widest 50% | 91.6% | 1,520,240 |
| Widest 75% | 85.4% | 2,280,360 |
| Every pair | 78.5% | 3,040,480 |
Pooled accuracy is 78.5% over 3,040,480 held-out pairs across 437 kinases
(MCC 0.570). On pairs whose measured potencies differ by more than tenfold it is
90.0%; on pairs closer than half a log it is 57.8%, barely above a coin
flip, and that is the model behaving correctly, because differences that small
are inside the range on which published assays disagree with each other.
The margin is a monotone transform of the score difference and is
not a calibrated probability. Use it to threshold, never to tell someone the
chance that a particular row is right.
Check the target before you read the ranking
This is the step that matters most, and it is not optional.
Performance varies far more between targets than between any two models tried on
this problem. There is no aggregate number that substitutes for reading the figure
for your own kinase. Against an outside database, per-target rank agreement ranges
from 0.09 to 0.83, some targets are at or below chance while the
pooled figure still reads 78.5%.
The kinase picker on the ranking page carries a measured
tier for each target: 70% or better (use it here), 60–70%
(triage only, confirm the top of the list), below 60% (close to a coin flip,
do not rely on it), and grey for targets with too few held-out pairs to measure.
Grey means unknown, which is not the same as reliable. A target scored on a
few dozen pairs carries only a handful of distinct compounds, so read the pair count
alongside the accuracy.
Held-out data flatters it; outside data is what to plan against
| Ranking a whole library | Same collection, withheld | Different database, never seen |
|---|
| Compounds ranked | 49,211 | 10,108 |
| Agreement with the measured order | 0.79 | 0.50 |
| Decisive pairs (>10×) called correctly | 90% | 77% |
| Top-decile enrichment | 5.5× | 3.7× |
Roughly a third of the agreement does not survive the move to an outside source.
Plan against the right-hand column. Even that is a retrospective benchmark
rather than an independent replication: only exact compound matches were removed
from it, not close analogues, so overlap between near-duplicate series has never
been quantified. On outside chemistry the widest-margin decile falls to about
88% from 98.8%.
What it does not tell you
It does not measure selectivity.
Each target is scored on its own scale, and an inactive compound does not score the
same on every target, so the gap between two targets' scores is
not a
selectivity estimate. Compare compounds within a single target only. For a
selectivity question use the
v2 selectivity model,
which was built for it.
- Mutants are trained on but not validated. The held-out split contains
zero mutant rows, so nothing measures how well the model predicts for a
point mutant. That is a separate claim and it has not been tested. Variants that
change length, such as exon-19 deletions and ITDs, cannot be represented at
all, and the model has never seen them.
- Kinases outside the panel are not supported. Ranking for a kinase the
model has never seen measured at 0.48–0.52, which is chance. A
pasted sequence outside the 478-kinase panel is scored but flagged, and results
carry the flag in the API response as well as on the page.
- Compounds that already have data do not need it. Where a measurement
exists against a related kinase, use the measurement; it is direct evidence. A simple lookup of a compound's average potency scores 73.1% where it
is defined, though it is undefined on nearly half of all pairs and can say
nothing about selectivity.
- Records reporting only a limit (“weaker than 10 µM”)
were excluded from both training and evaluation. They carry most of the evidence
that a compound fails to bind, so the model has been shown far fewer
non-binders than the corpus actually contains.
- It is a triage tool that runs before docking or crystallography, not a
replacement for either, and not a substitute for measurement.
Scope, and the two arms
- Trained on the Eidogen-Sertanty Kinase Knowledgebase, release Q2-2026, human
enzyme assays only. The validated arm was fitted on 425,524 exact-value
measurements.
- Compounds were held out by Bemis-Murcko scaffold within each kinase. No
protein and no family was ever withheld, so the held-out figures measure
generalisation to new chemistry against known kinases, not to new
kinases, and not to chemistry unlike the corpus.
- Every run scores two arms. Validated is the primary answer and is
the one every accuracy figure on this page refers to; reloading the shipped file
reproduced the published benchmark to within 0.000000 across 437 kinases.
Frontier is fitted on substantially more data with nothing held back, so
it has no measured accuracy and none is possible. It is offered for use,
not as a claim.
- Expect the two to differ. Agreement is corroboration; disagreement means the
extra data changed the answer, and that compound is worth a closer look rather
than a decision. Compare their rank positions, never their scores: the arms are not on a shared calibrated scale.