Limitations

Both models

Everything here is measured, not estimated, and comes from the model reports. Each model was evaluated on its own test set, so the numbers are not interchangeable between tabs. A confidence of 0.85 does not mean the same thing on any two of them.

Neither model predicts potency. Both are classifiers: they answer a comparison with an ordering and a strength. Neither returns a pIC50, a Kd or an affinity you can quote. A compound can be ranked first and still be inactive on everything you asked about, the question was which is better, not whether any of them works.

Accuracy by confidence band

Measured on the held-out ChEMBL test set. The right-hand column is what share of all comparisons landed in that band, and it is the part most people miss.

Confidence	Right this often	Share of comparisons
0.90 – 1.00	0.89	0.9%
0.80 – 0.90	0.90	3.4%
0.70 – 0.80	0.87	11.0%
0.60 – 0.70	0.78	27.6%
0.50 – 0.60	0.60	57.1%

The top band is not the best band, and that is measured rather than a slip. Only 0.9% of comparisons reach 0.90, some 16,000 of them, so that row carries a standard error of about 0.24 points and sits roughly 2.6 of them below the row beneath it. Read the top two bands as the same accuracy on very different volumes.

Half of all comparisons land in the bottom band. On the test set 57.1% of predictions came out between 0.5 and 0.6, where the model is right 0.60 of the time. Expect a run to contain calls that are barely better than a coin flip, and let the confidence decide which parts of the ordering you act on. The report names 0.70 as the useful operating point: 15.3% of comparisons answered at 0.88 accuracy, against 0.69 answering everything.

Confidence is not a calibrated probability and does not transfer between distributions: 0.70 is right about 0.99 of the time on the training distribution and about 0.88 of the time on held-out ChEMBL. Use it to rank and to threshold. Never present it to someone as a percentage chance that a particular row is correct.

Pasted sequences

A pasted sequence is unvalidated. This model was trained on 733 distinct targets. Of the 700 with more than one ligand, 695 reached the model. The published accuracy figures are measured across 477 targets on the ChEMBL test set. A sequence outside the training set will still be encoded and scored, but nothing in the report speaks to how well it does there, and results are flagged.

A pasted sequence that matches one the model already ships is resolved by lookup rather than re-encoded, so it is exactly as in-distribution as choosing that kinase from the list. Capitalization and line wrapping do not matter. Paste the full-length UniProt sequence, not a kinase-domain slice, every shipped vector was built from the complete sequence, so a domain-only fragment is encoded differently from anything the model has seen.

Where accuracy falls

Situation	Accuracy
Neither ligand new to the Knowledgebase	0.73
One ligand new	0.67
Both ligands new	0.58
Overall, held-out ChEMBL	0.69

Accuracy falls as ligand novelty rises, which is the opposite of what a screening campaign wants: the compounds you most need an answer about are the ones the model is weakest on. When both compounds are new to the Knowledgebase the model scores 0.58, over 265,128 comparisons, close enough to a coin flip that a ranking of entirely novel chemistry should not be acted on without confirmation. That is the screening case, and it is the weakest number on this page.

What “novel” means here

The table above asks only whether a compound appears in the Knowledgebase at all. A finer measure bins each test compound by its maximum Morgan Tanimoto similarity to the compounds the model was actually fitted on: novel below 0.35, distant 0.35 to 0.50, related 0.50 to 0.70, familiar 0.70 to 1.00, and identity at exactly 1.00. A potency comparison carries two compounds, so it is assigned the larger of its two similarities and only counts as novel when both are.

Tier of the test compound	Selectivity accuracy
Novel, under 0.35	0.62
Distant, 0.35 to 0.50	0.62
Related, 0.50 to 0.70	0.72
Familiar, 0.70 to 1.00	0.63
Fingerprint identity, exactly 1.00	0.76
All comparisons	0.75

A potency comparison carries two compounds, so it is placed by both of their tiers. The diagonal, where the two sit in the same tier:

Both compounds in tier	Potency accuracy
Novel	0.58
Distant	0.54
Related	0.57
Familiar	0.63
Fingerprint identity	0.72

Potency behaves differently from selectivity here. Selectivity reaches 0.72 in the related tier, while potency with both compounds related is 0.57. Potency only climbs once one side is a compound the model has seen: 0.66 for related against identity, 0.69 for familiar against identity, 0.72 for identity against identity.

Two cautions on reading that table. It is not monotonic: the related tier outscores the familiar tier by enough that sampling noise does not explain it, and we do not offer an explanation. And a Tanimoto cutoff is a two-dimensional convenience, not a statement about biology. Two compounds at 0.7 similarity can behave entirely differently in a protein, which reads three-dimensional shape, charge and contacts rather than a fingerprint. Treat the tiers as a rough index of how much the model has seen, not as a measure of how new a molecule really is. Two compounds that are close analogs will usually come back near 0.50, and that is the model behaving correctly, similar structures often do not differ much in potency, so it declines to call rather than guessing.

Scope

Only the 500 kinases shipped with the model can be chosen from the list.
Trained on the Eidogen-Sertanty Kinase Knowledgebase only, no ChEMBL, no BindingDB. ChEMBL was used exclusively as an unseen test set.
The ranking is relative to the compounds you supplied. Change the rivals and the scores change; they are not a property of a compound on its own.
Every figure quoted here is single-order scoring. The deployed service averages both ligand orders and measures marginally above the report's 0.69, so these numbers are a conservative floor.

Accuracy by confidence band

Measured on 3.1 million unseen ChEMBL comparisons. These are this model's own numbers and are markedly better than the potency model's. Do not carry either set across.

Confidence	Right this often
0.90 – 1.00	0.99
0.80 – 0.90	0.96
0.70 – 0.80	0.88
0.60 – 0.70	0.74
0.50 – 0.60	0.58

Answering every comparison gives 0.75. Acting only at 0.70 and above covers 37% of comparisons at 0.92; at 0.80 and above, 16% at 0.97.

What this model does not tell you

A preferred side is not activity. Every comparison names a winner, so one kinase is always preferred: including for a compound that binds neither. This model says which of two proteins a compound prefers, never whether it engages either at all.

It also gives no selectivity ratio. There is no fold-selectivity here and none can be derived: the output is an ordering with a strength, not a difference between two affinities.

Pasted sequences, and mutants

Pasting a sequence works. Mutants are not a special case, the model was trained on thousands of wild-type-against-mutant comparisons, and many mutant sequences ship with it. Paste one of those, in any capitalization or line wrapping, and it is matched to the stored sequence rather than re-encoded, which makes it every bit as in-distribution as picking a kinase from the list. Anything genuinely new is encoded on the spot with the same ESM2 recipe the shipped vectors were built with, and a single point mutation is a small move in that same space.

What changes for a sequence the model has not seen is only the accuracy claim. The figures above were measured between sequences it trained on, and because no comparison was withheld, wild-type-against-its-own-mutant has no measured accuracy and none can be quoted. The run is still worth doing; it just comes without a number attached to how often it is right.

Paste the full-length sequence, not just the kinase domain. Every shipped vector was built from the complete UniProt sequence, so a domain-only slice is encoded differently from everything the model has seen and is the one paste that quietly moves the answer. Pasting into a side overrides the kinase picked above it.

Scope

Covers 497 kinases. Three genes in the potency list are absent here: DCK and GALK1 have no second sequence to compare against, and TSSK1A is byte-identical to TSSK1B. The two lists must never be merged, a gene missing here cannot be scored by this model.
Trained on 4,340,117 KKB comparisons with nothing held back, which is why the mutant accuracy above cannot exist.
Each row is an independent question about one compound. The ordering on the results page is by confidence, for readability; it is not a ranking the model produced across compounds.

Full method, training set, test set and every figure quoted here: potency report · selectivity report.