Limitations of version 1

Version 1 is the original pointwise scorer: one kinase and one ligand per row, returning a number on the pIC50 scale, with comparisons made afterwards by subtracting two independent scores. It remains published in full. This page states what it does not do.

Looking for the current release? The two version 2 comparator models have their own limitations page, and their confidence scales are not interchangeable with version 1's margin or with each other.

Stated plainly

Quote it per target, never in aggregate. External per-target rank correlation runs from 0.09 to 0.83. A minority of targets perform at or below chance. An average across the panel hides that spread, so the tool reports each selected kinase separately rather than pooling them.

Absolute potency does not transfer; ordering does. Only 3.9% of the training data is inactive, because the literature reports exact numbers for compounds that bind and vague ones for compounds that do not. The model therefore predicts roughly two log units too potent on chemistry outside its corpus. Use it to rank, not to estimate an IC50.

It does not yet generalise to kinases it has never seen. Ranking compounds for a kinase outside the 478-member panel measured at 0.48 to 0.52, indistinguishable from chance. You may paste any sequence into the tool, and it will be scored, but anything outside the panel is labelled as unvalidated and should be treated that way.

It cannot answer selectivity. Each target's scores sit on that target's own scale, so subtracting two of them reports the difference between two scales as if it were a preference between two kinases. That question is what the version 2 selectivity model was built for; version 1 should not be used for it.

Two further results worth knowing. On compound pairs whose ranking flips between two kinases, the selectivity question, ligand information alone scores exactly 0.500 while the full model reaches 0.712, which is the clearest evidence that the protein half is carrying real signal. Against that, a simple baseline that looks up a compound’s average potency in the training set scores 0.731 on the shared benchmark versus the model’s 0.723, so on the headline metric the model does not yet beat remembering which compounds are generally potent. Both numbers are in the full results.

Where it works, target by target

Performance is not uniform, so an average across the panel would be misleading. These are held-out ChEMBL compounds, structures the model never saw, scored per target. Agreement is the rank correlation with the measured order; decisive pairs are those whose measured potencies differ by more than tenfold.

Reliable

Order these with confidence. Agreement of 0.6 or better, and the top of the list is strongly enriched in genuinely potent compounds.

Target	Compounds	Agreement with measured order	Decisive pairs correct	Enrichment in top tenth
AURKA	219	0.88	95%	2.7×
MTOR	169	0.77	89%	4.7×
ABL1	496	0.73	85%	3.2×
SRC	370	0.73	88%	5.4×
CDK2	291	0.71	87%	5.5×
CDK1	137	0.71	86%	7.7×
BRAF	216	0.64	85%	4.5×
FGFR1	477	0.64	84%	3.7×
PIM1	161	0.64	80%	4.4×
ERBB2	392	0.61	80%	3.1×

Useful for triage

Clearly better than chance and worth using to prioritise, but the ordering is loose enough that the top of the list should be confirmed rather than trusted.

Target	Compounds	Agreement with measured order	Decisive pairs correct	Enrichment in top tenth
MET	467	0.57	82%	5.1×
JAK3	377	0.52	77%	5.5×
SYK	363	0.51	83%	5.6×
CDK9	441	0.49	78%	4.1×
FLT1	132	0.47	76%	3.9×
MAPK14	138	0.46	75%	3.5×
PIK3CG	449	0.43	72%	3.8×
TYK2	487	0.42	71%	4.7×

Do not rely on these

Either the ordering is barely better than chance, or the top of the list is not enriched enough to act on. A target whose top tenth holds no more potent compounds than a random draw is not usable for screening however well the rest of the list is sorted. They are named rather than folded into an average.

Target	Compounds	Agreement with measured order	Decisive pairs correct	Enrichment in top tenth
AKT1	371	0.75	85%	1.1×
PIK3CA	364	0.55	76%	1.7×
CDK4	213	0.48	81%	1.0×
KDR	272	0.48	75%	1.1×
FLT3	465	0.39	68%	4.2×
BTK	397	0.39	70%	2.2×
PDGFRB	393	0.36	76%	0.0×
MAPK1	498	0.33	67%	1.0×
JAK2	255	0.31	65%	4.1×
GSK3B	318	0.31	65%	3.1×
EGFR	305	0.26	63%	2.7×
PIK3CD	475	0.14	59%	2.7×

Full per-target results for all 437 evaluated kinases, including those that perform at or below chance, are in the model results report.

How version 1 differs from version 2

	Version 1	Version 2 potency	Version 2 selectivity
What one row holds	one kinase, one ligand	ligand, sequence, ligand	sequence, ligand, sequence
What it returns	a score on the target's own scale	probability that ligand A is more potent	probability that protein A binds more potently
How two things are compared	a wrapper subtracts two independent scores	the forest is trained on the comparison	the forest is trained on the comparison
Answers selectivity?	no: scales are not comparable across targets	no: one target at a time	yes, this is what it is for
Confidence signal	margin, a monotone transform, not calibrated	prediction strength, 0.5 to 1.0	prediction strength, 0.5 to 1.0

Version 2 is a different architecture, not a retrain, and its numbers are not comparable to version 1's line by line. See the version 2 overview.