KinaseFoundationModel

Eidogen-Sertanty · live results

A comparator model that rank-orders molecules against a kinase

Give it one of the 478 kinases it has data on, plus two molecules, and it says which one binds more tightly. That single comparison is the primitive operation of virtual screening, so repeating it sorts an entire library in seconds where a docking campaign takes hours, and at no point does the model see a three-dimensional structure.

Two models are released. The Validated model carries every number in this report, measured on 2,965,273 pairs it never saw. The Frontier model uses the identical training method on every measurement we hold, including the data withheld to make that measurement possible, and is therefore untested by construction. Which target you ask about matters more than which model you pick: the reliability tiers below are the part to read before acting on any prediction.

Version 2026-08-09_0926 v1.1 Generated 09 August 2026 at 09:26 Cycles filed 40

What this is

A comparator. Given a kinase and descriptions of two drug-like molecules, it predicts which of the two binds more strongly. No three-dimensional structure, no docked pose and no binding site is required. Because a ranked library is nothing but this comparison repeated, the same model sorts a whole compound collection against a target in one pass. It was trained on human kinase measurements. The source corpus holds 841,187 rows over 500 genes. One third is held out for testing; of the remainder, records giving only a bound rather than a value are set aside and repeat measurements of the same compound against the same kinase are collapsed to their median, leaving 425,524 distinct measurements. The model is fitted on 425,524 of them, all of them.

Which kinases, precisely. The shipped model carries precomputed protein vectors for 478 named kinases over 695 distinct sequences, and those are the targets it will score. Being scoreable is not the same as being supported: the vectors were built for the whole panel, not for the subset the forest actually learned from, so the fit covers only the genes that carry usable measurements. Three exposed targets, NEK10, ADK and SPHK2, contributed no fitted rows; NEK10 is the fundamental case, because every one of its measurements is a limit rather than an exact value. Check a target against the reliability tiers before acting on any prediction for it, and treat a target absent from those tiers as unsupported. Handed a sequence it has never seen it refuses rather than guessing: a confident number derived from the wrong protein is the worst failure this system could produce. Nothing here supports a kinase with no training data.

Model architecture

Architecture clarification, v1.1, 9 August 2026. The released Random Forest never receives ligand A and ligand B jointly. It evaluates (kinase, ligand A) and (kinase, ligand B) as separate feature rows, and the comparison wrapper reports sA minus sB. The figure below has been redrawn to show the two independent runs explicitly. No result, method or number in this document changed: the pointwise design was already stated in Methods and is what produced every figure quoted here.

The question the model answers, drawn from the project's own data. The protein is the ABL1 kinase domain rendered from crystal structure PDB 1IEP, with imatinib in the ATP cleft and the twenty-five residues within 4.5 angstrom of it picked out in orange; the model itself sees only the amino-acid sequence, embedded with ESM2 and pooled to a single 480-number vector. The two compounds are real ABL1 inhibitors from the training table with their measured potencies, each described by a Morgan count fingerprint and fourteen physicochemical descriptors. The same fitted random forest is run twice, once on the kinase with compound A and once on the kinase with compound B, so the two compounds never share a feature row; a comparison wrapper then subtracts the two scores, which makes the comparison exactly antisymmetric.
The question the model answers, drawn from the project's own data. The protein is the ABL1 kinase domain rendered from crystal structure PDB 1IEP, with imatinib in the ATP cleft and the twenty-five residues within 4.5 angstrom of it picked out in orange; the model itself sees only the amino-acid sequence, embedded with ESM2 and pooled to a single 480-number vector. The two compounds are real ABL1 inhibitors from the training table with their measured potencies, each described by a Morgan count fingerprint and fourteen physicochemical descriptors. The same fitted random forest is run twice, once on the kinase with compound A and once on the kinase with compound B, so the two compounds never share a feature row; a comparison wrapper then subtracts the two scores, which makes the comparison exactly antisymmetric.

Screening performance: sorting compounds by potency

Two compounds, one kinase, which binds harder. Coin flip = 50%.

78.5%
Correct on 3,040,480 held-out ligand pairs across 437 kinases. Matthews correlation 0.570.
90.0%
When the two ligands genuinely differ by more than tenfold in potency, on 1,617,167 pairs. MCC 0.801. This is the call a chemist actually makes.
98.8%
On the tenth of calls carrying the largest model margins, 304,048 pairs, held-out KKB compounds. The margin is a monotone transform of the score difference, not a calibrated probability: larger margins are associated with higher accuracy, which is what makes a threshold useful.

Act on the most confident calls and this is what you get

Every prediction carries a confidence. Rank the calls by it and keep only the top slice: accuracy rises steeply as the slice narrows, so the operating point is yours to choose. The finest slice shown is the top tenth, which is the finest the confidence table resolves.

calls acted onpairsaccuracy
top 10%304,04898.8%
top 25%760,12096.5%
top 50%1,520,24091.6%
top 75%2,280,36085.4%
every pair3,040,48078.5%

Could you skip the model?

Two cheap shortcuts a screening group could use instead. Neither is an input to the model and neither changes what the model is given; they are rival predictors, and the model has to be better than both to be worth running.

methodpairsaccuracy
Rank by molecular size alone2,876,16058.4%
Shortcut 2: look up what this compound did against some other kinase and assume it behaves the same here1,504,96073.1%
This model, on those same pairs1,504,96078.5%
This model, where no prior measurement exists1,535,52078.5%

Shortcut 2 is the serious rival. Potency is correlated across kinases, so knowing what a compound did elsewhere predicts a lot, 73.1 percent, without any model at all. The model beats it on the same pairs, 76.8 percent, and is the only option on the 1.47 million pairs where no such prior measurement exists. Neither shortcut can address selectivity: both give one answer per compound regardless of which kinase you ask about.

Evaluation is capped at 20,000 pairs per kinase so that a few very large targets cannot dominate; 119 kinases hit that cap. Reported, never silent.

Proof the model actually reads the kinase

A fair worry about any model like this: maybe it never really uses the protein, and is just deciding which of the two compounds looks better in general. Here is the test that settles it.

The test

Find compound pairs that were measured against two different kinases, and keep the ones where the answer switches:

Compound A beats compound B on kinase 1.
On kinase 2, compound B beats compound A.
Same two compounds. Opposite answer. The only thing that changed is which kinase you asked about.

A model that ignores the protein produces one answer for that pair. It is therefore right on one kinase and wrong on the other, every time. It scores 50 percent because arithmetic forces it to, not because it is guessing. To do better than 50 percent here, a model has no choice but to use the kinase.

The result

modelgets these switched cases right
This model71.2%
The identical model with the protein taken away50.0%

30,650 switched cases, 5,264 compounds, 377 kinases. Both rows are the same random forest on the same pairs. The only difference is whether the protein vector changes from one kinase to the next.

The control lands exactly on 50.0 percent, as it must. That is how we know the test is built correctly and that the margin above it is real.
And where the answer does not switch, the protein barely matters. On pairs that keep the same order against both kinases, the full model scores 89.1 percent and the protein-free version 89.0 percent. That is the expected result and it is worth stating: the kinase only earns its place where the answer depends on it. Those are the cases behind off-target activity, resistance mutants and repurposing, and they are the ones a compound-quality score can never get right.

Package identity. Every number in this document was produced by the forest shipped as kfm_ranker_v1_final, SHA-256 b9a1a01be515c90c…. Reloading that exact file and rescoring reproduces the internal benchmark with a maximum per-kinase difference of 0.000000.

Not to be confused with

This has nothing to do with swapping the two compounds at the input. If you feed the same model compound B first and compound A second, the answer flips exactly, every time, by construction. That is a property of how the model is built and it is asserted in the package self-test. The test above is about something else entirely: the same pair giving genuinely opposite answers on two different proteins.

Ranking a whole library, checked against the bench

Screening needs a ranked list, not a single comparison. We gave the model a target and a whole set of compounds, had it sort them from most to least potent, and checked that order against what the assays actually measured.

We did this twice. First on compounds from our own collection that were held back from training. Then on compounds from ChEMBL, the public literature database, which come from different laboratories and which the model has never seen. The second is the harder test, and it is what the external numbers below describe. It is a retrospective benchmark on an outside dataset with exact training compounds removed, not an untouched independent replication: this corpus has been used elsewhere in the project, and only exact compound matches were excluded, not close analogs.

same collection, compounds withhelddifferent database, never seen
compounds ranked49,21110,108
pairwise comparisons46,330,1091,909,416
agreement with the measured order0.790.50
pairs called right when potencies differ by more than tenfold90%77%
enrichment in the top tenth of the list5.5x3.7x
The gap between those two columns is the finding. Ranking transfers to a corpus the model has never seen, every one of the 30 targets has a positive correlation, though the weakest of them, PIK3CD at 0.09, is not distinguishable from chance on its own, but roughly a third of the agreement does not survive the move. A holdout drawn from the same collection as the training data is measurably optimistic, and the right number to plan against is the one in the right-hand column.
How the external set was kept clean. It was built years earlier to contain nothing in training at the time, and it did. Training has since grown from 30 kinases to 500, and thousands of those compounds entered it through other targets. Every one of the 13,528 measurements was therefore re-checked against all current training compounds by canonical structure and by InChIKey; 3,420 matched and were removed, leaving 10,108. Without that re-check a quarter of this test would have been compounds the model had already been shown.

Which targets to trust, on unseen compounds

Performance is not uniform and the aggregate hides that. These are the same 30 targets, split by how well the model ordered compounds it had never seen. A target's tier is the first thing to check before acting on a prediction for it.

Reliable

Order these with confidence. Agreement of 0.6 or better, and the top of the list is strongly enriched in genuinely potent compounds.

targetcompoundsagreement with measured orderdecisive pairs correctenrichment in top tenth
AURKA2190.8895%2.7x
MTOR1690.7789%4.7x
ABL14960.7385%3.2x
SRC3700.7388%5.4x
CDK22910.7187%5.5x
CDK11370.7186%7.7x
BRAF2160.6485%4.5x
FGFR14770.6484%3.7x
PIM11610.6480%4.4x
ERBB23920.6180%3.1x

Useful for triage

Clearly better than chance and worth using to prioritize, but the ordering is loose enough that the top of the list should be confirmed rather than trusted.

targetcompoundsagreement with measured orderdecisive pairs correctenrichment in top tenth
MET4670.5782%5.1x
JAK33770.5277%5.5x
SYK3630.5183%5.6x
CDK94410.4978%4.1x
FLT11320.4776%3.9x
MAPK141380.4675%3.5x
PIK3CG4490.4372%3.8x
TYK24870.4271%4.7x

Do not rely on these

Either the ordering is barely better than chance, or the top of the list is not enriched enough to act on, a target whose top tenth holds no more potent compounds than a random draw is not usable for screening however well the rest of the list is sorted. They are named rather than folded into an average.

targetcompoundsagreement with measured orderdecisive pairs correctenrichment in top tenth
AKT13710.7585%1.1x
PIK3CA3640.5576%1.7x
CDK42130.4881%1.0x
KDR2720.4875%1.1x
FLT34650.3968%4.2x
BTK3970.3970%2.2x
PDGFRB3930.3676%0.0x
MAPK14980.3367%1.0x
JAK22550.3165%4.1x
GSK3B3180.3165%3.1x
EGFR3050.2663%2.7x
PIK3CD4750.1459%2.7x
Every target ordered independently, on unseen compounds. The spread across targets is wider than any single summary number conveys.
Every target ordered independently, on unseen compounds. The spread across targets is wider than any single summary number conveys.

Which targets is this model reliable on?

The headline describes no individual target. Across 372 kinases with enough held-out pairs to measure, 289 reach 70% pairwise accuracy or better, 63 sit between 60% and 70%, and 20 fall below 60% against a 50% coin flip. Use it where it is strong; the rows below say where that is.

Arm: c56_validated_rebuild. Sorted by the first metric column, best first, pairwise accuracy in ranker mode, rank correlation otherwise. Test ligands is the number of held-out compounds that target was scored on. Read it before the accuracy: a target scored on 50 pairs carries only about a dozen distinct ligands, and a high number there is not the same evidence as the same number over thousands of pairs.

kinasepairwise accuracyMCCtest pairstest ligands
GRK393.2%0.8657415
EPHB192.3%0.8506512
STK3991.2%0.8238015
PRKAA290.9%0.8186612
CAMK2D90.5%0.81019753279
ULK290.2%0.80510215
OXSR189.3%0.7868415
GRK788.9%0.7779014
DCLK188.5%0.77116619
TSSK1B87.8%0.75639329
MST1R87.6%0.752305779
RIPK186.6%0.73119948915
CAMK2A86.5%0.730151656
CSNK1D86.4%0.72719916433
IGF1R86.3%0.72719774694
STK2586.1%0.7208614
NEK785.9%0.7187113
PFKFB285.9%0.7207813
PTK285.7%0.71419609745
FGFR185.3%0.706199741176
IKBKE85.2%0.70519905256
RIOK385.2%0.7065411
MAP3K1185.0%0.70040129
TTK85.0%0.69919934741
MAPKAPK284.9%0.69819952413
PIK3R484.8%0.69619918375
PIM184.8%0.696199291689
RAF184.7%0.69519889834
TTBK184.6%0.6999114
LTK84.6%0.692185162
PKMYT184.4%0.68823122
MAPK984.2%0.68519895402
EPHB284.2%0.68429725
ROCK184.1%0.683199551002
WEE184.1%0.68219903301
GRK184.0%0.67715018
MAP3K1084.0%0.67934327
MTOR84.0%0.679199562222
PRKCE83.9%0.6796420114
ABL283.8%0.67556034
TBK183.7%0.67419865329
FGFR283.6%0.67319903777
CDK483.6%0.672199691374
ALK83.5%0.66919927697
RIOK183.3%0.6667813
AXL83.3%0.66619894468
ABL183.3%0.665199491145
CDK383.2%0.66466637
ROS183.2%0.6644922100
PRKCB83.2%0.66319968215
FGFR383.2%0.66319939910
MAP4K183.1%0.66219310247
PIK3CA83.0%0.660199712066
BRAF82.9%0.658199431455
GSK3A82.9%0.65819960772
CDK282.8%0.656199132211
MARK182.8%0.6578714
NTRK282.7%0.65419920213
MAP4K282.6%0.6538992136
PIK3CB82.6%0.65219956829
SYK82.5%0.650197071497
PAK182.4%0.64810665147
PDGFRB82.3%0.64519961636
JAK282.2%0.645199583500
CSF1R82.2%0.64419922603
EPHA682.1%0.64325223
GRK582.1%0.643316781
ATM82.1%0.6415619107
CLK182.1%0.64219964268
CSNK2A182.0%0.64119959449
STK17B82.0%0.64425023
AKT381.8%0.63719904317
NEK181.8%0.63485842
PKN181.8%0.63618720
MINK181.7%0.6354977102
PRKD381.7%0.63510463147
MAPK881.7%0.63319946635
CLK481.6%0.63219833353
CDC42BPG81.5%0.6326512
NTRK381.5%0.63010342145
EGFR81.4%0.628199572441
TYK281.4%0.628199801259
FLT481.4%0.62819877216
AKT181.3%0.62719957993
PRKCD81.3%0.62519898294
FGR81.2%0.625364186
PRKCA81.2%0.62419964256
MELK81.2%0.62319864252
STK3381.1%0.62322822
JAK181.1%0.622199312951
PRKCQ81.1%0.62219969681
CSNK1E81.1%0.62219974276
MAPK1081.1%0.62219951562
SRC81.0%0.621199641112
PRKD181.0%0.62090043
MAP3K981.0%0.62052633
PIK3CG81.0%0.620199781394
GSK3B81.0%0.619199621383
HIPK180.8%0.61343330
AURKB80.8%0.61719907876
TXK80.8%0.61666237
CAMK2G80.8%0.615388490
EPHA280.8%0.61616295187
MAPK1380.6%0.612109748
AKT280.6%0.61119911329
RPS6KA380.5%0.61019703231
INSR80.4%0.60919805301
TYRO380.4%0.60819895252
IRAK380.4%0.60715318
GRK680.4%0.60556034
TEC80.3%0.60681641
LIMK280.2%0.605436494
DMPK80.2%0.6018114
TNK280.1%0.60312050156
EPHA180.1%0.60329725
LRRK280.1%0.602199701048
KDR80.0%0.601199373026
MAP4K479.9%0.59819703276
NTRK179.9%0.59719962982
DYRK1A79.8%0.59619937660
CDK179.8%0.596199521008
PKN279.7%0.594317881
PRKAA179.7%0.59412487159
PRKDC79.6%0.59219853234
MAPK1279.6%0.591424993
CSNK2A279.6%0.591464397
CDK1979.5%0.590481899
PDK279.5%0.59011422152
PIM279.5%0.590199621117
HCK79.5%0.59012800161
TAOK379.5%0.58945332
FGFR479.4%0.58919950606
SRMS79.3%0.58697845
SIK379.3%0.58719947219
MARK379.3%0.587346385
PRKG279.2%0.58387543
BLK79.1%0.5835580107
AURKC79.0%0.58059235
CCNA279.0%0.580259784
PRKCG78.9%0.5794848100
MAP3K1278.9%0.57815007174
PGK178.9%0.57994244
PLK278.8%0.577131952
IRAK478.8%0.577199461020
BRSK178.8%0.577152257
JAK378.8%0.576199701950
HIPK378.8%0.57923122
MAPK178.7%0.575199801110
CAMK1D78.7%0.57476640
SIK178.7%0.57319979231
PDGFRA78.6%0.57319937432
MKNK278.6%0.57319925399
NEK478.6%0.573126252
ERBB278.6%0.572199071330
CLK278.6%0.57219877313
ERBB478.6%0.5725840109
FLT378.6%0.571199541152
MET78.6%0.571199741211
PLK478.5%0.57019950255
MARK478.5%0.56934927
LIMK178.5%0.57019852202
MERTK78.5%0.57019941310
ERN178.4%0.56815318
CDK578.4%0.56819912507
YES178.4%0.568426293
PTK2B78.4%0.5687247121
CAMK178.3%0.56724923
STK478.2%0.56534927
MAPK1478.2%0.564199691862
PIK3CD78.2%0.564198321225
CAMKK278.2%0.56497645
MAPKAPK378.2%0.56116519
ACVRL178.1%0.563117149
CHEK178.1%0.56119949639
PRKCH78.0%0.5615020101
EPHB478.0%0.55916707187
PIKFYVE77.9%0.55813617
KIT77.9%0.55819907657
EPHA877.9%0.55746131
SMG177.8%0.5575411
TSSK277.8%0.5555411
PTK677.6%0.553121250
DAPK177.6%0.55715218
MAP2K277.5%0.551141554
DDR277.5%0.550255172
DYRK1B77.4%0.54919925271
PRKCI77.4%0.5489591160
LCK77.4%0.54719917624
AURKA77.3%0.546199541247
FLT177.3%0.546198831128
EPHB377.3%0.5436612
ERBB377.3%0.5466612
PRKX77.2%0.545355486
ACVR177.2%0.5447828126
TNIK77.1%0.5425137102
TGFBR177.1%0.54219959593
BTK77.1%0.542199641608
DCLK277.0%0.53926524
RPS6KA276.9%0.538355885
MAPK376.8%0.53825023
IKBKB76.8%0.53619937405
RPS6KB176.8%0.53619940664
PDPK176.7%0.53419917361
PLK176.6%0.53319950341
CDK676.6%0.53219974526
LYN76.5%0.53019905205
CSK76.5%0.52748132
STK1076.4%0.52970038
EIF2AK376.4%0.52865337
TEK76.4%0.52716436182
PAK476.3%0.52513603166
ULK176.1%0.52229325
PDK176.1%0.52116046248
ROCK276.0%0.52119973961
STK2476.0%0.52110415
TAOK175.9%0.5175759110
MAPK1575.8%0.51712016
EPHA575.8%0.51712016
PIK3C2A75.8%0.5156612
PRKCZ75.7%0.514169359
MAP3K1475.6%0.51319896281
PANK375.6%0.5337813
MKNK175.4%0.50919940371
RET75.3%0.50619985770
PRKG175.3%0.506159958
MAP3K575.2%0.504292878
ITK75.2%0.50419933444
STK17A75.2%0.504309880
DDR175.1%0.502390789
AAK175.1%0.50219426198
RPS6KA175.0%0.50119918416
MAP3K175.0%0.501200864
MAP4K374.8%0.49311916
SPHK274.8%0.49411916
SIK274.8%0.49519952331
CDK774.6%0.49319839474
MAP2K174.5%0.49019968340
MAP3K374.4%0.4907813
CDK974.2%0.484199181322
MAP2K574.1%0.4826962119
NUAK174.1%0.48069838
TLK274.1%0.48113517
MAP3K774.0%0.4808545132
PRKD274.0%0.4796530116
FYN73.9%0.47919955342
NEK273.9%0.478357786
SGK173.9%0.47716695184
MARK273.5%0.470370088
DAPK373.5%0.4697159121
IRAK173.4%0.46911915156
CSNK1G173.4%0.468140754
PI4KB73.4%0.4675022101
PIM373.3%0.46619774977
DAPK273.3%0.47020621
PIK3C372.9%0.4599287137
EIF2AK272.7%0.4517713
SGK272.6%0.451128352
CSNK1G372.4%0.44883742
INSRR72.3%0.44513017
CHEK272.3%0.44613410165
RPS6KA572.2%0.444249172
CHKA72.1%0.44310415
CDK872.1%0.44219887248
CDC42BPA72.0%0.440264175
PBK71.9%0.4385317104
EIF2AK471.9%0.43743030
CLK371.8%0.437426993
HIPK271.8%0.436473599
MAP2K671.7%0.43511316
EPHB671.6%0.44019020
PRKACA71.6%0.43219932281
DGKZ71.4%0.42719885270
BMPR271.3%0.42513617
MAP3K271.3%0.42734827
EPHA371.2%0.4326612
STK1671.1%0.42432526
CSNK1G271.1%0.421268275
CDK11A70.8%0.4178914
LATS270.7%0.42018820
PKLR70.4%0.40827424
RIPK270.4%0.40811239151
ADK70.3%0.407253672
MAP3K870.3%0.407126451
DYRK270.3%0.40614159169
GRK270.3%0.406163858
MAPKAPK570.2%0.403108648
CSNK1A170.2%0.40319070197
STK2670.0%0.4009014
MYLK270.0%0.4049014
ATR69.9%0.3999021135
RIOK269.7%0.4007613
ACVR1B69.7%0.39420821
BMPR1A69.7%0.39723122
STK369.7%0.3948684133
EPHA769.6%0.38713517
MAP2K769.6%0.391169960
CCNE169.4%0.387352685
TNK169.2%0.3879114
CDKL269.2%0.3836512
MAP3K2069.0%0.379232770
PHKG268.9%0.379104047
PAK268.9%0.377382389
EPHA468.8%0.36917019
CAMK468.6%0.3795112
DYRK368.5%0.3698324130
CDC768.0%0.35919922204
NLK67.9%0.35818720
ZAP7067.8%0.355226568
IKBKG67.7%0.354379088
MATK67.6%0.35321922
TIE167.5%0.3567713
PIP5K1C67.3%0.3655211
BUB167.1%0.34219898233
TAOK266.9%0.34913317
CHUK66.8%0.3357040120
DYRK466.7%0.335282976
GSG266.7%0.334117649
CDK1366.7%0.3389014
DNAJC366.7%0.32510215
ULK366.7%0.3269014
CDK1266.4%0.3297264131
CAMK2B66.3%0.327231671
HIPK466.1%0.322274876
DGKA65.8%0.31719846339
MAP3K1965.8%0.31815218
PDK465.4%0.30813617
BMP2K65.3%0.30534927
MAP3K1365.1%0.3106612
FER65.0%0.299109648
FRK64.8%0.29619792298
PLK364.7%0.295367487
BMPR1B64.6%0.29235027
MAPK1164.4%0.288322381
CDK1664.3%0.28637528
MAP4K563.9%0.2798900136
IP6K263.8%0.27310515
SLK63.7%0.274463598
KHK63.7%0.26713517
TGFBR263.7%0.27358935
EIF2AK163.6%0.2745511
SRPK162.9%0.258150357
RPS6KA662.6%0.25266337
RIPK362.4%0.24918920
CSNK2B62.4%0.248161258
MYLK62.3%0.245158157
GAK61.3%0.227165158
MAP2K361.2%0.22229925
BMX61.0%0.2215549110
RPS6KA460.7%0.21322922
IP6K160.4%0.20859335
NUAK260.4%0.20816919
CDK1460.1%0.20143330
CDK1759.4%0.1596412
TNNI3K59.3%0.186192963
EEF2K58.6%0.17724923
FES58.5%0.17052333
ILK58.0%0.1599478140
CIT57.9%0.15720921
SPHK156.4%0.13166137
TK155.2%0.10513417
MAP2K454.5%0.09127524
PIP4K2B54.5%0.1066612
MYLK354.5%0.1165511
GALK153.8%0.07716919
CCNB153.3%0.07116584
MAPK750.0%0.00912016
PIP4K2C49.2%0.00012016
ICK47.1%-0.05811916
DCK44.6%-0.11142430
LATS143.0%-0.1406997119
GCK42.5%-0.16316719
CSNK2A339.0%-0.22711816

Understudied kinases perform no worse once data volume is matched

Of the 179 targets this project classes as understudied, the published model already scores 163, because they are targets with modest data rather than targets with none. Taken at face value they look weaker: median accuracy 0.736 against 0.769 for the rest.

They also carry far less held-out data, a median of 754 pairs against 19,893, a twenty-six-fold difference. Comparing them at matched data volume, in the one band where both groups are well populated, understudied targets score 0.715 (n=31) against 0.723 (n=22): a difference of -0.0077 ± 0.0403, indistinguishable from zero.

How far this goes, and no further. In the one band where both groups are well populated, no difference is detected. That is a failure to find a difference across 31 and 22 targets, not a demonstration that none exists, and the interval is wide enough to hide a real effect. It does not show that adding data to an understudied target would cause it to improve. ‘Understudied’ is itself defined by data volume, so this is close to asking whether volume explains volume.

Methods

Task definition

The model answers a comparison: given one kinase sequence and two ligands, which ligand is more potent against that kinase. Ranking a compound library is a sequence of such comparisons, so this is the primitive operation of virtual screening rather than a proxy for it. Because the orientation of every pair is randomized, a constant output scores exactly 0.500 and a Matthews correlation of 0.000, so the chance line needs no argument. That is not the same as there being no baseline worth beating, and two are reported beside the model below: ranking by molecular size alone, and looking a compound's potency up from another kinase where such data exists.

Data

Measurements are drawn from the Kinase Knowledgebase (KKB), Eidogen-Sertanty, release Q2-2026, restricted to human enzyme assays with a defined potency between pIC50 3 and 11.

From the corpus to the fitted model, every step:

stepmeasurementswhat changes
Source corpus841,187every KKB row on the 500-gene panel
Training half673,659167,528 rows are held out for testing and never trained on
Exact values only425,524records that give only a limit, such as weaker than 10 micromolar, rather than an exact potency, are set aside; repeat measurements of the same compound against the same kinase are collapsed to their median
Fitted425,524all of them; nothing is held back from the fit

The training half spans 500 kinase genes and 742 distinct sequences, including 13,218 rows carrying a mutated sequence; a mutant is represented by its own sequence rather than folded into the wild type.

The largest single reduction, and what it costs. 248,083 rows do not report a potency at all. They report a LIMIT: the assay shows only that the compound is weaker than the highest concentration tested, typically 10 micromolar. There is no number to learn from, and this model is trained to predict a number for one compound at a time, so those rows are set aside.
Why that matters, and why it is the first thing to fix. Those set-aside rows are most of the corpus's direct evidence that a compound does NOT bind, so the model has been shown far fewer genuine non-binders than the data contains. And a limit is often enough to settle a comparison outright: if compound A measures 8.0 and compound B is only known to be weaker than 5.0, A wins, with no ambiguity. Roughly a quarter of the corpus is usable evidence that this design cannot currently consume.

Molecular representation

Each ligand is described by a Morgan count fingerprint of radius 2 and 1,024 bits, computed with RDKit from the canonical SMILES, concatenated with fourteen physicochemical and compositional descriptors: molecular weight, heavy-atom count, bond count, rotatable bonds, ring count, and the counts of C, N, O, S, F, Cl, Br, I and P. The descriptors are part of the arm that was scored and are reported for exactness. Their measured contribution is negligible: paired per kinase over the 213 targets with at least 1,000 pairs, adding them is worth +0.0023 ± 0.0030 on count fingerprints. An earlier version of this document explained them by saying a circular fingerprint represents molecular size poorly; that explanation predicted a material gain on BINARY fingerprints, and the measured gain there was +0.0032 ± 0.0031, no different. The explanation is withdrawn and plain Morgan counts would serve as well.

Protein representation

Each kinase sequence is embedded with ESM2 (esm2_t12_35M_UR50D), giving one vector per residue, and reduced to a single fixed 480-dimensional vector by mean pooling over residues. Pooling is fixed rather than learned. Concatenated with the 1,038 ligand features, the vector presented to the model is 1,518 dimensions: 480 protein and 1,038 ligand.

No structural information reaches the model. This project extracts binding-site residues from co-complex and apo crystal structures, and that work underpins the mutant and site analyses reported separately. It is not an input to the model described here, which is given the amino-acid sequence and the ligand and nothing else. Nothing in these results should be read as a structure-aware prediction.

What the model is given, and what it is not

The model is trained on human kinase sequences and ligand structures from the Kinase Knowledgebase, and on nothing else. Its training table carries 673,659 rows over 500 kinases and 742 distinct sequences, of which 13,218 rows are point-mutant sequence records. Every row is an original measurement.

Mutants are trained on but not validated here. The published held-out split contains zero mutant rows, so nothing in this document measures how well the model predicts for a point mutant. Mutant prediction is a separate claim and it has not been tested.
Three things this project built that the model does NOT use. Binding-site residues were extracted from 1,417 crystal structures; cross-species orthologue sequences were assembled and screened by receptor-site conservation; and 821,370 BindingDB measurements were ingested and guarded against the held-out sets. None of the three is an input to the model described here, and none was used to train it. They are open directions under active test, and every number in this document was produced without them.

Relation to prior work

The structure-free framing of this project, predicting engagement from protein sequence and ligand chemistry alone, with no three-dimensional pose, using a protein language model together with a chemical language model, follows Fondrie and colleagues, Structure-free, site-resolved contrastive learning extends small-molecule discovery beyond the reach of structure-based modeling (Talus Bioscience, bioRxiv 2026), which introduces Ptarmigan-1.

The debt is specific and worth stating precisely. This project's neural arm was adapted from that architecture: frozen ESM2 residue embeddings and frozen ChemBERTa ligand embeddings projected into a shared space, with the temperature-scaled softmax pooling over residues that Ptarmigan-1 uses to turn residue-level scores into a protein-level call reused here as attention pooling, and its leakage-safe Bemis-Murcko scaffold-split discipline. Trainable projections and head only, both backbones frozen, as a lighter-weight analog of its LoRA fine-tuning.

The model reported here is not that model. The neural arm was beaten on every measure by the random forest described above, which uses fixed mean pooling and a count fingerprint rather than learned pooling and a chemical language model, and which is what all results in this document describe. The two also answer different questions: Ptarmigan-1 predicts and localizes engagement across the proteome, including at cryptic and disordered sites; this model ranks ligands by potency against a kinase for which measured data already exists.

Model and why it is antisymmetric

A random forest regressor of 200 trees (minimum two samples per leaf) is fitted pointwise on (protein vector, ligand features) against measured pIC50. Two ligands are then compared by the difference of the model's two predictions, so that P(A more potent than B) follows the sign of sA − sB. This construction is exactly antisymmetric: exchanging the two ligands reverses the prediction identically, and the model cannot return contradictory answers for the same pair presented in a different order. A forest given both ligands as joint input would carry no such guarantee. It also means a library of n compounds is scored in n forward passes rather than n2 comparisons.

Evaluation pairs

This model has no training pairs. The forest is fitted pointwise on individual pIC50 values; pairs exist only at evaluation, and every non-tied pair of ligands measured against the same sequence is scored. There is no gap filter on pair construction.

Results are then reported in three bands of true potency difference: under half a log, half to one log, and more than one log. The band most worth quoting is the last, because that is the separation a chemist acts on. For context on what a resolvable difference is, disagreement between two independent publications reporting the same compound against the same kinase has a 90th percentile of 0.715 log units over 204,438 comparisons, rising to 1.218 after deduplication; the source file recommends treating one log as the floor. Pairs closer than that are harder rather than impossible: the model still scores 57.8 percent under half a log, against a 50 percent coin flip.

Held-out evaluation

Held out per kinase, by design. The split withholds compounds by Bemis-Murcko scaffold within each kinase, keyed on the (sequence, ligand) pair. No protein and no family is ever withheld, because the deployment case is ranking compounds against a kinase you already have data on. A consequence is that a compound withheld from one kinase may appear in training against a DIFFERENT kinase: 57.2 percent of unique test compounds and 74.6 percent of test rows involve such a compound. That is the realistic case rather than a leak, since screening libraries are full of compounds measured against something. The exact unit being predicted, this compound against this kinase, is withheld: only 0.65 percent of test rows share a (kinase, ligand) combination with training.

The advantage that cross-kinase knowledge confers is not assumed away, it is measured. The compound-lookup baseline above is exactly that advantage made explicit, predicting from the same compound’s potency against another kinase, and it scores 73.1 percent. The model scores 76.8 percent on those same pairs, so it adds information beyond knowing the compound elsewhere. What these internal numbers do not measure is performance on chemistry absent from the corpus entirely; the external section speaks to that.

Ligands are withheld by Bemis-Murcko scaffold, keyed on the (sequence, ligand) pair. Every kinase sequence remains in training. No protein and no sequence family is withheld, because a kinase for which no data exists is not the deployment scenario: the kinome is among the most exhaustively characterized regions of the proteome. Evaluation covers 3,040,480 held-out ligand pairs across 437 targets. The kinase is the unit of replication for the kinase-level mean and its standard error, and only for those. Pooled accuracy, the margin deciles and the selectivity figures are computed over pairs that share compounds and are therefore not independent observations; treat them as descriptive rather than as quantities with an interval.

What this model does not use

The headline results are obtained without three bodies of work that are built and available: 4,555 CANDIDATE orthologue sequences across 475 organisms, screened by binding-site identity; per-residue contact maps from 1,417 solved co-complex structures covering 170 targets, with 977 identified ligands and 182 sites excluded because a nucleotide or crystallization additive rather than an inhibitor defined them; and 821,370 BindingDB measurements retained after removing every compound-target pair appearing in the frozen ChEMBL benchmarks. These are headroom, not caveats.

Data sources

Kinase Knowledgebase (KKB), Eidogen-Sertanty, release Q2-2026, the training and evaluation corpus: eidogen-sertanty.com/kinasekbmarvin.php.

BindingDB, ingested and held for future arms, not used in the results reported here. Gilson et al., Nucleic Acids Res. 44:D1045 (2016), doi:10.1093/nar/gkv1072; bindingdb.org.

ChEMBL, used only as a frozen external benchmark, never trained on; every compound-target pair appearing in it was removed from the BindingDB import. Zdrazil et al., Nucleic Acids Res. 52:D1180 (2024), doi:10.1093/nar/gkad1004; ebi.ac.uk/chembl.

RCSB Protein Data Bank, co-complex structures used for the binding-site analysis reported separately. Berman et al., Nucleic Acids Res. 28:235 (2000), doi:10.1093/nar/28.1.235; rcsb.org.

UniProt, sequence and accession mapping. UniProt Consortium, Nucleic Acids Res. 51:D523 (2023), doi:10.1093/nar/gkac1052; uniprot.org.

Models and software

ESM2 protein language model, checkpoint facebook/esm2_t12_35M_UR50D, 35M parameters, 12 layers, 480-dimensional per-residue representations. Lin et al., Science 379:1123 (2023), doi:10.1126/science.ade2574; code at github.com/facebookresearch/esm.

ChemBERTa chemical language model, checkpoint seyonec/ChemBERTa-zinc-base-v1, 768-dimensional embeddings, used in the comparison arm. Chithrananda et al., arXiv:2010.09885 (2020), arxiv.org/abs/2010.09885.

Morgan / ECFP count fingerprints, radius 2, 1,024 bits, and all physicochemical descriptors computed with RDKit: rdkit.org. Method: Rogers & Hahn, J. Chem. Inf. Model. 50:742 (2010), doi:10.1021/ci100050t.

Bemis-Murcko scaffolds, used to define the held-out ligand split. Bemis & Murcko, J. Med. Chem. 39:2887 (1996), doi:10.1021/jm9602928.

Random forest regressor, scikit-learn: scikit-learn.org. Breiman, Machine Learning 45:5 (2001), doi:10.1023/A:1010933404324.

Pairwise ranking formulation: Burges et al., Learning to Rank using Gradient Descent, ICML 2005, doi:10.1145/1102351.1102363.

Matthews correlation coefficient: Matthews, Biochim. Biophys. Acta 405:442 (1975), doi:10.1016/0005-2795(75)90109-9.

The two released models: Validated and Frontier

A model cannot be measured on data it was trained on. Maximum evidence and maximum data therefore cannot be the same object, so both are released rather than quietly choosing one.

425,524
VALIDATED model, training measurements. Every accuracy in this report was produced by this model, on data it never saw.
780,067
FRONTIER model, training measurements. Every measurement the project holds. No accuracy exists for it and none can.
If you are deploying, read only this paragraph. The VALIDATED model, bundle kfm_ranker_v1_final, arm c56_validated_rebuild, was trained on 425,524 measurements. The FRONTIER model, bundle kfm_ranker_production, arm c55_production_all_data, was trained on 780,067 measurements, which is every measurement in the corpus. If the model you are holding was trained on roughly four hundred thousand measurements it is the VALIDATED one; roughly eight hundred thousand means it is the FRONTIER one. Each bundle states its own count in config.json under trained_on, and its own release name under release. Check that field rather than the directory name.
VALIDATED
kfm_ranker_v1_final
FRONTIER
kfm_ranker_production
measurements trained on425,524780,067
data usedtraining half, exact values onlyboth halves, exact values and limit-only records
architectureidentical: 200 trees, same features, same seed, same training method
measured accuracy74.8% per target, 90.0% on decisive pairsnone, and none is possible
use it whenyou need a number you can defendyou want the most informed prediction available
Every number in this document is the Validated model. It was measured on 2,965,273 held-out pairs it never saw, and the file shipped is the file that produced those numbers. Nothing here describes the Frontier model, because anything it scored would be data it had already learned.
The Frontier model is untested, by construction and on purpose. It uses the same training method on more evidence: the held-out half, and the limit-only records that state a compound is weaker than the concentration tested. More data with an unchanged method should help, and that expectation is reasonable rather than measured. It is offered for use, not as a claim.

Frontier: 780,067 measurements against 425,524, 83 percent more.

How to use it, and where it stops

How to use it

Point it at a kinase you already have data on and a compound library you have not measured. It scores every compound in one pass and returns a ranked list, each comparison carrying a confidence. Act on the confident fraction: accuracy rises steeply as you narrow to the calls the model is most certain about, so the operating point is yours to choose.

Check the target first. This is the single most important step and it is not optional. Performance varies more between targets than it does between any two models we have tried. On its strongest targets AURKA, MTOR, ABL1, SRC, CDK2, CDK1, BRAF, FGFR1, it calls decisive pairs correctly around nine times in ten on chemistry from a source it has never seen. On its weakest it is close enough to a coin flip that a prediction should carry no weight. The tier table above is the guide, and there is no aggregate number that substitutes for reading it.

Where it stops

It ranks; it does not predict absolute potency. Two compounds whose true potencies differ by less than two publications routinely disagree by cannot be reliably adjudicated from this assay corpus, and the model is correctly close to chance on them rather than confidently wrong.

Where a compound already has measured potency against a related kinase, use that measurement: it is direct evidence. It is not, however, a stronger predictor than the model on this benchmark, that lookup scores 73.1 percent where it is defined, against the model's 76.8 percent on the same pairs. The lookup was undefined on 49.7 percent of evaluation pairs, and it is there that the model is the only option.

It is a triage tool that runs before docking or crystallography, not a replacement for either, and not a substitute for measurement.

A real limitation, stated plainly. Records that say only that a compound is weaker than some tested concentration are excluded from this model, in training and in evaluation. That is 309,111 of the source rows. They carry most of the evidence that a compound fails to bind, so the model has been shown far fewer non-binders than the corpus contains. An earlier version of this document claimed the opposite; the claim was wrong and is withdrawn.

What this model does

It sorts a compound library against a kinase you already have data on, and reports a margin on each call that tracks how often it is right. Used as a filter on its confident predictions it is right 90 percent of the time on chemistry from a source it has never seen. A coin flip is 50%.

Scores a pair in milliseconds with no structure needed, against minutes to hours for docking, so it triages a library before any structure-based work begins. Structural evidence behind the binding-site work.

75%
Every held-out pair. Averaged per kinase over 437 targets and 3,040,480 comparisons. A coin flip is 50%.
90%
When the answer matters. Pairs whose measured potencies differ by more than tenfold.
99%
Largest model margins, compounds held out for that kinase. On compounds from an outside source the same decile falls to about 88.0 percent. Use the lower figure to plan.
3.7×
On chemistry it has never seen. Enrichment in the top tenth of a ranked library, against a random ordering, on 10,108 compounds from an outside source.

The complete experiment record, including every claim that did not survive review, is kept separately in collab/cycles/ and is deliberately not part of this document.

Generated from collab/cycles/*/claude_manifest.json by scripts/37_live_report.py · 09 August 2026 at 09:26