Eidogen-Sertanty · live results

A comparator model that rank-orders molecules against a kinase

Give it one of the 478 kinases it has data on, plus two molecules, and it says which one binds more tightly. That single comparison is the primitive operation of virtual screening, so repeating it sorts an entire library in seconds where a docking campaign takes hours, and at no point does the model see a three-dimensional structure.

Two models are released. The Validated model carries every number in this report, measured on 2,965,273 pairs it never saw. The Frontier model uses the identical training method on every measurement we hold, including the data withheld to make that measurement possible, and is therefore untested by construction. Which target you ask about matters more than which model you pick: the reliability tiers below are the part to read before acting on any prediction.

Version 2026-08-08_2305 Generated 08 August 2026 at 23:05 Cycles filed 40

What this is

A comparator. Given a kinase and descriptions of two drug-like molecules, it predicts which of the two binds more strongly. No three-dimensional structure, no docked pose and no binding site is required. Because a ranked library is nothing but this comparison repeated, the same model sorts a whole compound collection against a target in one pass. It was trained on human kinase measurements. The source corpus holds 841,187 rows over 500 genes. One third is held out for testing; of the remainder, records giving only a bound rather than a value are set aside and repeat measurements of the same compound against the same kinase are collapsed to their median, leaving 417,507 distinct measurements. The model is fitted on 417,507 of them, all of them.

Which kinases, precisely. The shipped model carries precomputed protein vectors for 478 named kinases over 695 distinct sequences, and those are the targets it will score. Being scoreable is not the same as being supported: the vectors were built for the whole panel, not for the subset the forest actually learned from, so the fit covers only the genes that carry usable measurements. Three exposed targets, NEK10, ADK and SPHK2, contributed no fitted rows; NEK10 is the fundamental case, because every one of its measurements is a limit rather than an exact value. Check a target against the reliability tiers before acting on any prediction for it, and treat a target absent from those tiers as unsupported. Handed a sequence it has never seen it refuses rather than guessing: a confident number derived from the wrong protein is the worst failure this system could produce. Nothing here supports a kinase with no training data.

Model architecture

The question the model answers, drawn from the project's own data. The protein is the ABL1 kinase domain rendered from crystal structure PDB 1IEP, with imatinib in the ATP cleft and the twenty-five residues within 4.5 angstrom of it picked out in orange; the model itself sees only the amino-acid sequence, embedded with ESM2 and pooled to a single 480-number vector. The two compounds are real ABL1 inhibitors from the training table with their measured potencies, each described by a Morgan count fingerprint and fourteen physicochemical descriptors. A random forest scores each compound against the kinase and the two are compared by the difference of their scores, which makes the comparison exactly antisymmetric.
The question the model answers, drawn from the project's own data. The protein is the ABL1 kinase domain rendered from crystal structure PDB 1IEP, with imatinib in the ATP cleft and the twenty-five residues within 4.5 angstrom of it picked out in orange; the model itself sees only the amino-acid sequence, embedded with ESM2 and pooled to a single 480-number vector. The two compounds are real ABL1 inhibitors from the training table with their measured potencies, each described by a Morgan count fingerprint and fourteen physicochemical descriptors. A random forest scores each compound against the kinase and the two are compared by the difference of their scores, which makes the comparison exactly antisymmetric.

Screening performance: sorting compounds by potency

Two compounds, one kinase, which binds harder. Coin flip = 50%.

78.8%
Correct on 2,965,273 held-out ligand pairs across 413 kinases. Matthews correlation 0.576.
90.3%
When the two ligands genuinely differ by more than tenfold in potency, on 1,581,818 pairs. MCC 0.807. This is the call a chemist actually makes.
98.8%
On the tenth of calls carrying the largest model margins, 296,528 pairs, held-out KKB compounds. The margin is a monotone transform of the score difference, not a calibrated probability: larger margins are associated with higher accuracy, which is what makes a threshold useful.

Act on the most confident calls and this is what you get

Every prediction carries a confidence. Rank the calls by it and keep only the top slice: accuracy rises steeply as the slice narrows, so the operating point is yours to choose. The finest slice shown is the top tenth, which is the finest the confidence table resolves.

calls acted onpairsaccuracy
top 10%296,52798.8%
top 25%741,31896.6%
top 50%1,482,63691.8%
top 75%2,223,95585.7%
every pair2,965,27378.8%

Could you skip the model?

Two cheap shortcuts a screening group could use instead. Neither is an input to the model and neither changes what the model is given; they are rival predictors, and the model has to be better than both to be worth running.

methodpairsaccuracy
Rank by molecular size alone2,807,61458.5%
Shortcut 2: look up what this compound did against some other kinase and assume it behaves the same here1,492,76673.1%
This model, on those same pairs1,492,76678.7%
This model, where no prior measurement exists1,472,50779.0%

Shortcut 2 is the serious rival. Potency is correlated across kinases, so knowing what a compound did elsewhere predicts a lot, 73.1 percent, without any model at all. The model beats it on the same pairs, 76.8 percent, and is the only option on the 1.47 million pairs where no such prior measurement exists. Neither shortcut can address selectivity: both give one answer per compound regardless of which kinase you ask about.

Evaluation is capped at 20,000 pairs per kinase so that a few very large targets cannot dominate; 119 kinases hit that cap. Reported, never silent.

Proof the model actually reads the kinase

A fair worry about any model like this: maybe it never really uses the protein, and is just deciding which of the two compounds looks better in general. Here is the test that settles it.

The test

Find compound pairs that were measured against two different kinases, and keep the ones where the answer switches:

Compound A beats compound B on kinase 1.
On kinase 2, compound B beats compound A.
Same two compounds. Opposite answer. The only thing that changed is which kinase you asked about.

A model that ignores the protein produces one answer for that pair. It is therefore right on one kinase and wrong on the other, every time. It scores 50 percent because arithmetic forces it to, not because it is guessing. To do better than 50 percent here, a model has no choice but to use the kinase.

The result

modelgets these switched cases right
This model71.2%
The identical model with the protein taken away50.0%

30,650 switched cases, 5,264 compounds, 377 kinases. Both rows are the same random forest on the same pairs. The only difference is whether the protein vector changes from one kinase to the next.

The control lands exactly on 50.0 percent, as it must. That is how we know the test is built correctly and that the margin above it is real.
And where the answer does not switch, the protein barely matters. On pairs that keep the same order against both kinases, the full model scores 89.1 percent and the protein-free version 89.0 percent. That is the expected result and it is worth stating: the kinase only earns its place where the answer depends on it. Those are the cases behind off-target activity, resistance mutants and repurposing, and they are the ones a compound-quality score can never get right.

Package identity. Every number in this document was produced by the forest shipped as kfm_ranker_v1_final, SHA-256 4f36bf27f5d7c0a7…. Reloading that exact file and rescoring reproduces the internal benchmark with a maximum per-kinase difference of 0.000000.

Not to be confused with

This has nothing to do with swapping the two compounds at the input. If you feed the same model compound B first and compound A second, the answer flips exactly, every time, by construction. That is a property of how the model is built and it is asserted in the package self-test. The test above is about something else entirely: the same pair giving genuinely opposite answers on two different proteins.

Ranking a whole library, checked against the bench

Screening needs a ranked list, not a single comparison. We gave the model a target and a whole set of compounds, had it sort them from most to least potent, and checked that order against what the assays actually measured.

We did this twice. First on compounds from our own collection that were held back from training. Then on compounds from ChEMBL, the public literature database, which come from different laboratories and which the model has never seen. The second is the harder test, and it is what the external numbers below describe. It is a retrospective benchmark on an outside dataset with exact training compounds removed, not an untouched independent replication: this corpus has been used elsewhere in the project, and only exact compound matches were excluded, not close analogues.

same collection, compounds withhelddifferent database, never seen
compounds ranked49,21110,161
pairwise comparisons46,330,1091,926,222
agreement with the measured order0.790.50
pairs called right when potencies differ by more than tenfold90%78%
enrichment in the top tenth of the list5.5x3.6x
The gap between those two columns is the finding. Ranking transfers to a corpus the model has never seen, every one of the 30 targets has a positive correlation, though the weakest of them, PIK3CD at 0.09, is not distinguishable from chance on its own, but roughly a third of the agreement does not survive the move. A holdout drawn from the same collection as the training data is measurably optimistic, and the right number to plan against is the one in the right-hand column.
How the external set was kept clean. It was built years earlier to contain nothing in training at the time, and it did. Training has since grown from 30 kinases to 500, and thousands of those compounds entered it through other targets. Every one of the 13,528 measurements was therefore re-checked against all current training compounds by canonical structure and by InChIKey; 3,367 matched and were removed, leaving 10,161. Without that re-check a quarter of this test would have been compounds the model had already been shown.

Which targets to trust, on unseen compounds

Performance is not uniform and the aggregate hides that. These are the same 30 targets, split by how well the model ordered compounds it had never seen. A target's tier is the first thing to check before acting on a prediction for it.

Reliable

Order these with confidence. Agreement of 0.6 or better, and the top of the list is strongly enriched in genuinely potent compounds.

targetcompoundsagreement with measured orderdecisive pairs correctenrichment in top tenth
AURKA2210.8794%3.7x
CDK11370.8192%7.7x
MTOR1690.7588%5.8x
SRC3700.7288%5.7x
PIM11620.7184%4.4x
ABL14960.6983%4.0x
CDK22930.6785%4.2x
BRAF2170.6384%4.0x
ERBB23940.6181%3.1x

Useful for triage

Clearly better than chance and worth using to prioritise, but the ordering is loose enough that the top of the list should be confirmed rather than trusted.

targetcompoundsagreement with measured orderdecisive pairs correctenrichment in top tenth
FGFR14770.6082%3.3x
MET4670.5581%5.5x
SYK3630.5383%6.2x
FLT11320.5178%3.9x
MAPK141450.4674%3.7x
PIK3CG4540.4271%3.6x
CDK94420.4274%4.3x
TYK24870.4171%2.8x
JAK33770.4170%5.0x

Do not rely on these

Either the ordering is barely better than chance, or the top of the list is not enriched enough to act on, a target whose top tenth holds no more potent compounds than a random draw is not usable for screening however well the rest of the list is sorted. They are named rather than folded into an average.

targetcompoundsagreement with measured orderdecisive pairs correctenrichment in top tenth
AKT13710.7786%1.6x
PIK3CA3710.5878%1.6x
KDR2740.4975%1.1x
CDK42140.4881%1.0x
BTK3980.4071%2.7x
FLT34670.3767%3.4x
JAK22580.3466%4.2x
EGFR3050.3266%2.4x
MAPK14990.3166%1.0x
GSK3B3320.3063%1.2x
PDGFRB3930.2773%0.0x
PIK3CD4760.0957%3.3x
Every target ordered independently, on unseen compounds. The spread across targets is wider than any single summary number conveys.
Every target ordered independently, on unseen compounds. The spread across targets is wider than any single summary number conveys.

Which targets is this model reliable on?

The headline describes no individual target. Across 332 kinases with enough held-out pairs to measure, 260 reach 70% pairwise accuracy or better, 58 sit between 60% and 70%, and 14 fall below 60% against a 50% coin flip. Use it where it is strong; the rows below say where that is.

Arm: c51_rf_full_data. Sorted by the first metric column, best first, pairwise accuracy in ranker mode, rank correlation otherwise. Test ligands is the number of held-out compounds that target was scored on. Read it before the accuracy: a target scored on 50 pairs carries only about a dozen distinct ligands, and a high number there is not the same evidence as the same number over thousands of pairs.

kinasepairwise accuracyMCCtest pairstest ligands
ULK298.1%0.9625211
GRK793.3%0.8709014
STK2593.2%0.8647313
EPHB192.3%0.8466512
CAMK2D90.4%0.80819730279
MAP3K988.0%0.76027424
DCLK187.4%0.74316619
EPHB287.2%0.74627324
CAMK2A86.9%0.739130252
OXSR186.9%0.7388415
TSSK1B86.8%0.73739329
RIPK186.2%0.72419958915
PIM186.1%0.723199391689
PKMYT186.0%0.72113617
PTK286.0%0.71919620745
IGF1R85.8%0.71619781694
CSNK1D85.8%0.71619915433
PKN185.6%0.70718720
TTK85.3%0.70619911741
RAF185.3%0.70619880834
IKBKE85.1%0.70319914256
ROCK185.0%0.700199521002
FGFR185.0%0.699199621176
STK3384.6%0.69411716
CAMK1D84.6%0.69268938
PRKCB84.6%0.69219966215
MTOR84.5%0.689199752222
LTK84.4%0.687185162
CDK484.3%0.687199631374
ABL284.3%0.68656034
MAPKAPK284.3%0.68519951413
FGFR284.1%0.68319917777
ABL184.0%0.680199591145
MST1R83.9%0.679180661
TBK183.8%0.67619883329
MAPK983.7%0.67419906402
MAPKAPK383.6%0.67316519
ACVRL183.6%0.67270138
MAP3K1083.6%0.67226824
PRKCE83.4%0.6696420114
PIK3CA83.4%0.668199702066
WEE183.4%0.66719905301
AXL83.4%0.66719889468
MAP4K183.3%0.66719326247
RIOK183.3%0.6667813
FGFR383.3%0.66619952910
PAK183.3%0.66610665147
PIK3R483.3%0.66519923375
CDK283.2%0.665199012211
ALK83.2%0.66419930697
GSK3A83.0%0.66119956772
CDK383.0%0.66066637
HIPK183.0%0.65937628
FGR82.8%0.656364186
JAK282.8%0.656199763500
MARK182.8%0.6518714
PRKD382.7%0.65510463147
TXK82.6%0.65366237
ROS182.5%0.6514922100
MAP4K282.5%0.6508992136
NTRK382.5%0.65010342145
BRAF82.4%0.648199171455
GRK582.4%0.64757435
SYK82.3%0.646197121497
CAMK182.3%0.64724923
NTRK282.3%0.64619918213
PRKD182.2%0.64534827
AKT382.1%0.64219909317
CLK182.0%0.64019959268
CSF1R82.0%0.64019913603
PDGFRB82.0%0.63919968636
TYK281.9%0.639199821259
EGFR81.9%0.637199592441
CAMK2G81.8%0.637388490
MELK81.8%0.63619858252
PIK3CB81.8%0.63619965829
PRKCA81.8%0.63619955256
CLK481.8%0.63619836353
CSNK2A181.8%0.63519956449
EPHA681.8%0.63225223
ERN181.7%0.63215318
AKT181.7%0.63319957993
ATM81.7%0.6335619107
CDC42BPG81.5%0.6266512
TSSK281.5%0.5945411
EPHA281.4%0.62816295187
MAPK881.4%0.62819948635
SRC81.3%0.626199601112
PIK3CG81.3%0.626199661394
PTK681.3%0.62593444
MAPK1381.2%0.624109748
JAK181.2%0.624199302951
FLT481.2%0.62419871216
MAP3K1181.1%0.62020621
PRKCQ81.0%0.62019972681
INSR80.9%0.61819804301
EPHA880.9%0.61846131
DMPK80.9%0.6146813
PRKCD80.8%0.61619899294
PRKCG80.8%0.6154848100
DAPK180.7%0.60713517
PRKCH80.7%0.6155020101
MAPK1080.7%0.61319939562
NTRK180.6%0.61219966982
AKT280.5%0.61019916329
GSK3B80.5%0.610199461383
RPS6KA380.4%0.60819714231
KDR80.4%0.607199343026
CSNK1E80.4%0.60719968276
MINK180.3%0.6074977102
PRKG280.2%0.60587543
MAP2K280.2%0.603120950
TAOK380.1%0.60345332
MAPK1280.1%0.602424993
PRKDC80.0%0.59919860234
CSNK2A279.9%0.598464397
DYRK1A79.9%0.59719930660
LRRK279.8%0.597199651048
MAP4K479.8%0.59719713276
STK2479.8%0.60010415
TEC79.8%0.59481641
TYRO379.8%0.59519896252
AURKB79.7%0.59319908876
EIF2AK479.6%0.5965411
FGFR479.6%0.59219951606
HCK79.6%0.59112800161
IRAK479.5%0.591199401020
EPHA179.5%0.58929725
AURKC79.4%0.58859235
PRKAA179.4%0.58812487159
PIM279.3%0.587199641117
JAK379.3%0.586199731950
TNK279.2%0.58412050156
PLK279.1%0.583131952
PDK279.1%0.58311422152
LIMK179.1%0.58319850202
CDK579.1%0.58219913507
CDK179.1%0.582199571008
MARK379.0%0.581346385
PDGFRA79.0%0.58019938432
CDK1979.0%0.580472098
SIK379.0%0.57919949219
SIK178.9%0.57919981231
STK478.8%0.57817019
MAP3K1278.8%0.57615007174
HIPK378.8%0.57523122
PLK478.7%0.57419931255
MET78.6%0.573199401211
EPHB478.6%0.57316707187
NEK478.6%0.572126252
MAPK178.6%0.572199761110
YES178.6%0.572426293
PTK2B78.6%0.5717247121
MERTK78.6%0.57119918310
PKN278.5%0.571317881
ERBB478.4%0.5685840109
STK17B78.4%0.5998814
FLT178.4%0.568198891128
BRSK178.4%0.568152257
SRMS78.3%0.56697845
LCK78.3%0.56619928624
ERBB278.3%0.565199051330
MKNK278.2%0.56419919399
ACVR178.1%0.5637828126
RIOK278.1%0.5616412
CHEK178.1%0.56219949639
PRKX78.0%0.561355486
FLT378.0%0.559199561152
AURKA78.0%0.559199591247
CLK278.0%0.55919888313
MAPK1477.6%0.552199761862
PIK3CD77.5%0.551198601225
MAP4K377.3%0.54911916
TEK77.3%0.54516436182
KIT77.2%0.54419906657
PLK177.1%0.54319942341
PRKCI77.1%0.5429591160
DCLK277.0%0.54426524
DYRK1B77.0%0.53919911271
DDR276.9%0.538142754
BTK76.8%0.537199611608
BLK76.7%0.5345580107
MAPK1576.7%0.52912016
ROCK276.4%0.52819971961
MAPK376.4%0.53025023
LYN76.4%0.52819905205
PDPK176.4%0.52819914361
PRKCZ76.4%0.528169359
ERBB376.4%0.5275511
CDK676.3%0.52619960526
RPS6KB176.2%0.52319956664
IKBKB76.2%0.52319934405
INSRR76.1%0.51013017
MAP2K676.1%0.52711316
PAK475.9%0.51813603166
TAOK175.8%0.5155759110
PDK175.7%0.51416035248
DDR175.7%0.513247871
TGFBR175.7%0.51319963593
ITK75.6%0.51219930444
MKNK175.6%0.51219950371
CDK1375.6%0.5109014
MYLK275.6%0.5149014
NEK275.5%0.511357786
RET75.4%0.50819982770
RPS6KA275.2%0.503170459
CDK775.2%0.50319822474
AAK175.0%0.50119426198
MAP3K1475.0%0.49919907281
MAP2K174.9%0.49919955340
RPS6KA174.9%0.49819920416
EPHB674.7%0.48919020
MAP3K774.7%0.4938545132
CSK74.6%0.49048132
STK1074.6%0.49370038
PRKD274.5%0.4906530116
PBK74.4%0.4895317104
STK17A74.1%0.483309880
IRAK174.0%0.48011915156
STK1674.0%0.48630025
CLK374.0%0.479426993
PIM373.9%0.47819796977
SGK173.9%0.47816695184
SIK273.8%0.47719949331
PIK3C373.7%0.4749287137
MARK273.7%0.474370088
TNIK73.3%0.467142854
FYN73.3%0.46719958342
PI4KB73.3%0.4665022101
HIPK273.2%0.464473599
DAPK373.2%0.4637159121
CSNK1G273.1%0.462268275
CAMKK273.1%0.46280641
CSNK1G373.0%0.46083742
EIF2AK372.8%0.45514718
DGKZ72.7%0.45419891270
CDK972.7%0.453199311322
PRKG172.6%0.452159958
CSNK2B72.5%0.45175040
PRKACA72.5%0.45119938281
EPHA572.5%0.45112016
CHEK272.4%0.44813410165
LIMK272.3%0.447182661
SGK271.8%0.435128352
CSNK1G171.3%0.424135453
MAP2K771.1%0.42429825
STK2671.1%0.4289014
CDK871.0%0.42019883248
RIPK270.9%0.41811239151
DAPK270.9%0.41720621
PAK270.7%0.413382389
IRAK370.5%0.40910515
CDC42BPA70.4%0.409264175
EPHA470.4%0.39615218
NEK170.4%0.40813517
CSNK1A170.3%0.40719070197
MATK70.3%0.40521922
ZAP7070.3%0.405194063
IKBKG70.1%0.402379088
NUAK170.0%0.38712016
ATR70.0%0.3999021135
TAOK269.9%0.40113317
MAPKAPK569.9%0.398108648
DYRK269.8%0.39714159169
ACVR1B69.7%0.39020821
BMPR1A69.7%0.39423122
MAP3K1369.7%0.3456612
STK369.6%0.3928684133
MAP3K269.2%0.39034827
MAP3K2069.2%0.385232770
PIP5K1C69.2%0.3235211
BMPR269.2%0.3959114
CDC769.0%0.38119923204
CAMK2B68.5%0.369204267
DYRK368.4%0.3698324130
RPS6KA568.3%0.366195964
GRK268.1%0.3629114
MYLK67.8%0.35545031
GSG267.7%0.35132526
BMP2K67.6%0.34834927
PHKG267.6%0.352104047
SLK67.5%0.351463598
BMPR1B67.4%0.35035027
EPHA767.4%0.36213517
DGKA67.4%0.34819851339
MAP2K567.3%0.34630025
DYRK467.0%0.340282976
PDK466.9%0.33813617
NLK66.8%0.33818720
BUB166.7%0.33419893233
CDK1266.6%0.3327264131
MAP3K866.5%0.32993744
NUAK266.4%0.33013417
CDK11A66.3%0.3228914
EIF2AK266.1%0.3116512
GRK666.1%0.3516512
CIT66.0%0.31920921
RPS6KA666.0%0.32029725
TNK165.9%0.2919114
ULK165.9%0.3188814
CDK1665.7%0.31335027
CHUK65.6%0.3137040120
ULK365.6%0.3229014
FRK65.5%0.31119784298
MAP3K1965.2%0.31613517
EPHA365.1%0.2686612
HIPK465.0%0.301274876
TIE164.9%0.2927713
PLK364.9%0.298367487
MARK464.5%0.2887613
MAP4K564.4%0.2878900136
MAPK1164.0%0.279322381
CDKL263.1%0.2326512
MAP3K362.8%0.2567813
FER62.6%0.253104947
CDK1762.5%0.2756412
MAP2K362.2%0.24411916
FES61.0%0.21752333
TNNI3K59.8%0.19588743
CAMK458.8%0.1985112
RPS6KA458.7%0.17918920
SRPK158.1%0.161139255
GRK157.9%0.1447613
GAK57.7%0.155132552
MAP2K457.6%0.1006612
BMX57.3%0.147430898
TGFBR256.1%0.12534627
PIP4K2C51.5%0.0076612
MAPK750.9%-0.0145511
RIPK350.4%0.00911916
ICK42.0%-0.16411916
CSNK2A340.7%-0.20311816

Understudied kinases perform no worse once data volume is matched

Of the 179 targets this project classes as understudied, the published model already scores 163, because they are targets with modest data rather than targets with none. Taken at face value they look weaker: median accuracy 0.736 against 0.769 for the rest.

They also carry far less held-out data, a median of 754 pairs against 19,893, a twenty-six-fold difference. Comparing them at matched data volume, in the one band where both groups are well populated, understudied targets score 0.715 (n=31) against 0.723 (n=22): a difference of -0.0077 ± 0.0403, indistinguishable from zero.

How far this goes, and no further. In the one band where both groups are well populated, no difference is detected. That is a failure to find a difference across 31 and 22 targets, not a demonstration that none exists, and the interval is wide enough to hide a real effect. It does not show that adding data to an understudied target would cause it to improve. ‘Understudied’ is itself defined by data volume, so this is close to asking whether volume explains volume.

Methods

Task definition

The model answers a comparison: given one kinase sequence and two ligands, which ligand is more potent against that kinase. Ranking a compound library is a sequence of such comparisons, so this is the primitive operation of virtual screening rather than a proxy for it. Because the orientation of every pair is randomised, a constant output scores exactly 0.500 and a Matthews correlation of 0.000, so the chance line needs no argument. That is not the same as there being no baseline worth beating, and two are reported beside the model below: ranking by molecular size alone, and looking a compound's potency up from another kinase where such data exists.

Data

Measurements are drawn from the Kinase Knowledgebase (KKB), Eidogen-Sertanty, release Q2-2026, restricted to human enzyme assays with a defined potency between pIC50 3 and 11.

From the corpus to the fitted model, every step:

stepmeasurementswhat changes
Source corpus841,187every KKB row on the 500-gene panel
Training half673,659167,528 rows are held out for testing and never trained on
Exact values only417,507records that give only a limit, such as weaker than 10 micromolar, rather than an exact potency, are set aside; repeat measurements of the same compound against the same kinase are collapsed to their median
Fitted417,507all of them; nothing is held back from the fit

The training half spans 500 kinase genes and 742 distinct sequences, including 13,218 rows carrying a mutated sequence; a mutant is represented by its own sequence rather than folded into the wild type.

The largest single reduction, and what it costs. 248,083 rows do not report a potency at all. They report a LIMIT: the assay shows only that the compound is weaker than the highest concentration tested, typically 10 micromolar. There is no number to learn from, and this model is trained to predict a number for one compound at a time, so those rows are set aside.
Why that matters, and why it is the first thing to fix. Those set-aside rows are most of the corpus's direct evidence that a compound does NOT bind, so the model has been shown far fewer genuine non-binders than the data contains. And a limit is often enough to settle a comparison outright: if compound A measures 8.0 and compound B is only known to be weaker than 5.0, A wins, with no ambiguity. Roughly a quarter of the corpus is usable evidence that this design cannot currently consume.

Molecular representation

Each ligand is described by a Morgan count fingerprint of radius 2 and 1,024 bits, computed with RDKit from the canonical SMILES, concatenated with fourteen physicochemical and compositional descriptors: molecular weight, heavy-atom count, bond count, rotatable bonds, ring count, and the counts of C, N, O, S, F, Cl, Br, I and P. The descriptors are part of the arm that was scored and are reported for exactness. Their measured contribution is negligible: paired per kinase over the 213 targets with at least 1,000 pairs, adding them is worth +0.0023 ± 0.0030 on count fingerprints. An earlier version of this document explained them by saying a circular fingerprint represents molecular size poorly; that explanation predicted a material gain on BINARY fingerprints, and the measured gain there was +0.0032 ± 0.0031, no different. The explanation is withdrawn and plain Morgan counts would serve as well.

Protein representation

Each kinase sequence is embedded with ESM2 (esm2_t12_35M_UR50D), giving one vector per residue, and reduced to a single fixed 480-dimensional vector by mean pooling over residues. Pooling is fixed rather than learned. Concatenated with the 1,038 ligand features, the vector presented to the model is 1,518 dimensions: 480 protein and 1,038 ligand.

No structural information reaches the model. This project extracts binding-site residues from co-complex and apo crystal structures, and that work underpins the mutant and site analyses reported separately. It is not an input to the model described here, which is given the amino-acid sequence and the ligand and nothing else. Nothing in these results should be read as a structure-aware prediction.

What the model is given, and what it is not

The model is trained on human kinase sequences and ligand structures from the Kinase Knowledgebase, and on nothing else. Its training table carries 673,659 rows over 500 kinases and 742 distinct sequences, of which 13,218 rows are point-mutant sequence records. Every row is an original measurement.

Mutants are trained on but not validated here. The published held-out split contains zero mutant rows, so nothing in this document measures how well the model predicts for a point mutant. Mutant prediction is a separate claim and it has not been tested.
Three things this project built that the model does NOT use. Binding-site residues were extracted from 1,417 crystal structures; cross-species orthologue sequences were assembled and screened by receptor-site conservation; and 821,370 BindingDB measurements were ingested and guarded against the held-out sets. None of the three is an input to the model described here, and none was used to train it. They are open directions under active test, and every number in this document was produced without them.

Relation to prior work

The structure-free framing of this project, predicting engagement from protein sequence and ligand chemistry alone, with no three-dimensional pose, using a protein language model together with a chemical language model, follows Fondrie and colleagues, Structure-free, site-resolved contrastive learning extends small-molecule discovery beyond the reach of structure-based modeling (Talus Bioscience, bioRxiv 2026), which introduces Ptarmigan-1.

The debt is specific and worth stating precisely. This project's neural arm was adapted from that architecture: frozen ESM2 residue embeddings and frozen ChemBERTa ligand embeddings projected into a shared space, with the temperature-scaled softmax pooling over residues that Ptarmigan-1 uses to turn residue-level scores into a protein-level call reused here as attention pooling, and its leakage-safe Bemis-Murcko scaffold-split discipline. Trainable projections and head only, both backbones frozen, as a lighter-weight analogue of its LoRA fine-tuning.

The model reported here is not that model. The neural arm was beaten on every measure by the random forest described above, which uses fixed mean pooling and a count fingerprint rather than learned pooling and a chemical language model, and which is what all results in this document describe. The two also answer different questions: Ptarmigan-1 predicts and localises engagement across the proteome, including at cryptic and disordered sites; this model ranks ligands by potency against a kinase for which measured data already exists.

Model and why it is antisymmetric

A random forest regressor of 200 trees (minimum two samples per leaf) is fitted pointwise on (protein vector, ligand features) against measured pIC50. Two ligands are then compared by the difference of the model's two predictions, so that P(A more potent than B) follows the sign of sA − sB. This construction is exactly antisymmetric: exchanging the two ligands reverses the prediction identically, and the model cannot return contradictory answers for the same pair presented in a different order. A forest given both ligands as joint input would carry no such guarantee. It also means a library of n compounds is scored in n forward passes rather than n2 comparisons.

Evaluation pairs

This model has no training pairs. The forest is fitted pointwise on individual pIC50 values; pairs exist only at evaluation, and every non-tied pair of ligands measured against the same sequence is scored. There is no gap filter on pair construction.

Results are then reported in three bands of true potency difference: under half a log, half to one log, and more than one log. The band most worth quoting is the last, because that is the separation a chemist acts on. For context on what a resolvable difference is, disagreement between two independent publications reporting the same compound against the same kinase has a 90th percentile of 0.715 log units over 204,438 comparisons, rising to 1.218 after deduplication; the source file recommends treating one log as the floor. Pairs closer than that are harder rather than impossible: the model still scores 57.8 percent under half a log, against a 50 percent coin flip.

Held-out evaluation

Held out per kinase, by design. The split withholds compounds by Bemis-Murcko scaffold within each kinase, keyed on the (sequence, ligand) pair. No protein and no family is ever withheld, because the deployment case is ranking compounds against a kinase you already have data on. A consequence is that a compound withheld from one kinase may appear in training against a DIFFERENT kinase: 57.2 percent of unique test compounds and 74.6 percent of test rows involve such a compound. That is the realistic case rather than a leak, since screening libraries are full of compounds measured against something. The exact unit being predicted, this compound against this kinase, is withheld: only 0.65 percent of test rows share a (kinase, ligand) combination with training.

The advantage that cross-kinase knowledge confers is not assumed away, it is measured. The compound-lookup baseline above is exactly that advantage made explicit, predicting from the same compound’s potency against another kinase, and it scores 73.1 percent. The model scores 76.8 percent on those same pairs, so it adds information beyond knowing the compound elsewhere. What these internal numbers do not measure is performance on chemistry absent from the corpus entirely; the external section speaks to that.

Ligands are withheld by Bemis-Murcko scaffold, keyed on the (sequence, ligand) pair. Every kinase sequence remains in training. No protein and no sequence family is withheld, because a kinase for which no data exists is not the deployment scenario: the kinome is among the most exhaustively characterised regions of the proteome. Evaluation covers 2,965,273 held-out ligand pairs across 413 targets. The kinase is the unit of replication for the kinase-level mean and its standard error, and only for those. Pooled accuracy, the margin deciles and the selectivity figures are computed over pairs that share compounds and are therefore not independent observations; treat them as descriptive rather than as quantities with an interval.

What this model does not use

The headline results are obtained without three bodies of work that are built and available: 4,555 CANDIDATE orthologue sequences across 475 organisms, screened by binding-site identity; per-residue contact maps from 1,417 solved co-complex structures covering 170 targets, with 977 identified ligands and 182 sites excluded because a nucleotide or crystallisation additive rather than an inhibitor defined them; and 821,370 BindingDB measurements retained after removing every compound-target pair appearing in the frozen ChEMBL benchmarks. These are headroom, not caveats.

Data sources

Kinase Knowledgebase (KKB), Eidogen-Sertanty, release Q2-2026, the training and evaluation corpus: eidogen-sertanty.com/kinasekbmarvin.php.

BindingDB, ingested and held for future arms, not used in the results reported here. Gilson et al., Nucleic Acids Res. 44:D1045 (2016), doi:10.1093/nar/gkv1072; bindingdb.org.

ChEMBL, used only as a frozen external benchmark, never trained on; every compound-target pair appearing in it was removed from the BindingDB import. Zdrazil et al., Nucleic Acids Res. 52:D1180 (2024), doi:10.1093/nar/gkad1004; ebi.ac.uk/chembl.

RCSB Protein Data Bank, co-complex structures used for the binding-site analysis reported separately. Berman et al., Nucleic Acids Res. 28:235 (2000), doi:10.1093/nar/28.1.235; rcsb.org.

UniProt, sequence and accession mapping. UniProt Consortium, Nucleic Acids Res. 51:D523 (2023), doi:10.1093/nar/gkac1052; uniprot.org.

Models and software

ESM2 protein language model, checkpoint facebook/esm2_t12_35M_UR50D, 35M parameters, 12 layers, 480-dimensional per-residue representations. Lin et al., Science 379:1123 (2023), doi:10.1126/science.ade2574; code at github.com/facebookresearch/esm.

ChemBERTa chemical language model, checkpoint seyonec/ChemBERTa-zinc-base-v1, 768-dimensional embeddings, used in the comparison arm. Chithrananda et al., arXiv:2010.09885 (2020), arxiv.org/abs/2010.09885.

Morgan / ECFP count fingerprints, radius 2, 1,024 bits, and all physicochemical descriptors computed with RDKit: rdkit.org. Method: Rogers & Hahn, J. Chem. Inf. Model. 50:742 (2010), doi:10.1021/ci100050t.

Bemis-Murcko scaffolds, used to define the held-out ligand split. Bemis & Murcko, J. Med. Chem. 39:2887 (1996), doi:10.1021/jm9602928.

Random forest regressor, scikit-learn: scikit-learn.org. Breiman, Machine Learning 45:5 (2001), doi:10.1023/A:1010933404324.

Pairwise ranking formulation: Burges et al., Learning to Rank using Gradient Descent, ICML 2005, doi:10.1145/1102351.1102363.

Matthews correlation coefficient: Matthews, Biochim. Biophys. Acta 405:442 (1975), doi:10.1016/0005-2795(75)90109-9.

The two released models: Validated and Frontier

A model cannot be measured on data it was trained on. Maximum evidence and maximum data therefore cannot be the same object, so both are released rather than quietly choosing one.

417,507
VALIDATED model, training measurements. Every accuracy in this report was produced by this model, on data it never saw.
780,067
FRONTIER model, training measurements. Every measurement the project holds. No accuracy exists for it and none can.
If you are deploying, read only this paragraph. The VALIDATED model, bundle kfm_ranker_v1_final, was trained on 417,507 measurements. The FRONTIER model, bundle kfm_ranker_production, was trained on 780,067 measurements, which is every measurement in the corpus. If the model you are holding was trained on roughly four hundred thousand measurements it is the VALIDATED one; roughly eight hundred thousand means it is the FRONTIER one. Each bundle states its own count in config.json under trained_on, and its own release name under release. Check that field rather than the directory name.
VALIDATED
kfm_ranker_v1_final
FRONTIER
kfm_ranker_production
measurements trained on417,507every measurement held
data usedtraining half, exact values onlyboth halves, exact values and limit-only records
architectureidentical: 200 trees, same features, same seed, same training method
measured accuracy74.0% per target, 90.3% on decisive pairsnone, and none is possible
use it whenyou need a number you can defendyou want the most informed prediction available
Every number in this document is the Validated model. It was measured on 2,965,273 held-out pairs it never saw, and the file shipped is the file that produced those numbers. Nothing here describes the Frontier model, because anything it scored would be data it had already learned.
The Frontier model is untested, by construction and on purpose. It uses the same training method on more evidence: the held-out half, and the limit-only records that state a compound is weaker than the concentration tested. More data with an unchanged method should help, and that expectation is reasonable rather than measured. It is offered for use, not as a claim.

How to use it, and where it stops

How to use it

Point it at a kinase you already have data on and a compound library you have not measured. It scores every compound in one pass and returns a ranked list, each comparison carrying a confidence. Act on the confident fraction: accuracy rises steeply as you narrow to the calls the model is most certain about, so the operating point is yours to choose.

Check the target first. This is the single most important step and it is not optional. Performance varies more between targets than it does between any two models we have tried. On its strongest targets AURKA, CDK1, MTOR, SRC, PIM1, ABL1, CDK2, BRAF, it calls decisive pairs correctly around nine times in ten on chemistry from a source it has never seen. On its weakest it is close enough to a coin flip that a prediction should carry no weight. The tier table above is the guide, and there is no aggregate number that substitutes for reading it.

Where it stops

It ranks; it does not predict absolute potency. Two compounds whose true potencies differ by less than two publications routinely disagree by cannot be reliably adjudicated from this assay corpus, and the model is correctly close to chance on them rather than confidently wrong.

Where a compound already has measured potency against a related kinase, use that measurement: it is direct evidence. It is not, however, a stronger predictor than the model on this benchmark, that lookup scores 73.1 percent where it is defined, against the model's 76.8 percent on the same pairs. The lookup was undefined on 49.7 percent of evaluation pairs, and it is there that the model is the only option.

It is a triage tool that runs before docking or crystallography, not a replacement for either, and not a substitute for measurement.

A real limitation, stated plainly. Records that say only that a compound is weaker than some tested concentration are excluded from this model, in training and in evaluation. That is 309,111 of the source rows. They carry most of the evidence that a compound fails to bind, so the model has been shown far fewer non-binders than the corpus contains. An earlier version of this document claimed the opposite; the claim was wrong and is withdrawn.

What this model does

It sorts a compound library against a kinase you already have data on, and reports a margin on each call that tracks how often it is right. Used as a filter on its confident predictions it is right 88 percent of the time on chemistry from a source it has never seen. A coin flip is 50%.

Scores a pair in milliseconds with no structure needed, against minutes to hours for docking, so it triages a library before any structure-based work begins. Structural evidence behind the binding-site work.

74%
Every held-out pair. Averaged per kinase over 413 targets and 2,965,273 comparisons. A coin flip is 50%.
90%
When the answer matters. Pairs whose measured potencies differ by more than tenfold.
99%
Largest model margins, compounds held out for that kinase. On compounds from an outside source the same decile falls to about 88.0 percent. Use the lower figure to plan.
3.6×
On chemistry it has never seen. Enrichment in the top tenth of a ranked library, against a random ordering, on 10,161 compounds from an outside source.

The complete experiment record, including every claim that did not survive review, is kept separately in collab/cycles/ and is deliberately not part of this document.

Generated from collab/cycles/*/claude_manifest.json by scripts/37_live_report.py · 08 August 2026 at 23:05