LigASeqLigB_v2_potency

Potency ranking model, version 2. 1,024-bit Morgan baseline. Built and measured 10 August 2026.

Revision 1.2, 13 August 2026. Limitations and the novelty stratification of this test set: kinasefoundationmodel.com/v2/limitations.html. Removed a binomial standard-error statement: the comparisons share compounds and are drawn with replacement, so they are not independent observations and no such interval applies. Treat pooled figures as descriptive.

What the model does

Given two ligands and one protein sequence, it predicts which of the two ligands is more potent against that sequence. It is used to rank order molecules against a single target. No protein structure and no docked pose are used; the protein enters only as a sequence embedding.

How the model works

Input — one row of 2,556 numbers, in this exact order

row = [ ligand A (1038) | sequence (480) | ligand B (1038) ]

The 14 descriptors follow the fingerprint in this fixed order: MolWt, HeavyAtomCount, NumBonds, NumRotatableBonds, RingCount, then the atom counts of C, N, O, S, F, Cl, Br, I, P. Blocks 1 and 3 use the same encoder, so a ligand produces the same 1,038 numbers wherever it appears.

Output

A RandomForestClassifier with two classes. predict_proba returns two values that sum to 1, in the order [p(0), p(1)].

Swapping the two ligands reverses the answer. Measured on the output values, the mean departure from exact reversal is 0.0271.

Training set

Source: the Eidogen-Sertanty Kinase Knowledgebase, Q2-2026 release (data/kkb_classifier_train_full.csv). No ChEMBL, no BindingDB, no other source. Targets are keyed by sequence, so a mutant is a target in its own right.

Every training comparison is entered twice, once as given and once with the two ligands and the label both reversed. Pairs whose two measurements are identical exact values are entered with the same label in both orders, so the model returns about 0.5 for them.

Test set

Source: ChEMBL, chembl_kkb_kinase_activities.csv, pulled 4 August 2026, 865,549 rows over 534 accessions. Structures for every compound were resolved from the ChEMBL API at 100% coverage.

Exclusion rule. A ChEMBL comparison is removed only if that exact ligand-target-ligand triple was actually drawn into training. Nothing else is withheld. A ligand is never excluded on its own account, and a comparison is kept even when both of its ligands appear elsewhere in KKB.

The scored set is drawn on an even quota per target so that heavily studied kinases do not dominate.

A note on aggregation, so no number here is misread. Every figure in this report is measured with single-order scoring, which is how the evaluation script scores each comparison. The shipped predict.py averages both ligand orders and measures 0.6928 on the identical 1,836,100 comparisons. The numbers below are therefore a conservative floor for what the deployed model does, never an overstatement. Same model, same weights, same test set; only the inference-time aggregation differs.

Results

Methods — how a model this size was trained

The full training set cannot be fitted in one pass. There are 2,229,954,204 ligand pairs on a shared target, and one row of 2,556 float32 values is 10,224 bytes, so even a single million rows is 10.2 GB. Fitting all of it at once would need terabytes.

The model was therefore trained incrementally, in chunks, and the resulting forests were pooled into one. A random forest is an ensemble of independently grown trees, so forests fitted on different samples can be merged by concatenating their trees. The merged model predicts by averaging votes over every tree it holds. This is ordinary bagging, with the bags drawn from a pool larger than memory rather than from a table that fits in it.

The procedure, run exactly as follows:

1. Draw about 1.33 million ligand pairs, on an even quota per target so that heavily screened kinases do not crowd out the rest. A target that cannot fill its quota contributes every pair it has, and the remainder is redistributed to targets that can.
2. Enter each pair twice, once as ligand A against ligand B and once reversed with the label reversed with it. That gives about 2.67 million rows, a design matrix of 25.4 GB.
3. Fit a 20-tree forest on that chunk, using every core, and write it to disk.
4. Discard the chunk from memory and repeat with a fresh draw under a new seed.
5. After 15 chunks, pool the 15 forests into a single 300-tree model.

Chunks were run sequentially, each using all cores. A single random forest fit already parallelises across cores, so running several fits concurrently would not have been faster and would have risked memory contention. Every chunk was checkpointed as it completed, so an interruption would have cost one chunk rather than the run. The 15 checkpoints are retained alongside the pooled model as evidence that it is the sum of the runs described.

The minimum leaf size of 20 is what keeps the artifact deliverable. Tree size grows as roughly twice the row count divided by the leaf floor, so the leaf size of 2 used in version 1.1 would have produced a pooled model in the tens of gigabytes rather than 0.76 GB.

Across the 15 chunks the model saw 15,256,017 distinct triples, which is 0.684% of the pairs that exist, but 302,863 of 302,999 ligands and 695 of 700 usable targets reached it. What is sampled is which pairings are used, not which ligands or which targets.

Prediction strength and accuracy

Prediction strength is the larger of the two output probabilities, so it runs from 0.5, a coin flip, to 1.0. Accuracy rises with it on both sets.

Acting only on predictions at or above a threshold. The point of this table is the trade: accuracy rises as the threshold rises, but the number of comparisons you can still answer falls away. The right cutoff is where the accuracy is worth the predictions you give up.

On the test set the accuracy gained per prediction surrendered falls off sharply after 0.70. Going from 0.50 to 0.70 buys 16 accuracy points and costs 80% of the answers. Going from 0.70 to 0.90 buys only 3 more points and costs 85% of what remained. On that reading 0.70 is the useful operating point: a fifth of all comparisons answered at 0.842 against a 0.680 baseline.

The relationship holds on unseen ChEMBL data: acting only on predictions at 0.7 or above covers a fifth of comparisons at 0.842 accuracy, against 0.680 across the board. Note that the strength value does not mean the same thing on the two sets. On the training pairs a 0.7 prediction is right 98% of the time; on ChEMBL the same 0.7 is right 84% of the time. The ordering is reliable, the absolute number is not transferable.

Test figures here are computed over 1,719,096 comparisons drawn under the same even-quota rule; the headline 0.690 is over the 1,836,100-comparison draw in the Results section. The two draws differ only by sampling.

Worked example

One comparison, end to end, drawn from the training set. The target is ABL1; the RCSB entry 3UE4 is the ABL1 kinase domain in complex with bosutinib, which is ligand A here. The classic ABL1 structure 2HYY, the kinase domain with imatinib, is an alternative reference for the same protein.

The two molecules share almost no substructure: Tanimoto similarity 0.075 on Morgan radius-2 fingerprints. The measured potency gap is 4.46 log units.

The model picks bosutinib, which is correct. Swapping the two ligands moves the output from 0.846 to 0.151; the two sum to 0.997 against an ideal 1.000, so the answer reverses as the architecture requires.

Model

Each fit used about 1.33 million pairs, 2.67 million rows after the swap. The 15 forests were then merged into one by pooling their trees, so the single delivered model averages votes over all 300.

Authority

Every number above comes from data/LigASeqLigB_v2_potency_results.json, data/LigASeqLigB_v2_potency_insample_and_exclusion.json and the audit in scripts/235_ligaseqligb_v2_potency_data_audit.py. The specification is LIGASEQLIGB_SPEC.md.