Traceable to What?

Every AI tool in drug discovery now says it can show you its evidence. That is real progress, but I think the more useful question is where that evidence comes from.

I am glad the field has landed here. Not long ago the summary of a prediction was "the model says so". Once you can see the evidence, though, there is an obvious follow-up that we do not ask often enough, which is evidence from where?

What are the two kinds of evidence behind an AI prediction?

The first kind is curated knowledge, meaning findings taken from published papers that experts have reviewed and structured. It is reliable, well annotated and easy to interpret, and I would not want to work without it. Its limit comes from how it is made rather than from its quality, because it can only contain what has already been studied, and that work is not spread evenly. Using Claude Science, I pulled the current gene-to-publication links from NCBI. Across 20,606 human protein-coding genes, the top 1 percent account for 25.6% of all links, half of the entire literature sits on just 1,080 genes, and another 1,150 genes have no linked paper at all.

That skew does not tell us which genes matter most for biology. Stoeger and colleagues showed that the attention a human gene receives can largely be predicted from a small set of its own chemical, physical and biological properties, together with what was already known from model organisms, and the same is true for the year it was first reported and the funding it attracts. Haynes and colleagues put it more directly, finding that annotation inequality leads researchers toward richly annotated genes rather than toward the genes with the strongest molecular data.

The second kind is data-derived evidence or patterns computed directly from experimental measurements, such as gene expression across many independent studies. It can surface relationships that nobody has written about yet, but it has limits of its own. Gene co-expression only tells you that genes move together across samples, not that one causes the other. It is association, so it has to be reproducible before you trust it and validated before it counts as knowledge.

Why does the source of evidence matter before you commit to an experiment?

A hypothesis built only from what is already known tends to lead back to what is already known. The most expensive moment in an R&D program is when you commit months of lab work to one direction, and that is exactly when you should ask whether the data support it independently of the literature.

There is a well documented example of this. In 2013, a group built gene-regulatory networks from 1,647 postmortem brain samples and ranked the resulting modules by their relevance to late-onset Alzheimer's disease. The top-ranked module was immune and microglial, and the data nominated TYROBP as its key regulator. If you search PubMed for TYROBP or DAP12 together with Alzheimer's in the title or abstract, you find two papers up to the end of 2012 and 160 from 2014 onwards. Its signalling partner TREM2, which human genetics implicated in the same few months, goes from 11 papers to 1,408. To be fair, genetics and network analysis reached the same biology at almost the same time and from independent directions, and the genetics drove much of the attention that followed. This means that the signal was in the data before the literature described it, and a tool reasoning only over the text available in 2012 could not have put that module in front of you.

We recently ran a similar test on our own models. In a brain disease study with an academic group, we took 35 findings from our network analyses, wrote out the proposition each one would need to already exist in the literature, and searched for it in full text. Four were directly confirmed. Twenty-one recovered an established axis but added a specificity that the source papers do not state. Ten were not described anywhere we could find, and eight further candidates looked novel until we searched again with alternative gene names. Most of the ten that survived were statements about edges rather than genes, such as which partner a gene keeps, in which context, and whether that partnership holds. The literature records gene and disease associations, but it does not record context-conditional wiring, because no sentence anywhere says it.

What does trustworthy data-derived evidence look like?

For me it comes down to four things. The evidence has to be reproducible, meaning a relationship holds across many independent studies, labs and sources of noise rather than in a single dataset. Ballouz and colleagues found that aggregating across experiments is what drives co-expression quality, and that thousands of samples are needed to approach a gold-standard network. It has to be large enough that random correlations wash out. It has to be traceable at the level of each connection, so you can see which datasets support it and how strongly. And it has to be honest about what it is, which is a map of which genes consistently work together and a starting point for validation rather than a final answer.

This is the standard that Mavatar Discovery was built against. It is based on Deep Integrated Network Analysis (DINA), our approach of building context-specific co-expression networks by integrating more than 350,000 curated transcriptomes from many independent studies, technologies and conditions. Every connection is weighted by how many independent datasets support it, so you can inspect the evidence behind an edge rather than taking the network's word for it.

How should you use curated knowledge and data-derived evidence together?

Curated knowledge tells you what is known, and data shows you what the biology looks like across many studies. The strongest hypotheses are the ones where both point the same way, and the most interesting ones are where the data points somewhere the literature has not been yet. So when you check the evidence behind an AI answer, do you know which one you are looking at?

Our networks are open to explore at https://discovery.mavatar.com/landing

Would you like to discuss a project? Contact us at info@mavatar.com to talk to our team

Frequently asked questions

What is the difference between curated knowledge and data-derived evidence in drug discovery?

Curated knowledge also known as literature mining consists of findings from published papers that experts have reviewed and structured, so it can only contain what has already been studied. Data-derived evidence is computed directly from experimental measurements, such as gene expression across many independent studies, and can surface relationships that nobody has written about yet.

Why is literature-based evidence biased toward some genes?

Research attention is very unevenly spread. Across 20,606 human protein-coding genes, the top 1 percent account for 25.6% of all gene-to-publication links, half of the literature sits on 1,080 genes, and 1,150 genes have no linked paper at all. The attention a gene receives can largely be predicted from properties of the gene itself rather than from its biological importance.

What is Deep Integrated Network Analysis (DINA)?

DINA is the approach behind Mavatar Discovery. It builds context-specific gene co-expression networks by integrating more than 350,000 curated transcriptomes from independent studies, technologies and conditions. Unlike single-dataset correlation networks, DINA is built by aggregating relationships across many independent studies, retaining only the connections that are reproducible across biological contexts.

Data analysis and PubMed queries were performed with Claude Science; draft revisions were assisted by Claude (Anthropic). The author reviewed and takes responsibility for the final content.

References

Stoeger T, Gerlach M, Morimoto RI, Nunes Amaral LA. Large-scale investigation of the reasons why potentially important genes are ignored. PLoS Biol 2018;16(9):e2006643. doi:10.1371/journal.pbio.2006643

Haynes WA, Tomczak A, Khatri P. Gene annotation bias impedes biomedical research. Sci Rep 2018;8:1362. doi:10.1038/s41598-018-19333-x

Edwards AM, Isserlin R, Bader GD, Frye SV, Willson TM, Yu FH. Too many roads not taken. Nature 2011;470:163-165. doi:10.1038/470163a

Zhang B, Gaiteri C, Bodea LG, et al. Integrated systems approach identifies genetic nodes and networks in late-onset Alzheimer's disease. Cell 2013;153(3):707-720. doi:10.1016/j.cell.2013.03.030

Guerreiro R, Wojtas A, Bras J, et al. TREM2 variants in Alzheimer's disease. N Engl J Med 2013;368:117-127. doi: 10.1056/nejmoa1211851

Jonsson T, Stefansson H, Steinberg S, et al. Variant of TREM2 associated with the risk of Alzheimer's disease. N Engl J Med 2013;368:107-116. doi: 10.1056/nejmoa1211103

Ballouz S, Verleyen W, Gillis J. Guidance for RNA-seq co-expression network construction and analysis: safety in numbers. Bioinformatics 2015;31(13):2123-2130. doi:10.1093/bioinformatics/btv118

van Dam S, Vosa U, van der Graaf A, Franke L, de Magalhaes JP. Gene co-expression analysis for functional classification and gene-disease predictions. Brief Bioinform 2018;19(4):575-592. doi: 10.1093/bib/bbw139

Lee HK, Hsu AK, Sajdak J, Qin J, Pavlidis P. Coexpression analysis of human genes across many microarray data sets. Genome Res 2004;14(6):1085-1094. doi: 10.1101/gr.1910904