bioRxiv preprint doi: https://doi.org/10.64898/2026.05.28.728612
1. Introduction
ADMET property prediction is an indispensable component of modern drug discovery
pipelines, enabling early identification of liabilities related to pharmacokinetics and
toxicity before experimental validation. Despite substantial progress in computational
chemistry, predictive modeling of ADMET endpoints remains challenging due to the
inherent heterogeneity and complexity of molecular data. Molecules exhibit multi-scale
structure–property relationships, ranging from atom-level connectivity and local
substructures to global physicochemical characteristics, that are not fully captured by any
single data modality.
Recent advances in deep learning for molecular representation have introduced
architectures tailored to specific input formats. Graph neural networks (GNNs) efficiently
encode topological and relational information from molecular graphs, while Transformer[1]based models capture global dependencies across fingerprint features, including large-scale
molecular language models such as MolFormer [1-6]. Similarly, descriptor-based neural
networks can leverage curated physicochemical features such as hydrophobicity, polar
surface area, and molecular weight [7-8]. However, single-modality models are inherently
limited in their representational capacity, as they often overlook cross-modal correlations
and complementary information across feature spaces [9-12].
To address this limitation, we propose a multimodal ADMET prediction framework that
integrates three complementary molecular representations: (1) MACCS structural
fingerprints, which encode fragment-level structural patterns; (2) graph-based molecular
representations, which capture atomic connectivity and local neighborhood interactions;
and (3) physicochemical descriptors derived from RDKit, PaDEL, and Mordred, which
summarize global molecular properties. The modality-specific embeddings are
subsequently integrated through a fusion network that learns complementary structure–
property relationships from these heterogeneous molecular representations.
Beyond predictive performance, an important objective of this work is to improve the
interpretability and reliability of ADMET prediction models. To this end, DMPKformer
incorporates modality-specific attention mechanisms for substructure-level interpretation
and a latent-space out-of-distribution (OOD) confidence estimation framework to identify
regions of reliable model operation. This combination of multimodal representation
learning, interpretability, and reliability-aware prediction distinguishes DMPKformer from
purely performance-driven ADMET benchmarking models.
available under a CC-BY-NC-ND 4.0 International license.
(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made
bioRxiv preprint doi: https://doi.org/10.64898/2026.05.28.728612; this version posted May 29, 2026. The copyright holder for this preprint
2. Methodology
2.1 Data Sources
All datasets used in this study were obtained from the Therapeutics Data Commons (TDC),
a standardized benchmark suite for machine learning in drug discovery. TDC provides a
diverse collection of datasets spanning multiple ADMET (Absorption, Distribution,
Metabolism, Excretion, and Toxicity) endpoints, along with predefined data splits to
facilitate consistent and reproducible evaluation [13].
The use of these standardized splits ensures consistency with prior benchmarks and enables
fair comparison across different modelling approaches and the distribution of the number
of datapoints across various ADMET endpoints is graphically depicted as below in Fig. 1
Fig. 1 Data distribution of TDC across various ADMET endpoints
2.2 Model Architecture
The proposed architecture consists of three independent modality-specific subnetworks
and a final multimodal fusion network which is graphically depicted as below
available under a CC-BY-NC-ND 4.0 International license.
(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made
bioRxiv preprint doi: https://doi.org/10.64898/2026.05.28.728612; this version posted May 29, 2026. The copyright holder for this preprint
Fig. 2 Overall architectural design of DMPKformer
2.2.1 MACCS Fingerprint Track Architecture
The MACCS track is designed to extract expressive, task-adaptive representations from
fixed-length binary molecular fingerprints, specifically the MACCS structural keys. Each
molecule is represented as a 166-bit binary vector, where each bit encodes the presence or
absence of a predefined chemical substructure or functional motif. While MACCS
fingerprints are traditionally used with linear or shallow models, their fixed semantics and
compact dimensionality make them well suited for contextual modelling via attention
mechanisms. The architectural details of the MACCS self-attention transformer within the
DMPKformer framework is given in Fig.3
Input Representation and Embedding
The raw MACCS fingerprint, a binary vector of length 166, is first reshaped into a sequence
of length 166 with a single feature channel per position. Each bit position is treated
analogously to a token in a sequence, enabling the model to learn interactions between
structural keys. A learnable linear embedding layer maps each scalar bit value into a
continuous embedding space of dimension ��model. This projection allows the network to
represent both the presence of a structural feature and its contextual role relative to other
features.
To preserve positional identity, since each MACCS bit corresponds to a specific chemical
key, a fixed sinusoidal positional encoding is added to the embedded sequence. This
ensures that the Transformer encoder can distinguish between different fingerprint indices
while remaining invariant to batch ordering.
available under a CC-BY-NC-ND 4.0 International license.
(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made
bioRxiv preprint doi: https://doi.org/10.64898/2026.05.28.728612; this version posted May 29, 2026. The copyright holder for this preprint
Transformer Encoder for Inter-Bit Dependency Modelling
The embedded fingerprint sequence is processed by a stack of Transformer encoder blocks,
each consisting of a multi-head self-attention module followed by a position-wise feed[1]forward network. Unlike conventional fingerprint models that treat each bit independently
or aggregate them linearly, self-attention enables the MACCS track to model higher-order
dependencies between structural keys. This is particularly important in cheminformatics,
where the predictive relevance of a substructure often depends on the presence or absence
of other motifs elsewhere in the molecule.
Multi-head attention allows the model to attend to multiple interaction patterns in parallel,
capturing diverse relationships such as co-occurrence of functional groups, mutually
exclusive fragments, or long-range structural complementarities. The feed-forward
sublayers further transform these context-enriched representations, while residual
connections and normalization stabilize training and preserve information flow across
layers.
Global Attention Pooling and Feature Attribution
Following the Transformer encoder, a global attention pooling mechanism aggregates the
per-bit representations into a fixed-length molecular vector. This pooling layer learns a
gating function that assigns an importance weight to each fingerprint position, effectively
performing a soft selection over MACCS keys. The gated representations are then summed
to produce a compact, molecule-level descriptor.
In addition to pooling, the attention weights are retained as per-bit importance scores,
providing interpretability by highlighting which structural keys contribute most strongly to
the model’s prediction. This aligns well with the interpretability expectations of
fingerprint-based models in medicinal chemistry.
Latent Embedding and Prediction Head
The pooled molecular representation is passed through a normalization and non-linear
activation stage to produce a latent embedding vector, denoted ��MACCS. This embedding
serves as a task-conditioned fingerprint representation that captures both local structural
information and global context learned through attention. ��MACCS can be used
independently for downstream analysis, similarity comparisons, or as an input to
multimodal fusion architectures.
For prediction, the latent embedding is processed by a dense feed-forward head composed
of one or more fully connected layers with non-linear activation and dropout regularization.
The final output layer produces a single logit, which is passed through a sigmoid activation
to yield a probability for binary ADMET classification tasks. Training is performed using
available under a CC-BY-NC-ND 4.0 International license.
(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made
bioRxiv preprint doi: https://doi.org/10.64898/2026.05.28.728612; this version posted May 29, 2026. The copyright holder for this preprint
a weighted binary cross-entropy objective to account for class imbalance commonly
observed in ADMET datasets.
Fig. 3 Architectural details of MACCS transformer
2.2.2 Molecular Graph Track Architecture
The Graph Track is designed to learn chemically grounded molecular representations
directly from molecular structure, encoded as graphs derived from SMILES strings. Unlike
fixed fingerprints, graph-based representations preserve explicit atomic connectivity and
local chemical environments, allowing the model to reason about molecular structure in a
manner that is closely aligned with physical chemistry. The architecture details of the
Graph Attention Transformer adapted for molecular property prediction task within the
DMPKformer framework is given in Fig. 4
Graph Construction and Feature Encoding
Each molecule is represented as a labelled graph �� = (��, ��), where nodes ��correspond to
atoms and edges ��correspond to chemical bonds. Atom-level features are constructed
using numerical descriptors that capture intrinsic chemical properties, such as atomic
number, formal charge, degree, hybridization state, valence, aromaticity, and ring
available under a CC-BY-NC-ND 4.0 International license.
(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made
bioRxiv preprint doi: https://doi.org/10.64898/2026.05.28.728612; this version posted May 29, 2026. The copyright holder for this preprint
membership. These features are encoded as continuous or categorical values rather than
one-hot vectors, enabling compact representations and smoother optimization.
Bond-level features encode the nature of chemical interactions between atoms, including
bond order (single, double, triple), aromaticity, conjugation, and ring status. Together,
these atom and bond descriptors provide a rich, physics-informed representation of
molecular structure while remaining invariant to atom ordering and SMILES
canonicalization.
Graph Attention Network for Local and Contextual Message Passing
The molecular graphs are processed using a Graph Attention Network (GAT), which
performs message passing between neighbouring atoms while learning to weight the
relative importance of different chemical interactions. In each GAT layer, node
embeddings are updated by aggregating features from neighbouring nodes, where the
contribution of each neighbour is modulated by a learned attention coefficient. These
coefficients are computed as a function of both the source and target node representations
(and, optionally, edge features), allowing the network to prioritize chemically relevant
interactions such as polar bonds, aromatic systems, or functional group connectivity.
Multi-head attention is employed to enable the model to capture diverse interaction patterns
simultaneously. Each attention head can focus on different aspects of the local chemical
environment, such as steric effects, electronic properties, or substructure motifs. The
outputs of multiple heads are concatenated or averaged to form the updated node
embeddings, improving representational capacity and robustness.
By stacking multiple GAT layers, the model progressively expands the receptive field of
each atom, allowing information to propagate across larger molecular substructures. This
hierarchical message passing enables the Graph Track to capture both local atomic
environments and higher-order structural motifs such as rings, scaffolds, and functional
group arrangements.
Global Attention Pooling and Molecular Representation
After graph-level message passing, node embeddings are aggregated into a fixed-length
molecular representation using a global attention pooling mechanism. Rather than simply
averaging or summing node features, attention pooling learns a weighting over atoms that
reflects their relative importance to the prediction task. This allows the model to emphasize
chemically salient atoms—such as reactive centres, pharmacophores, or key substituents—
while down-weighting less informative regions of the molecule.
The pooling operation produces a single vector representation for each molecule that
integrates information across the entire graph while retaining interpretability via learned
available under a CC-BY-NC-ND 4.0 International license.
(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made
bioRxiv preprint doi: https://doi.org/10.64898/2026.05.28.728612; this version posted May 29, 2026. The copyright holder for this preprint
attention weights. These weights can be inspected post hoc to identify which atoms or
substructures most strongly influenced the model’s prediction, supporting mechanistic
interpretation and hypothesis generation.
Prediction Head and Latent Graph Embedding
The pooled molecular graph representation is passed through a dense feed-forward
prediction head composed of fully connected layers with non-linear activations and
regularization. The final layer outputs a single logit, which is transformed via a sigmoid
activation to produce a probability for binary classification tasks such as ADMET property
prediction.
The intermediate pooled graph vector, extracted prior to the final prediction layer, is
denoted as ��Graph . This latent embedding captures a task-conditioned, structure-aware
representation of the molecule that reflects both atomic-level chemistry and global
molecular context. ��Graphcan be used independently for downstream analyses, similarity
comparisons, or integration with other molecular modalities.
Fig. 4 Architectural details of Graph Attention Transformer for molecular property
prediction
2.2.3 Physicochemical Descriptor Track
Physicochemical and topological descriptors are computed from RDKit, PaDEL, and
Mordred, providing high-dimensional continuous features summarizing global molecular
properties. These features are fed into a tanh-activated multilayer perceptron (MLP)
network that captures nonlinear dependencies among descriptors. An embedding vector
available under a CC-BY-NC-ND 4.0 International license.
(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made
bioRxiv preprint doi: https://doi.org/10.64898/2026.05.28.728612; this version posted May 29, 2026. The copyright holder for this preprint
from an internal dense layer serves as E_Desc, representing the learned descriptor-level
representation. The architectural details of the descriptor track is provided in Fig. 5
Fig. 5 Architectural details of physicochemical descriptor track in the DMPKformer
framework
2.3 Phase-wise Training Strategy
Each subnetwork is independently trained for binary classification using a dataset of
molecules annotated with the target ADMET label. Binary cross-entropy loss is used for
optimization, and L2 regularization and dropout are applied to prevent overfitting. Once
individual models converge, intermediate embeddings E_MACCS, E_Graph, and E_Desc
are extracted for all molecules.
These embeddings form the input to the fusion network, where they are concatenated and
passed through a tanh-activated dense network that models inter-modality interactions. The
final classification layer produces the unified ADMET prediction output. During this
phase, the modality encoders remain frozen to preserve previously learned representations,
and only the fusion network is trained.
In the section that follows we present and discuss the results obtained for the independent
tracks and the combined track and compare them against state of art benchmarks and
demonstrate improvement over state of art.
available under a CC-BY-NC-ND 4.0 International license.
(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made
bioRxiv preprint doi: https://doi.org/10.64898/2026.05.28.728612; this version posted May 29, 2026. The copyright holder for this preprint
3. Results and Discussion
The following section presents the predictive performance of DMPKformer across
multiple TDC benchmarks together with an analysis of the interpretability provided by
the framework. In addition, we introduce a latent-space out-of-distribution (OOD) metric
for TDC tasks that enables reliability-aware confidence estimation and identification of
high-confidence regions of model operation.
3.1 Track-wise results of DMPKformer
The results from the MACCS self-attention transformer, Graph Attention transformer and
the physicochemical descriptor track are presented below for the classifications and
regression tasks in TDC in table 1a, 1b, 2a, 2b & 3a, 3b respectively.