The article details presented on this page reflect the published version of record. Readers may access the full-text PDF, review the citation information, and export the reference in supported citation formats.
Abstract
The rapid advancement of large language models (LLMs) has opened up opportunities for AI applications in pharmaceutical sciences. However, integrating diverse biological data modalities continues to remain challenging. We propose SciMind, a multi-modal mixture-of-experts (MoE) having the capability of integrated representation learning from pharmaceutical data sources, including biomedical text, DNA sequence, protein sequence, and molecular structure. The proposed method includes method-specific tokenization strategies, sparse expert routing mechanisms, and cross-modal pre-training for improved knowledge transfer across multiple biological representations. An expert initialization approach based on limited K-means and adaptive top-k routing can use the parameters effectively while preserving domain-specific knowledge. Through experimental evaluations across four applications, including biomedical natural language processing, molecular understanding, promoter prediction and protein-related tasks, SciMind achieves competitive performance against existing domain-specific and general-purpose models. According to the results, unified multi-modal learning can assist in the representation quality, drug reasoning capabilities, and applications in drug acceptance, molecule analysis, and personalized medicine.
Keywords:Large Language ModelsMixture-Of-ExpertsPharmaceutical SciencesBioinformaticsMulti-Modal LearningRepresentation Learning
Cite this Article
Use the following citation when referencing this publication in academic or professional work.
A. Bhople, "A Unified Multi-Modal Mixture-of-Experts Model for Integrated Representation Learning in Pharmaceutical Sciences", Int. J. Clin. Res. Med. Sci., vol. 1, no. 2, Oct. 2026, doi: https://doi.org/10.67231/kytb0e28.
DOWNLOAD CITATION
A Unified Multi-Modal Mixture-of-Experts Model for Integrated Representation Learning in Pharmaceutical Sciences | International Journal of Clinical Research and Medical Sciences