INTERNATIONAL JOURNAL OF CLINICAL RESEARCH AND MEDICAL SCIENCES

A Unified Multi-Modal Mixture-of-Experts Model for Integrated Representation Learning in Pharmaceutical Sciences

1.New York University

Published in issue

October 2026

Volume / Issue

Vol. 1 No. 2

Access

Open access

View PDF

The article details presented on this page reflect the published version of record. Readers may access the full-text PDF, review the citation information, and export the reference in supported citation formats.

Abstract

The rapid advancement of large language models (LLMs) has opened up opportunities for AI applications in pharmaceutical sciences. However, integrating diverse biological data modalities continues to remain challenging.  We propose SciMind, a multi-modal mixture-of-experts (MoE) having the capability of integrated representation learning from pharmaceutical data sources, including biomedical text, DNA sequence, protein sequence, and molecular structure. The proposed method includes method-specific tokenization strategies, sparse expert routing mechanisms, and cross-modal pre-training for improved knowledge transfer across multiple biological representations. An expert initialization approach based on limited K-means and adaptive top-k routing can use the parameters effectively while preserving domain-specific knowledge. Through experimental evaluations across four applications, including biomedical natural language processing, molecular understanding, promoter prediction and protein-related tasks, SciMind achieves competitive performance against existing domain-specific and general-purpose models.  According to the results, unified multi-modal learning can assist in the representation quality, drug reasoning capabilities, and applications in drug acceptance, molecule analysis, and personalized medicine.
Keywords:Large Language ModelsMixture-Of-ExpertsPharmaceutical SciencesBioinformaticsMulti-Modal LearningRepresentation Learning

Cite this Article

Use the following citation when referencing this publication in academic or professional work.

A. Bhople, "A Unified Multi-Modal Mixture-of-Experts Model for Integrated Representation Learning in Pharmaceutical Sciences", Int. J. Clin. Res. Med. Sci., vol. 1, no. 2, Oct. 2026, doi: https://doi.org/10.67231/kytb0e28.

DOWNLOAD CITATION

A Unified Multi-Modal Mixture-of-Experts Model for Integrated Representation Learning in Pharmaceutical Sciences | International Journal of Clinical Research and Medical Sciences