SEARCH · PubMed Health

Results for “Multimodal”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2Linked to original sources

PubMed2026

mmContext: an open framework for multimodal contrastive learning of omics and text data.

SUMMARY: Multimodal approaches are increasingly leveraged for integrating omics data with textual biological knowledge. Yet there is still no accessible, standardized framework that enables systematic comparison of omics representations with different text encoders within a unified workflow. We present mmContext, a lightweight and extensible multimodal embedding framework built on top of the open-source Sentence Transformers library. The software allows researchers to train or apply models that jointly embed omics and text data using any numeric representation stored in an AnnData.obsm layer and any text encoder available in Hugging Face. mmContext supports integration of diverse biological text sources and provides pipelines for training, evaluation, and data preparation. We train and evaluate models for a RNA-Seq and text integration task, and demonstrate their utility through zero-shot classification of cell types and diseases across four independent datasets. By releasing all models, datasets, and tutorials openly, mmContext enables reproducible and accessible multimodal learning for omics-text integration. AVAILABILITY AND IMPLEMENTATION: Pretrained checkpoints and full source code for our custom MMContextEncoder are available on Hugging Face huggingface.co/jo-mengr. The Python package github.com/mengerj/mmcontext provides the model implementation and training and evaluation scripts for custom training. The releases for the publication can be accessed via zenodo: adata_hf_datasets: doi.org/10.5281/zenodo.19185217 and mmContext: doi.org/10.5281/zenodo.19185493.

Computational Biology↗

PubMed2026

Multimodal intervention benefits: Responder analysis of J-MINT PRIME Kanagawa trial.

INTRODUCTION: The J-MINT PRIME Kanagawa trial was an 18-month multimodal intervention (incorporating exercise, nutrition, and metabolic management) for dementia prevention. Because the primary analysis showed no significant benefits, we performed an exploratory responder analysis to identify responsive subpopulations. METHODS: We analyzed the Full Analysis Set comprising 188 participants. Classification and regression tree (CART) analysis, applied to the intervention arm, identified baseline predictors of cognitive improvement. These rules were then applied to the entire cohort to evaluate treatment effects on the Mini-Mental State Examination (MMSE) using fully adjusted mixed-effects models for repeated measures (MMRM). RESULTS: CART identified a "Target Group" (N = 108) characterized by baseline profiles such as an MMSE score < 28 or specific metabolic ranges (e.g., LDL-C < 135 mg/dL). Within this target group, the intervention significantly preserved MMSE trajectories compared with the control group (group × time interaction, P = 0.022). In contrast, the Non-Target Group (N = 80), consisting of high-functioning individuals (MMSE ≥ 28), exhibited no significant group × time interaction. DISCUSSION: Multimodal interventions may effectively preserve global cognition in older adults with sub-threshold cognitive decline. Careful targeting of appropriate populations, while considering potential longitudinal measurement artifacts (e.g., practice effects), is essential. These findings provide a hypothesis-generating framework that warrants external validation in future prevention trials.

Humans↗

PubMed2025

Diagnosing the undiagnosed: AI-enhanced multimodal modeling for placental mesenchymal dysplasia in high-risk pregnancies.

Placental mesenchymal dysplasia (PMD) is a rare vascular placental disorder that mimics molar pregnancy but often coexists with a viable fetus, making its misdiagnosis potentially devastating. In high-risk pregnancies, artificial intelligence (AI)-enhanced multimodal modeling - incorporating imaging, genomics, proteomics, and clinical features - offers a transformative diagnostic strategy. Leveraging Bayesian hyperparameter optimization for model refinement, this approach improves diagnostic accuracy while reducing uncertainty and clinician hesitation. Recent clinical studies support its efficacy and interpretability through SHAP and LIME models, while real-time surgical enhancements using Bayesian methods highlight its broader clinical utility. Despite current challenges such as data heterogeneity and integration barriers, multimodal AI provides unprecedented resolution in placental analysis, enabling precise differentiation between PMD and similar fetopathies. Ultimately, this advancement supports timely, non-invasive diagnosis, personalized management, and emotionally informed decision-making aligned with ethical AI implementation standards.

Bayesian optimization↗

PubMed1977

A computer program for fitting multimodal probability density functions.

A FORTRAN IV program is described, which may be run interactively with tutorial assistance or in batch and which allows a user to selectively fit any of seven probability density functions (p.d.f.'s) or a combination of the p.d.f.'s to a unimodal or multimodal histogram of empirical data. A "best-fit", uni- or multimodal p.d.f., which may be obtained by a method of nonlinear least squares or a generated p.d.f. may be displayed on a Tektronix 4010 terminal as a continuous curve against the background of a bar, square wave, symbol or point-plot histogram. The following, supportive statistical information is also displayed: (1) Kolmogorov-Smirnov probability of goodness of fit, (2) mean square error, (3) correlation coefficient, and (4) parameter estimates. The resident driver program and six overlayable segments have been implemented on a Digital Equipment Corporation LAB-11 minicomputer (PDP-11/20).

Computers↗

PubMed2026

Multimodal alignment improves generalizability of genomic biomarker prediction in computational pathology.

Computational pathology models that use digitized histopathology whole-slide images have the potential to become a cost-effective and scalable alternative to molecular assays for the prediction of genomic biomarkers, a key task in precision oncology. However, as new genomic biomarkers are discovered or quantified, large, labeled datasets must be prospectively collected to train new models. To address this challenge, we developed multimodal alignment for biomarker learning and generalization (MARBLE), a multimodal contrastive pretraining strategy that integrates structured biomarker knowledge into representation learning of histopathology images. MARBLE aligns histopathology-derived representations with representations of genomic biomarkers generated by a large language model (LLM) and a protein language model (PLM). This biologically informed alignment enables data-efficient generalization to novel, out-of-distribution biomarkers. Using the MSK-IMPACT cohort of over 40,000 patients across multiple biomarker panel versions, we design experiments grounded in real-world data to demonstrate the value of our proposed approach.

CP: computational biology↗

PubMed2026

Automated CEAP Classification of Venous Duplex Reports Using Multimodal Artificial Intelligence.

OBJECTIVE: To develop and internally validate a prototype multimodal artificial intelligence system for automated CEAP (Clinical, Etiological, Anatomical and Pathophysiological) classification of venous duplex ultrasound (VDUS) reports, integrating natural language processing of free-text components with computer vision analysis of hand-drawn anatomical diagrams. METHODS: Single centre retrospective observational study using routinely collected clinical data. One thousand consecutive venous duplex ultrasound reports from Cambridge University Hospitals NHS Foundation Trust, UK (July 2024 - May 2025) were labelled according to the CEAP classification, excluding the Etiological component, which could not be reliably determined from duplex reports alone. Transfer learning was applied using ClinicalBERT for text and MobileNetV3 for diagrammatic data. Clinical classes were predicted from request line text. Text- and image-based pathophysiological models were developed for four anatomical territories (Great Saphenous Vein, Small Saphenous Vein, Deep system, Perforators), combined using late fusion with probability averaging. RESULTS: The clinical CEAP model achieved accuracy of 0.91, macro-F1 of 0.82, and macro-AUC of 0.98. Pathophysiological prediction varied, with text models broadly outperforming image models. Fusion yielded heterogeneous benefits, improving SSV performance but reducing Deep system accuracy. The performance of the final pathophysiological CEAP fusion models varied across anatomical territories: accuracy ranged from 0.70-0.92 and macro-AUC from 0.80-0.92. CONCLUSION: This study demonstrates the feasibility of automated CEAP classification from VDUS reports. Despite class imbalance affecting minority class predictions, the strong discriminatory performance validates this multimodal ML model for extracting clinically meaningful information from real-world data. This approach offers potential, pending external validation, to streamline vascular services through automated triage and guideline-compliant decision making.

Artificial intelligence↗

PubMed2026

TCRspec: A Recognition Interface-Informed Multimodal Method for TCR-pMHC Specificity Prediction.

Specific recognition between T-cell receptors (TCRs) and peptide-major histocompatibility complexes (pMHCs) is central to adaptive immunity, yet accurate prediction of TCR-pMHC specificity remains challenging. Existing models mainly rely on sequence features or isolated molecular structures, limiting their ability to capture interface-level determinants within the ternary recognition complex. Here, we constructed the multimodal TCR-pMHC ternary complex (MM-TCR) data set, integrating paired TCR-pMHC sequences, V/J gene annotations, and modeled TCR-pMHC complex structures refined by short molecular dynamics-based relaxation. Based on MM-TCR, we developed TCRspec, an interpretable multimodal framework combining sequence embeddings, gene-usage features, and complex-level structural representations. Under a stringent CD-HIT TCR-cluster-disjoint split, TCRspec achieved an average AUROC of 0.896 and AUPRC of 0.882 across seven antigen-specific test data sets, outperforming representative baseline models. Cross-validation and ablation analyses confirmed the contribution of ternary complex structural information and MD-refined structures. In independent OOD peptide-TCR systems, TCRspec retained discriminative performance and identified model-inferred peptide positions associated with TCR recognition, providing a structure-informed framework for TCR specificity prediction.

Receptors, Antigen, T-Cell↗

PubMed2026

PASTA: versatile tyramide-oligonucleotide amplification for multimodal spatial biology.

Spatial proteomics is limited by detection sensitivity, multiplexing and multimodal integration, leaving a gap between discovery and clinical assays. Here we present protein and nucleic acid serial tyramide amplification (PASTA), using horseradish peroxidase-mediated oligonucleotide deposition and cyclical imaging for high-plex, multimodal spatial profiling. Compatible with conjugated antibodies and in situ hybridization, PASTA enables simultaneous protein and RNA codetection from formalin-fixed, paraffin-embedded samples, providing a cost-effective bridge from discovery to clinical validation.

Tyramine↗

PubMed2026

MethylModes: computationally efficient detection of multimodal distributions in DNA methylation data.

SUMMARY: MethylModes is an R package and Shiny application to identify multimodal distributions in human DNA methylation at individual CpG sites. Multimodal distributions, which can be the result of nearby genetic variation, environmental exposures, or assay artifacts, are susceptible to confounding and important to identify for methylation analysis. MethylModes is easily incorporated into existing quality control pipelines of array-based DNA methylation data. The underlying algorithm uses kernel smoothing of probe-level data to locate the number and location of peaks. The algorithm can be parallelized across probes for efficient implementation at genome-scale. We provide a case study implementation of MethylModes in the Health and Retirement Study as well as the Airwave Health Monitoring Study. AVAILABILITY AND IMPLEMENTATION: MethylModes is available on GitHub at https://github.com/lutiffan/methylModes as an R package wrapping an R Shiny application. We include a toy dataset to validate installation. The codebase is also published on Zenodo at https://doi.org/10.5281/zenodo.17448517.

DNA Methylation↗

PubMed2026

BriGHT: transcriptome-regularized multimodal neuroimaging for brain disorder prediction.

MOTIVATION: Hypergraph-based models for brain disorder prediction mainly adopt imaging-derived hypergraphs as propagation backbones. However, the entanglement of topology construction and feature propagation leaves regional representations weakly constrained by underlying biological organization, making them vulnerable to subject-specific variation and noise, particularly in heterogeneous multimodal settings. RESULTS: We present BriGHT, a Brain transcriptome-reGularized Hypergraph framework for mulTimodal disorder prediction. BriGHT employs a transcriptome-derived structural reference as a soft anchoring prior to regularize neuroimaging ROI embeddings, stabilizing representation geometry while preserving disease-relevant subject-specific variation. BriGHT further incorporates a reliability-aware fusion module to estimate subject-specific modality reliability from prediction confidence, cross-modal consistency, and decision certainty, enabling adaptive integration under heterogeneous modality quality. Experiments on three neuroimaging cohorts (ADNI, ADHD-200, REST-meta-MDD) and four modalities (VBM, fMRI, FDG, AV45) demonstrate that BriGHT consistently outperforms competing graph/hypergraph learning methods across six brain disorder prediction tasks. Perturbation analyses show that BriGHT benefits from the spatial correspondence between transcriptomic modules and imaging ROIs, rather than from arbitrary hypergraph regularization alone. Ablation and meta-analytic interpretability analyses support the contribution of transcriptomic anchoring and adaptive fusion to robust and biologically meaningful brain disorder prediction. AVAILABILITY: The software is publicly available at: https://github.com/Yaolab-fantastic/BriGHT. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Journal Article↗

PubMed2025

Alzheimer's subtypes A supervised, unsupervised, multimodal, multilayered embedded recursive (SUMMER) AI study.

Since Alzheimer's disease (AD) is a heterogeneous disease, different subtypes may have distinct biological, genetic, and clinical characteristics, requiring tailored interventions. While several proposed subtypes of AD exist, there is still no clear consensus on a definitive classification. By leveraging complementary AI approaches, including supervised and unsupervised learning, within a recursive pipeline (SUMMER) that integrates multimodal datasets encompassing MRI measurements, phenotypes, and genetic data, our goal was to generate robust scientific evidence for identifying AD subtypes. Data was downloaded from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database and included neuroimaging data (MRI), genetics (SNPs), clinical diagnosis, and demographics. 1133 European American participants' images, aged 55-95, were included in this study. The analysis was multi-fold, where the first step involved applying an unsupervised application to a subset of the MRI sample (AD + cognitively normal (CN) aged matched groups, 100 men aged 68-85 years, and 76 women aged 68-85 years). The MRI brain gray matter was segmented into 44 regions of interest (ROIs) according to a standard atlas, and 618 features were extracted, including ROI voxel intensity measurements such as minimum, maximum, and histogram variables. Results identified a cluster of subtype AD men and a cluster of subtype AD women that were distinct from the rest of their respective samples. In the next step, the integrity of the identified subtype AD clusters was investigated using the XGBoost supervised machine learning application with genetic features (SNPs, N=36,724) and labels: the identified subtype AD cluster vs. the rest of the sample, stratified by sex. A significant AD subtype men model (accuracy=0.85, F1=0.72, AUC=0.83) and a significant women AD subtype model (accuracy=0.81, F1=0.81, AUC=0.81) were built, confirming the homogeneity of the isolated AD subtype clusters. Discriminative biomarkers were extracted from the significant models, including selected ROIs and SNPs. Finally, the subtype models were tested on an unseen subset of ADNI data. The genetic-based models identified clusters of AD subtype participants consisting of 34% of the men AD group and 47% of the women AD group. Phenotypic analysis indicates that lower body weight was associated with the women's AD subtype. Complex diseases like AD demand a sophisticated, multimodal approach for precise diagnosis. Effectively identifying disease subtypes enhances the potential for personalized treatment, ultimately improving patient outcomes.

Journal Article↗

PubMed2026

Unifying multimodal single-cell data with a mixture-of-experts β-variational autoencoder framework.

Multimodal single-cell assays profile complementary layers of cell state, but integration is complicated by modality mismatch, sparsity, and uneven cohort coverage. Here, we present Unified Variational Inference (UniVI), a scalable mixture-of-experts β-variational autoencoder that learns a shared latent space while preserving modality-specific structure. UniVI couples modality-specific encoders/decoders with a shared latent prior and a symmetric cross-modal alignment objective, enabling consistent integration of paired measurements without curated feature-link graphs or preannotated reference atlases; optional supervised heads can be added when labels are available. Across paired RNA-protein (CITE-seq) and RNA-chromatin (10x Genomics Multiome, SHARE-seq) data spanning human PBMCs and mouse back skin-a nonhematopoietic tissue with continuous differentiation hierarchies-UniVI produces coherent embeddings, improves label transfer, and enables cross-modal reconstruction and denoising. Extending to trimodal measurements, UniVI maintains robust three-way alignment among RNA, chromatin accessibility, and surface proteins (TEA-seq), and accommodates DNA methylation in a paired scNMT-seq mouse gastrulation proof-of-concept under beta-binomial likelihoods. Performance degrades gracefully under severe cell type imbalance and in the presence of modality-exclusive populations. In an acute myeloid leukemia mosaic design, a paired RNA-protein bridge anchors independent RNA-only and protein+genotype cohorts, revealing genotype-associated neighborhoods that sharpen with mutation-aware fine-tuning. UniVI thus provides a flexible, interpretable framework for multimodal integration across paired, trimodal, and mosaic study designs and supports practical reference-to-query projection in partially observed studies.

Journal Article↗

PubMed2026

Multimodal Therapy With Metformin, Inositol and Dietary Restriction Improves Insulin Resistance and Endocrine Outcomes in Women With Polyendocrine Metabolic Ovarian Syndrome: A Randomized Controlled Trial.

INTRODUCTION: Polyendocrine metabolic ovarian syndrome (PMOS), formerly known as polycystic ovary syndrome (PCOS), is a common endocrine-metabolic disorder characterized by insulin resistance, hyperandrogenism and ovulatory dysfunction. Metformin, inositol supplementation and lifestyle modification are widely used treatments, but direct comparative evidence remains limited. Multimodal therapy combining metformin, inositol and dietary restriction produces greater metabolic and reproductive improvement than single-modality interventions. METHODS: We conducted a 12-week randomized controlled trial in 192 women aged 18-35 years diagnosed with PMOS according to Rotterdam criteria. Participants were allocated to metformin (1500-2000 mg/day), inositol (myo-inositol 2 g plus d-chiro-inositol 50 mg twice daily), calorie-restricted diet (1200-1500 kcal/day), or combination therapy. Primary outcomes included changes in body mass index (BMI) and insulin resistance assessed by HOMA-IR. Secondary outcomes included testosterone, LH/FSH ratio and menstrual regularity. Analysis was performed using analysis of covariance (ANCOVA), with post-intervention values as dependent variables and corresponding baseline values as covariates. Categorical outcomes were compared using the Chi-square test. RESULTS: All interventions improved metabolic and endocrine parameters. Combination therapy resulted in the greatest reduction in HOMA-IR (- 2.64, 95% CI - 2.82 to -2.46, p < 0.001) and BMI (- 2.8 kg/m2, 95% CI - 3.05 to -2.55, p < 0.001). Menstrual cyclicity improved across all groups, with the highest proportion of participants reporting cycle regularisation in the combination therapy group (85.4%), compared with dietary restriction (72.9%), inositol (64.6%), and metformin (39.6%) (p < 0.001). Given the short follow-up duration, these findings reflect early improvements rather than sustained normalisation. CONCLUSION: Multimodal therapy was associated with superior metabolic and reproductive outcomes compared with single-modality interventions in women with PMOS. CLINICAL TRIAL REGISTRATION: ClinicalTrials. gov (NCT07380841).

Humans↗

PubMed2026

Efficacy of a multimodal conservative rehabilitation program for bladder control in individuals with incomplete spinal cord injury: A randomized controlled trial.

ObjectiveThe study aimed to evaluate the efficacy of a multimodal intervention on urodynamic outcomes, urinary incontinence severity, and pelvic floor muscle strength in individuals with overactive bladder after incomplete spinal cord injury.MethodsA single-blind randomized controlled trial was conducted on 74 male participants diagnosed with overactive bladder and incomplete spinal cord injury. Participants were randomly assigned to an experimental group or a control group. Treatment was conducted for 8 weeks, three sessions per week. Outcomes were assessed at baseline, post-intervention, and the 8-week follow-up.ResultsThe experimental group showed significantly greater improvements in the measured outcomes compared with the control group (P < 0.001).ConclusionIncorporating a multimodal regimen demonstrated significant efficacy in enhancing bladder capacity, continence, and muscle function in individuals with overactive bladder due to incomplete spinal cord injury.Clinical trial registry (ID: NCT07008157).https://clinicaltrials.gov/study/NCT07008157?cond=Spinal%20Cord%20Injuries&intr=Multimodal%20Rehabilitation%20Program&viewType=Card&rank=1.

Humans↗

PubMed1977

Evaluation of brain function in severe human head trauma with multimodality evoked potentials. Part 1: Evoked brain-injury potentials, methods, and analysis.

Methods for obtaining multimodality evoked potentials, somatosensory, visual, auditory, and auditory brain-stem potentials in patients with severe head trauma are described. A method of analyzing abnormal multimodality evoked potentials (graded evoked brain-injury potentials) is proposed that defines the degree of abnormality of the electrophysiological data and expresses it simply in four grades per modality. Data from 20 normal subjects are given for comparison with the abnormal data obtained from 51 patients with head trauma.

Acoustic Stimulation↗

PubMed2026

Integrating metagenomic next-generation sequencing into a multimodal diagnostic framework for spinal infection: enhancing etiological identification and clinical prediction.

BACKGROUND: Spinal infection (SI) remains diagnostically challenging because of heterogeneous etiologies, nonspecific clinical manifestations, and the limited sensitivity of conventional microbiological approaches, particularly following empirical antimicrobial exposure. Although metagenomic next-generation sequencing (mNGS) enables unbiased pathogen detection, its incremental clinical value beyond pathogen identification and its role within integrated diagnostic strategies remain incompletely established. METHODS: We retrospectively analyzed 208 consecutive patients with suspected SI between August 2022 and August 2025. Final diagnoses were established using a multidisciplinary-adjudicated composite reference standard incorporating clinical, radiological, microbiological, and histopathological evidence. The diagnostic performance of mNGS was compared with conventional culture and histopathology. Furthermore, multimodal predictive models integrating clinical variables and microbiological information were developed using L1-regularized logistic regression. RESULTS: In the comparative cohort, mNGS achieved a significantly higher diagnostic yield than culture (66.5% vs. 27.41%, P < 0.001). Among confirmed SI cases, mNGS demonstrated higher sensitivity than conventional culture (91.67% vs. 40.15%, P < 0.001). mNGS identified a substantially broader pathogen spectrum, ranging from fastidious organisms such as Mycobacterium tuberculosis and Brucella to rare pathogens including Talaromyces marneffei and Coxiella burnetii, and maintained robust sensitivity (98.2%) despite prior antibiotic exposure. While an integrated clinical model achieved an AUC of 0.916, mNGS as a standalone modality provided superior discriminative power (AUC = 0.889) compared to histopathology (AUC = 0.836), the Conventional Biomarker Model (AUC = 0.742), and culture (AUC = 0.693). CONCLUSIONS: mNGS is a high-yield diagnostic tool for spinal infection, particularly in culture-negative and antibiotic-pretreated scenarios. Integrating mNGS into a multimodal clinical framework facilitates etiological clarity and precision antimicrobial therapy.

Humans↗

PubMed2026

Multimodal atlas of human atherosclerosis links granular vascular cell states to coronary artery disease risk.

Advances in single-cell and spatial assays have revolutionized the scale and resolution of molecular tissue profiling. Here we present MetaPlaq, a multimodal atlas of human atherosclerotic arterial beds comprising over a million cells across single-cell transcriptomics, epigenomics and high-resolution spatial expression assays. We map granular cell states and disease-relevant transcriptional programs within the native tissue context of coronary arteries. Furthermore, we map cardiovascular GWAS signals to smooth muscle cells (SMCs) and endothelial cells (ECs) and uncover the cis-regulatory architecture governing their phenotypic transitions. Our comprehensive epigenomic reference allowed us to build cell-specific enhancer-gene link maps and multimodal gene regulatory networks (GRNs) underlying disease-relevant states such as osteogenic SMCs and ECs undergoing mesenchymal transition. We also integrate SMC and EC disease-associated gene sets with GRNs to nominate key transcription factors such as PRRX1, BNC2 and ELK3 regulating atherosclerosis-relevant transcriptional programs. Finally, we layer single-cell and spatial modalities to fine-map GWAS variants with improved cell and anatomical context. We highlight candidate cell-specific regulatory mechanisms at less characterized CAD loci, including FGD5 and MCF2L in ECs. Together, this atlas represents an important step towards fully interpreting genetic risk loci and informing new therapeutic strategies for cardiovascular disease.

Journal Article↗

PubMed1977

Agoraphobia: indications for the application of the multimodal behavioral conceptualization.

Agoraphobia, characterized as fear of going into public places, vehicles, shops, streets, and so forth, is a prevalent syndrome which permeates the patient's life. The efficacy of psychoanalytic, behavioral, and pharmacological treatments has been disappointing. The pervasive nature of agoraphobia dictates that its treatment is most efficacious when a combination of separate although interactive techniques is applied in concert. A multimodal behavioral conceptualization provides the vehicle for the systematic combined application of seemingly diverse individual approaches through monitoring and treating the BASIC ID, an acronym for behavior, affect, sensation, imagery, cognition, interpersonal relationships, and drugs. In an illustrative case, application of the multimodal behavioral conceptualization was instrumental in bringing relief to an agoraphobic patient's distress and disabilities.

Adult↗