PubMed Health⌕ Search

SEARCH · PubMed Health

Results for “multimodal integration”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12Linked to original sources

Biological Foundation Models for Complex Disease Research and Clinical Translation.

Complex diseases, including cancer, rare genetic disorders, neurodevelopmental and psychiatric conditions, and neurodegenerative diseases, arise from interactions among genetic variation, gene regulation, and cellular states that are difficult to capture using a single data type or biological scale. Biological foundation models address this challenge by treating nucleotides and genes as tokens and learning representations that can be transferred to downstream biomedical and clinical tasks. In this review, we examine two major model classes, genomic sequence foundation models and cell foundation models, and compare their tokenization strategies, model architectures, pretraining objectives, and adaptation methods. We summarize their emerging applications in regulatory variant interpretation, disease-associated cell-state analysis, drug-response prediction, and therapeutic target discovery across complex diseases. We distinguish applications supported by experimental or retrospective validation from those that remain primarily computational or conceptual. We further discuss key challenges to clinical translation, including multimodal data integration, model interpretability, benchmarking, patient-specific prediction, and privacy protection. We highlight future opportunities to integrate biological foundation models with emerging frameworks of medical digital twins, agentic AI, and federated learning. By linking model design to translational goals, this review provides a practical framework for evaluating biological foundation models and their readiness for complex disease research and clinical use.

biological foundation model↗

18F-FDG PET mapping of regional brain activity in runners.

BACKGROUND: To examine regional metabolic changes in the human brain induced by free running in upright posture. METHODS. EXPERIMENTAL DESIGN: Regional brain changes in glucose uptake induced by running were examined by comparing brain images obtained by positron emission tomography (PET) and 18F-fluorodeoxyglucose (FDG). SETTING: The study was conducted at a research institute and involved participation of healthy young volunteers. Data sampling and analysis required special imaging device and special computer hardware/software. PARTICIPANTS: Subjects were 17 healthy male volunteers. They were divided at random into two equal groups, those who ran 4-5 km and the others sat in the room. MEASURES: differences in regional cerebral glucose uptake between runners and control groups were assessed statistically. Plasma glucose level was also measured and global cerebral uptake was estimated. RESULTS: Running was associated with a relative increase of glucose uptake in the temporoparietal association cortex, occipital cortex, premotor cortex and the cerebellar vermis. The highest activity was noted in the temporoparietal association cortex. Activity of the primary sensorimotor cortex was higher in the superomedial part (leg motor area) than the lateral part (thorax and arm). CONCLUSIONS: Running augmented energy consumption in the parieto-occipital region relative to the motor area, probably due to the higher energy consumption necessary for integration of multimodal sensory information than for generation of motor output. Our results indicate that FDG PET is a useful tool for brain mapping under various physiological conditions.

Adult↗

[Psychosocial rehabilitation at the dawn of the 21st century: II: Therapeutic or rehabilitative modalities and institutional disposition].

The main goal of psychosocial rehabilitation is to compensate the vulnerability underlying psychiatric disorders through intermediate institutions when the persistence and recurrence of these disorders have led to social and professional exclusion. Intermediate institutions refer to services which allow transition between the state of dependence on the hospital to the state of relative autonomy in social community. Psychosocial rehabilitation is a comprehensive approach which link the type of interventions: treatment, rehabilitation and support integrated in multimodal and individualized programs. A study of the out-patients followed by the rehabilitation unit of the psychiatric department in Lausanne has shown that provision of services is divided into 60% for rehabilitation, 20% for treatment and 20% for support independently of the psychiatric disorders. The implementation of these programs necessitates institutional support from psychiatric hospital to outpatient clinics through different types of facilities in order to offer a medical and psychosocial device of rehabilitation into the community.

Combined Modality Therapy↗

Integrating metagenomic next-generation sequencing into a multimodal diagnostic framework for spinal infection: enhancing etiological identification and clinical prediction.

BACKGROUND: Spinal infection (SI) remains diagnostically challenging because of heterogeneous etiologies, nonspecific clinical manifestations, and the limited sensitivity of conventional microbiological approaches, particularly following empirical antimicrobial exposure. Although metagenomic next-generation sequencing (mNGS) enables unbiased pathogen detection, its incremental clinical value beyond pathogen identification and its role within integrated diagnostic strategies remain incompletely established. METHODS: We retrospectively analyzed 208 consecutive patients with suspected SI between August 2022 and August 2025. Final diagnoses were established using a multidisciplinary-adjudicated composite reference standard incorporating clinical, radiological, microbiological, and histopathological evidence. The diagnostic performance of mNGS was compared with conventional culture and histopathology. Furthermore, multimodal predictive models integrating clinical variables and microbiological information were developed using L1-regularized logistic regression. RESULTS: In the comparative cohort, mNGS achieved a significantly higher diagnostic yield than culture (66.5% vs. 27.41%, P < 0.001). Among confirmed SI cases, mNGS demonstrated higher sensitivity than conventional culture (91.67% vs. 40.15%, P < 0.001). mNGS identified a substantially broader pathogen spectrum, ranging from fastidious organisms such as Mycobacterium tuberculosis and Brucella to rare pathogens including Talaromyces marneffei and Coxiella burnetii, and maintained robust sensitivity (98.2%) despite prior antibiotic exposure. While an integrated clinical model achieved an AUC of 0.916, mNGS as a standalone modality provided superior discriminative power (AUC = 0.889) compared to histopathology (AUC = 0.836), the Conventional Biomarker Model (AUC = 0.742), and culture (AUC = 0.693). CONCLUSIONS: mNGS is a high-yield diagnostic tool for spinal infection, particularly in culture-negative and antibiotic-pretreated scenarios. Integrating mNGS into a multimodal clinical framework facilitates etiological clarity and precision antimicrobial therapy.

Humans↗

Scalable, generalizable and uncertainty-aware integration of spatial multiomics across diverse modalities and platforms with SCIGMA.

Recent advances in spatial omics technologies have enabled simultaneous profiling of transcriptomic, proteomic, epigenomic, metabolomic and imaging data at high spatial resolution, offering unprecedented opportunities to dissect tissue complexity. However, integrating these diverse and large-scale spatial multimodal datasets remains a major computational challenge. We present SCIGMA, a scalable and generalizable deep learning framework for spatial multiomics integration. SCIGMA introduces an uncertainty-aware contrastive learning objective and multiview graph neural networks to preserve modality-specific signals while learning biologically meaningful joint representations. Unlike previous methods, SCIGMA provides spatially resolved uncertainty estimates, interpretably identifying regions of biological or technical heterogeneity. SCIGMA supports integration of up to five modalities, and its modular framework is extensible to future technologies with even more modalities. It also scales to more than 1 million spatial locations, enabling analysis of high-resolution datasets such as Visium HD and Xenium Prime. We evaluated SCIGMA across 19 datasets spanning 8 modalities, 10 tissues and 9 platforms. On benchmarkable datasets, SCIGMA outperformed other methods in spatial domain detection, modality preservation, feature reconstruction and reproducibility. SCIGMA identifies biologically meaningful structures, refined spatial domains and modality-specific regulatory programs, providing a robust, flexible and future-ready solution for scalable spatial multimodal integration.

Multiomics↗

Multidisciplinary research of multimodal stimulation of premature infants: an integrated review of the literature.

PURPOSE: To review and integrate current multidisciplinary research literature about multiple sensory stimulation of premature infants. SOURCES: Computer search using CINAHL, MEDLINE, PSYLIT, SOCIAL SCIENCE INDEX and the topics "premature," "infant," "neonate," and "stimulation." Review of the results of the research and integration of findings for nursing. RESULTS: Overall, infants responded positively to the interventions reviewed, including immediate and long-term effects, some up to 2 years. IMPLICATIONS FOR NURSING PRACTICE: Nursing research of multiple sensory stimulation of premature infants can be improved by the use of (a) a theoretical base for research, (b) better descriptions of the infant samples and the environmental context of the research, and (c) individualization of infant stimulation to permit conservation of epigenetic integrity of the infant.

Female↗

Laparoscopic colonic surgery--mission accomplished or work in progress?

Laparoscopic colonic resection may facilitate early postoperative recovery due to reduced surgical stress, pain and ileus. However, large randomised studies have only shown marginal improvements in outcome compared with open surgery, reporting a median hospital stay of about 5-7 days. Concomitant with these developments multimodal rehabilitation, which involves a revision of general postoperative care principles, improved pain relief with epidural analgesia and early oral nutrition and mobilization, has demonstrated greater improvements in recovery after open surgery, resulting in a median hospital stay of about 2-4 days. Recent single centre, randomised studies where laparoscopic and open colonic resection are combined with multimodal rehabilitation have not resolved the debate regarding which is the optimal operative technique. Therefore, new strategies are required to integrate laparoscopy with multimodal rehabilitation in order to establish its advantages, cost effectiveness and indications in specific groups of patients or colorectal procedures, thus justifying widespread application of the laparoscopic technique.

Colectomy↗

Multimodal Deep Learning and Foundation Models for Early Detection and Forecasting of Plant Diseases.

Plant diseases destroy 20-40% of global food production annually, posing a critical threat to food security for a projected population of 9.7 billion by 2050. Conventional diagnostic approaches relying on expert visual assessment are slow, costly, and unsuitable for modern agricultural scales. While deep convolutional neural networks demonstrated early promise, single-modality, image-centric systems consistently fail under real-world field conditions characterized by variable lighting, co-occurring infections, and cultivar diversity. This review synthesizes a decade of progress across four interconnected frontiers: the evolution of deep learning architectures for plant disease detection; the adaptation of foundation models including CLIP, SAM, and DINOv2 to agricultural contexts; the development of multimodal fusion frameworks integrating imagery, environmental, genomic, and hyperspectral data; and the transition from static disease diagnosis to descriptive comparison of reported metrics, which suggested that multimodal approaches frequently reported improved diagnostic performance relative to corresponding single-modality baselines, although direct cross-study comparison was limited by methodological heterogeneity. A systematic review following PRISMA guidelines identifies eligible comparative studies. Descriptive comparison of reported performance metrics across these studies indicated that multimodal approaches generally achieved higher accuracy and sensitivity than single-modality models, particularly for pre-symptomatic disease detection. Eight critical research gaps are identified, including the absence of a unified agricultural foundation model and limited climate-aware forecasting under non-stationary climate projections. A structured research agenda is proposed to accelerate translation from laboratory performance to globally equitable, field-deployable crop protection systems.

convolutional neural networks↗

Enterocutaneous Fistula-Associated Sepsis and Mortality: Development and Validation of a Multimodal Artificial Intelligence Prediction Model.

BACKGROUND: Predicting enterocutaneous fistula (ECF)-associated sepsis and mortality poses significant challenges in digital health care due to the disease's complexity and heterogeneous clinical manifestations. Current approaches that rely on single-modal data or traditional scoring systems often fail to capture the intricate immune-inflammatory dynamics and multisystem involvement in patients with ECF. OBJECTIVE: This study aims to develop an artificial intelligence (AI)-driven multimodal fusion model integrating clinical, imaging, and transcriptomic data for early prediction of ECF-associated sepsis and 28-day mortality, addressing the limitations of conventional single-dimensional models. METHODS: This study leveraged publicly available datasets (Medical Information Mart for Intensive Care III [MIMIC-III], electronic Intensive Care Unit [eICU], and The Cancer Genome Atlas) to construct a multimodal framework. Clinical parameters were processed using Extreme Gradient Boosting, abdominal imaging features were extracted via convolutional neural networks, and transcriptomic profiles were analyzed with variational autoencoders. A Transformer-based fusion network was employed for joint prediction and validated through cross-validation and external testing. Key features were identified using Shapley Additive Explanations and Local Interpretable Model-Agnostic Explanations interpretability algorithms, while immune regulatory mechanisms were explored via weighted gene co-expression network analysis. RESULTS: The multimodal model achieved an area under the curve (AUC) of 0.89 for predicting sepsis and 28-day mortality, outperforming unimodal models (clinical-only model, AUC 0.72, and imaging-only model, AUC 0.78). Critical predictors included Sequential Organ Failure Assessment score, lactate levels, intra-abdominal free fluid on imaging, and immunoregulatory genes (programmed death-ligand 1 [PD-L1] and indoleamine 2,3-dioxygenase 1 [IDO1]). Mechanistic analysis revealed distinct immune reprogramming in patients with sepsis, characterized by increased regulatory T cells and M2 macrophages, along with downregulated cluster of differentiation 8+ (CD8+) T cells. CONCLUSIONS: This multimodal AI model offers an innovative digital solution in medical informatics, enabling precise early risk stratification for ECF-associated sepsis. By integrating multisource data and providing interpretable insights into immune-inflammatory pathways, the model enhances health care quality for patients with ECF and paves the way for personalized intervention strategies.

Humans↗

Integration of sulcal and functional information for multimodal neuronavigation.

OBJECT: The authors present the use of cortical sulci, segmented from magnetic resonance (MR) imaging, and functional data from functional (f)MR imaging and magnetoencephalography (MEG) in the image-guided surgical management of lesions adjacent to the sensorimotor cortex. METHODS: In an initial set of 11 patients, sulci near lesions were automatically segmented from MR imaging data sets, then MEG and fMR imaging examinations were performed. Relevant functional information was preoperatively interpreted and selected from MEG and fMR imaging and subsequently transferred to the navigation system for selected sulci. A neuronavigation system consisting of a surgical microscope with enhanced reality overlay display was used. Data were displayed as contours on the cut-plane images of a stereotactic workstation and as contours on the overlay screen of the head-up display within the optical path of the right eyepiece of the surgical microscope. CONCLUSIONS: This method, in which both sulcal and functional mapping are used for surgery planning and neuronavigation, provides helpful information. It is a promising procedure for the treatment of patients who harbor lesions in areas around the eloquent cortex.

Adult↗

Protocol to perform integrative analysis of high-dimensional single-cell multimodal data using an interpretable deep learning technique.

The advent of single-cell multi-omics sequencing technology makes it possible for researchers to leverage multiple modalities for individual cells. Here, we present a protocol to perform integrative analysis of high-dimensional single-cell multimodal data using an interpretable deep learning technique called moETM. We describe steps for data preprocessing, multi-omics integration, inclusion of prior pathway knowledge, and cross-omics imputation. As a demonstration, we used the single-cell multi-omics data collected from bone marrow mononuclear cells (GSE194122) as in our original study. For complete details on the use and execution of this protocol, please refer to Zhou et&#xa0;al.1.

Deep Learning↗

An Integrative Morphological and Genomic Analysis With a Refined Fluorescence In Situ Hybridization (FISH) Threshold and Novel Kinase Fusions in a Large Asian Cohort of Spitzoid Neoplasms.

Differentiating atypical Spitz tumors (ASTs) from true Spitz melanomas (SMs) and conventional melanomas with spitzoid features (MSFs) remains a formidable diagnostic challenge. Because current molecular epidemiological data are overwhelmingly derived from Caucasian cohorts, the genomic landscape of Asian populations remains largely unexplored. To elucidate the molecular progression landscape and refine the diagnostic criteria, we performed a comprehensive multimodal analysis-integrating histomorphology, immunohistochemistry, multiprobe fluorescence in situ hybridization (FISH), and targeted RNA/DNA-based next-generation sequencing (NGS)-on a cohort of 140 spitzoid neoplasms. This cohort, comprising 126 ASTs, 8 SMs, and 6 MSFs, represents the largest Asian cohort to date. Malignant phenotype strongly correlated with lesional asymmetry, deep atypical mitoses, a sheet-like growth pattern, diffuse preferentially expressed antigen of melanoma positivity, and significant loss of p16 expression (64.3% in SM/MSF vs 9.5% in ASTs; P < .0001). Building upon the established melanoma FISH criteria, we optimized a prognostic threshold of &#x2265;2 FISH abnormalities specifically tailored for spitzoid neoplasms. We demonstrated that isolated single chromosomal aberrations (particularly MYB loss) are relatively stable events that are frequent in indolent ASTs, whereas our refined &#x2265;2 threshold yielded 100% sensitivity and 92.5% specificity for predicting regional lymph node metastasis/local recurrence. Molecularly, NGS identified mutually exclusive initiating driver alterations (comprising kinase fusions and HRAS mutations) in 89.9% of true Spitz neoplasms, a remarkably high prevalence suggesting a distinct genetic background in Asian populations. We also characterized 5 entirely novel kinase fusions (ZNF24::ROS1, PCBP1::ROS1, NUMA1::RET, CBWD1::ALK, and TPR::NTRK1). Furthermore, NGS definitively segregated true Spitz neoplasms from morphological mimics (MSF), which lacked fusions and were driven by canonical genomic alterations of the conventional melanoma pathway. Integrating these genomic landscapes validated a stepwise progression model. Although isolated kinase fusions drove indolent ASTs, malignant SM invariably harbored concurrent pathogenic secondary alterations, demonstrating a profound reliance on CDKN2A/B, TP53, and CDK4 aberrations. Ultimately, we propose an integrated diagnostic algorithm combining morphological evaluation, the refined FISH threshold, and comprehensive NGS profiling, providing a precise, evidence-based framework for pathway classification and clinical management of spitzoid neoplasms.

fluorescence in situ hybridization↗

Modern multimodal neuroimaging for radiosurgery: the example of PET scan integration.

Radiosurgery relies critically on medical imaging modalities. Leksell Gamma Knife (LGK) radiosurgery presents the highest requirements in terms of imaging accuracy as the treatment is applied in a single high-dose session with no other spatial control than medical imaging. The advent of new imaging modalities opens challenges for LGK planning strategies. The integration of stereotactic PET in LGK represents an example of such application of modern multimodality imaging in radiosurgery. Our experience consists of 80 patients treated with the combination of MR/CT and PET guidance. In order to analyze the specific contribution of PET findings, we developed a classification reflecting the strategy used to define the target volume. When combining PET and MR information, 102 target volumes were defined, because some patients presented with multiple lesions or multifocal tumor areas. Abnormal PET uptake was found in 86% of the lesions, and this information altered significantly the MR-defined tumor in 73%. In conclusion, integration of PET in radiosurgery provides additional information opening new perspectives for the treatment of brain tumors. The use of a standardized classification allows to assess the relative role of PET. A similar approach could be useful and may serve as a template for the evaluation of the integration of other new imaging modalities in radiosurgery.

Adenoma↗

The role of multimodality therapy in locoregional non-small cell lung cancer.

Surgical treatment is offered to all patients with stage I or II disease and to specific groups of patients with stage III or IV disease. Cisplatin based on regimens of induction chemotherapy or chemoradiotherapy have proven to be valuable in stage IIIA (N2) disease. We now recommend induction therapy on an investigational basis to most patients with stage I or II tumors and to all those with stage IIIA tumors, when the 5 year survival is anticipated to be less than 50% with conventional therapy. Many new chemotherapy agents effective in advanced stage lung cancer are currently integrated into this multimodality approach in hopes of further improvement in tumor control and survival.

Antineoplastic Agents↗

Unifying multimodal single-cell data with a mixture-of-experts &#x3b2;-variational autoencoder framework.

Multimodal single-cell assays profile complementary layers of cell state, but integration is complicated by modality mismatch, sparsity, and uneven cohort coverage. Here, we present Unified Variational Inference (UniVI), a scalable mixture-of-experts &#x3b2;-variational autoencoder that learns a shared latent space while preserving modality-specific structure. UniVI couples modality-specific encoders/decoders with a shared latent prior and a symmetric cross-modal alignment objective, enabling consistent integration of paired measurements without curated feature-link graphs or preannotated reference atlases; optional supervised heads can be added when labels are available. Across paired RNA-protein (CITE-seq) and RNA-chromatin (10x Genomics Multiome, SHARE-seq) data spanning human PBMCs and mouse back skin-a nonhematopoietic tissue with continuous differentiation hierarchies-UniVI produces coherent embeddings, improves label transfer, and enables cross-modal reconstruction and denoising. Extending to trimodal measurements, UniVI maintains robust three-way alignment among RNA, chromatin accessibility, and surface proteins (TEA-seq), and accommodates DNA methylation in a paired scNMT-seq mouse gastrulation proof-of-concept under beta-binomial likelihoods. Performance degrades gracefully under severe cell type imbalance and in the presence of modality-exclusive populations. In an acute myeloid leukemia mosaic design, a paired RNA-protein bridge anchors independent RNA-only and protein+genotype cohorts, revealing genotype-associated neighborhoods that sharpen with mutation-aware fine-tuning. UniVI thus provides a flexible, interpretable framework for multimodal integration across paired, trimodal, and mosaic study designs and supports practical reference-to-query projection in partially observed studies.

Journal Article↗

The importance of postoperative radiation therapy in multimodality management of locally advanced breast cancer: a phase II trial of neoadjuvant MVAC, surgery, and radiation.

PURPOSE: To determine the impact of postoperative radiation on locoregional relapse and overall survival rate in a multimodality protocol for locally advanced breast cancer (LABC). MATERIAL AND METHODS: Of the patients entered in the protocol, 55 were evaluable. Treatment consisted of: neoadjuvant MVAC (methotrexate, vinblastine, adriamycin, and cisplatin) until a maximum response had been achieved; modified radical mastectomy; 6 courses of postoperative adjuvant MVAC chemotherapy, and chest wall irradiation (CWXRT). Multivariate analysis of locoregional response and overall survival was done. RESULTS: Of the total, 42 patients received chest wall radiation; 28 of these also received radiation to regional lymph nodes. Chest wall doses ranged from 45 Gy to 50.4 Gy to the whole chest wall, with 31 patients receiving an additional chest-wall boost. The incidence of locoregional relapse with and without radiation was 7% vs. 31%, respectively (p = 0.026). An overall survival benefit was seen in those receiving radiation, with a mean overall survival of 50 months vs. 20 months, and a 3-year overall survival of 88% vs. 46% with and without radiation, respectively (p = 0.003). Multivariate analysis showed that overall survival was affected by the presence of pathological CR (p = .047), the number of pre-operative chemotherapy cycles (p = .036) and whether or not they received radiation (p = 0.003). Neither the interval between surgery and radiation, technique of radiation, nor radiation modality significantly affected local control. CONCLUSION: The significant improvement in local regional control and overall survival with the addition of radiation suggests that radiation should be an integral part of multimodality management of locally advanced breast cancer.

Adult↗