SEARCH · PubMed Health

Results for “Multimodal learning”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2Linked to original sources

PubMed2026

Multimodal alignment improves generalizability of genomic biomarker prediction in computational pathology.

Computational pathology models that use digitized histopathology whole-slide images have the potential to become a cost-effective and scalable alternative to molecular assays for the prediction of genomic biomarkers, a key task in precision oncology. However, as new genomic biomarkers are discovered or quantified, large, labeled datasets must be prospectively collected to train new models. To address this challenge, we developed multimodal alignment for biomarker learning and generalization (MARBLE), a multimodal contrastive pretraining strategy that integrates structured biomarker knowledge into representation learning of histopathology images. MARBLE aligns histopathology-derived representations with representations of genomic biomarkers generated by a large language model (LLM) and a protein language model (PLM). This biologically informed alignment enables data-efficient generalization to novel, out-of-distribution biomarkers. Using the MSK-IMPACT cohort of over 40,000 patients across multiple biomarker panel versions, we design experiments grounded in real-world data to demonstrate the value of our proposed approach.

CP: computational biology↗

PubMed1986

Distributional expectations and the induction of category structure.

Previous research on how categories are learned from observation of exemplars has largely ignored the possible role of prior expectations concerning how exemplars will be distributed. The experiments reported here explored this issue by presenting subjects with category-learning tasks in which the distributions of exemplars defining the categories were varied. In Experiments 1 and 2 the distributional form of a category was found to affect speed of learning. Learning was faster when a category's distribution was normal than when it was multimodal. Also, subjects in the early stages of learning a multimodal category responded as if it were unimodal. These results suggested that subjects enter category-learning tasks with expectations of unimodal, possibly normal, distributions of exemplars. Experiments 3 and 4 attempted to manipulate subjects' prior expectations by varying the distribution of exemplars in the first of two consecutive category-learning tasks. Learning a multimodal category was influenced by the shape of a previously learned distribution and was facilitated when the earlier distribution was either multimodal or skewed, rather than normal. These results are interpreted as support for a dual-process model of category learning that incorporates the effects of prior expectations concerning exemplar distributions.

Adolescent↗

PubMed2026

Deep learning-based cross-attention fusion of multimodal MRI for survival prediction and risk stratification in IDH-wildtype glioblastoma: a multicenter study.

BACKGROUND: Glioblastoma (GBM) exhibits profound molecular and spatial heterogeneity, complicating prognostic evaluations. While multiparametric MRI provides crucial multidimensional biological information, conventional end-to-end deep learning integration strategies, such as early or late fusion, often fail to capture complex nonlinear cross-modal interactions. We aimed to systematically evaluate a cross-attention fusion (CAF) architecture for GBM survival prediction and quantify its incremental prognostic value relative to existing clinical tools. METHODS: In this multicenter retrospective study, 386 adults with IDH-wildtype, WHO grade 4 GBM were assembled from an institutional cohort (n = 226), the Chinese Glioma Genome Atlas (CGGA, n = 62), and The Cancer Genome Atlas (TCGA, n = 98). Using a unified 3D ResNet-18 backbone, we compared single-modality models, early fusion, late fusion, and CAF on preoperative T1-weighted, contrast-enhanced T1-weighted (T1CE), and T2-weighted MRI, and integrated the resulting deep learning risk score with routine clinical variables through multivariable Cox regression. Performance was assessed using Harrell's C-index, time-dependent AUC, and decision curve analysis. RESULTS: CAF showed numerically higher, more consistent C-index trends than early fusion, late fusion, and single-modality models (pooled C-index 0.629, 95% CI 0.594-0.664), although pairwise differences in time-dependent AUC were not statistically significant. Integrating clinical variables raised the pooled C-index to 0.691 (95% CI 0.660-0.721) in the treatment-era model, with comparable performance across the three cohorts (Local 0.688; CGGA 0.716; TCGA 0.689); a pre-treatment configuration excluding adjuvant therapy yielded a pooled C-index of 0.642. Under leave-one-cohort-out external validation, the combined model retained significant risk stratification in all held-out cohorts (C-index 0.63-0.71; all log-rank P < 0.01), albeit with attenuated discrimination. The deep learning risk score remained independent after multivariable adjustment (HR 1.41 per SD, 95% CI 1.26-1.57; P < 0.001). Kaplan-Meier analysis confirmed significant high- versus low-risk separation in all cohorts, and decision curve analysis showed greater net benefit than clinical-only and deep-learning-only models. CONCLUSION: The CAF-derived risk score offers prognostic information complementary to routine clinical variables, representing a promising noninvasive tool for individualized risk stratification when molecular profiling is incomplete or unavailable; these findings warrant prospective external validation before clinical use.

cross-attention fusion↗

PubMed2026

H&E to recurrence score: A step forward, but not yet a substitute for genomic testing.

Shamai and colleagues developed a multimodal deep-learning model that predicts Oncotype DX recurrence scores from routine H&E slides and clinicopathological variables in hormone receptor‑positive, HER2‑negative early breast cancer. Validated across the TAILORx trial and six external cohorts (over 5000 patients), the model achieved an AUC of 0.898 for identifying recurrence score ≥26 and recapitulated genomic assay patterns of chemotherapy benefit. Notably, 31% of clinically high-risk postmenopausal women were downgraded to low risk by AI, suggesting potential to reduce overtreatment. However, several limitations preclude immediate clinical substitution for genomic testing. First, intratumoural heterogeneity leads to discordant predictions with unclear management guidance. Second, the model's chemotherapy benefit estimates rely on TAILORx's age-based menopausal surrogates, which may not reflect real-world hormonal status or LHRH agonist use. Third, predictive value in node-positive disease remains untested in randomised datasets such as RxPONDER. Additionally, calibration uncertainty near risk thresholds and global scalability issues (including IHC requirements and digital pathology infrastructure) persist. While this represents a landmark step toward democratising precision oncology, the AI tool should currently serve as a complementary decision aid, with genomic testing remaining the gold standard for intermediate, borderline, or discordant cases.

Breast cancer↗

PubMed2026

AI-driven diagnostic and prognostic models for metabolic dysfunction-associated steatotic liver disease: insights from clinical, imaging, and multi-omics studies-a scoping review.

Metabolic dysfunction-associated steatotic liver disease (MASLD), formerly known as non-alcoholic fatty liver disease (NAFLD), is the most common chronic liver disease around the world, affecting 33.6% of the adult population (95% CI: 28.1%-39.5%; I 2 = 99.9%), or roughly one in three. The extent of the liver damage is variable, from simple steatosis to metabolic dysfunction-associated steatohepatitis (MASH, formerly NASH), cirrhosis and hepatocellular carcinoma (HCC). Early diagnosis is essential to prevent serious liver damage. Traditional diagnostic techniques such as liver biopsy, imaging, and biomarker testing are all invasive, costly, reduced sensitive to early-stage disease, and they also have variability among observers. Modern diagnostic and prognostic approaches based on the principles of Artificial Intelligence (AI) and specifically on machine learning (ML) and deep learning (DL) have enabled multimodal approaches integrating clinical, imaging and molecular data. This scoping review conducted per PRISMA-ScR guidelines, synthesizes findings from 73 studies (search window 2020-2026) across three dimensions: clinical data driven models, imaging-based classifiers (ultrasound, CT and MRI), and multi-omics (genomics, transcriptomics and proteomics) techniques. Moreover, emergence of models such as U-Net and LiverNet 2.x, classification models like DeepLiverNet and BiLSTM models, as well as transformer frameworks and the identification of biomarkers models are also described. This study also investigates challenges such as data heterogeneity, data interpretability, fairness and real-world clinical application. Finally, important areas of research opportunities and future directions are highlighted to present a developing clinically applicable, explainable and ethical AI solutions to manage MASLD.

MASLD↗

PubMed2026

GBFN: A gated bimodal fusion network leveraging foundation model embeddings for cancer drug sensitivity prediction.

Despite recent progress in deep learning for cancer drug sensitivity prediction, many existing models still rely on task-specific representation learning or relatively simple multimodal fusion, which may limit their ability to capture complex drug-cell interactions. To address this issue, we developed GBFN, a gated bimodal fusion network for continuous IC50 prediction that integrates pretrained drug and cell-line representations. Specifically, drug embeddings were obtained from SMI-TED, whereas cell-line embeddings were derived from transcriptomic profiles using BulkFormer. These two modalities were then combined through a dimension-wise gated fusion module and used to predict IC50 values in matched drug-cell line pairs. On the CCLE-based benchmark, GBFN outperformed representative neural baselines, including GraphDRP, TGSA, and TransEDRP, and achieved the best overall performance, with an R² of 0.8714 and an RMSE of 0.8938. Moreover, ablation analysis showed that the model using drug features and cell-line expression data with gated fusion performed better than the corresponding model using direct concatenation, indicating that the improvement was associated with the fusion strategy rather than with the input modalities alone. In addition, cell-line expression data were more informative than mutation data in the present setting, and adding mutation data to the model using drug features and expression data did not further improve performance. Across major cancer types, GBFN maintained generally high cell-line-level predictive performance, and perturbation-based attribution identified biologically relevant transcriptomic programs in selected drug-cell line settings. Together, these findings support GBFN as a compact and effective framework for continuous drug response prediction.

Humans↗

PubMed2026

GICPIdb: an archival repository of multimodal data focusing on pathological images for gastrointestinal cancers.

INTRODUCTION: Deep learning (DL) shows great potential for predicting biomarkers from routine histopathological slides of gastrointestinal (GI) cancers. Yet most existing models are validated on limited patient cohorts, while pathological image annotation and molecular marker standardization demand substantial professional expertise. To address these gaps, we constructed the Gastrointestinal Cancer Pathological Image Archive (GICPIdb, gicpidb.shubuzuo.top), a dedicated database and web platform covering seven major GI cancer types. METHODS: High-quality hematoxylin and eosin (H&E)-stained whole-slide images were collected from multiple sources and uniformly processed. Image annotations were performed by board-certified pathologists following standardized protocols. GICPIdb offers five interactive web modules for data uploading, quality control, feature extraction, online annotation and AI-based prediction. Its intuitive interface supports data browsing, retrieval, visualization and downloading. RESULTS: The database houses 2,863 pathologist-annotated, uniformly processed, high-quality H&E stained images collected from 2,655 patients. Of these, 1,699 patients were sourced from The Cancer Genome Atlas (TCGA), 182 from the Clinical Proteomic Tumor Analysis Consortium (CPTAC), and 424 from China-Japan Friendship Hospital and 350 from Chifeng Municipal Hospital in Inner Mongolia, China. It also integrates data on over 50 key molecular markers (e.g., MSI, TMB) and prognostic labels related to survival, recurrence and metastasis. DISCUSSION: GICPIdb aims to promote the development of DL-driven AI tools for cancer research and clinical translation. The multi-institutional data collection and standardized annotation pipeline are expected to enhance the generalizability and reproducibility of AI-based prediction models across diverse patient populations.

deep learning↗

PubMed2026

Multimodal features and prognostic risk assessment in locally advanced gastric cancer patients following neoadjuvant therapy based on machine learning algorithms: a multicenter study.

BACKGROUND: Neoadjuvant therapy (NAT) is recommended for locally advanced gastric cancer (LAGC), but some patients respond poorly. We aimed to construct a multimodal model integrating CT images, transcriptomic sequencing, and clinicopathological data to assess prognosis in LAGC patients receiving NAT. MATERIALS AND METHODS: This multicenter study included 505 LAGC patients who underwent NAT. Radiomic features were extracted from preoperative CT images of 505 patients. RNA-seq was performed on 277 post-NAT specimens, with additional data from The Cancer Genome Atlas (TCGA) and Gene Expression Omnibus (GEO) databases (n = 804). Patients were divided into training (168 cases), internal validation (72 cases), and external validation cohorts. Machine learning algorithms identified key radiomic, molecular, and clinical features associated with NAT response, which were then integrated into a multimodal model to predict overall survival (OS) and disease-free survival (DFS). RESULTS: Six radiomic and three molecular features significantly associated with NAT response were selected. Radiomic risk (hazard ratio [HR]: 4.0, P < 0.001) and molecular risk (HR: 7.1, P < 0.001) were independent prognostic factors. By integrating radiomic risk, molecular risk, and clinical characteristics, a multimodal model (MuMo) was constructed.The C-index results (OS, C-index = 0.855; DFS, C-index = 0.786) demonstrated that MuMo outperformed the single-modality models and ypTNM staging.Mechanistic analysis suggested that the efficacy of neoadjuvant therapy was significantly enriched in immune-inflammatory pathways. CONCLUSIONS: MuMo can effectively predict postoperative survival risk in LAGC patients receiving NAT, serving as a powerful tool for optimizing prognostic assessment.

Humans↗

PubMed2026

Machine learning for population-level risk prediction of future cholangiocarcinoma.

BACKGROUND: The poor prognosis of cholangiocarcinoma (CCA) is largely driven by rapid, asymptomatic disease progression, which usually results in a late diagnosis in the absence of established screening strategies. An early, cost-effective, and universally applicable risk assessment strategy would therefore be valuable. METHODS: We developed machine learning (ML) models on prospective, multimodal data from 487,495 UK Biobank (UKB) participants, of whom 649 developed CCA during follow-up. Data from England (80%) were utilised for ML development via five-fold cross-validation, and then all models were tested on withheld data from Scotland, Wales, and Newcastle (20%). Iterative ablation studies reduced inputs from >150 features across demographic data, lifestyle, health records, blood parameters, genomics, and metabolomics to models built on five and ten routinely available clinical parameters. These were externally validated in the Penn Medicine Biobank (PMBB; n = 2638; 28 CCA), All of Us Research Program (AOU; n = 330,433; 362 CCA), Japan Medical Data Centre Claims Database (JMDC; n = 8,425,522; 723 CCA) and TriNetX (n = 728,886; 1592 CCA). FINDINGS: We show that ML models integrating biliary-disease associated health records and Gamma glutamyltransferase can stratify risk of future CCA. Evaluation on the UKB test set as well as three independent cohorts revealed robust performance and generalisability across ethnicities. We achieved AUROCs of 0.71 [95% CI: 0.703-0.711], 0.77 [95% CI: 0.764-0.778 ], 0.796 [95% CI: 0.795-0.798] and 0.8 [95% CI: 0.794-0.805] for UKB, PMBB, AOU, and JMDC respectively, with respective AUPRCs of 0.014 [95% CI: 0.009-0.018], 0.042 [95% CI: 0.037-0.048], 0.038 [95% CI: 0.033-0.042] and 0.001 [95% CI: 0.001-0.001]. In AOU, application of the Youden J-optimised threshold yielded a number needed to screen of 79. Separate models for intra- and extrahepatic CCA did not improve performance. In line with the pathophysiology, performance declined for longer intervals between assessment and event. A group-level analysis in the TriNetX cohort revealed hazard ratios of up to 82.5 [95% CI: 26.4-257.96]. We provide extensive interpretability results and release all source codes used to develop the presented models. INTERPRETATION: We provide a comprehensive framework for early CCA risk stratification in the general population, identifying key predictors, and demonstrating the potential of data-driven models in personalised screening for hepatobiliary cancer. FUNDING: German Cancer Aid (grant #70115730), Junior Principal Investigator Fellowship programme of RWTH Aachen Excellence strategy.

Humans↗

PubMed2025

JASMINE: A powerful representation learning method for enhanced analysis of incomplete multi-omics data.

Integrative analysis of multi-omics data provides a more comprehensive and nuanced view of a subject's biological state. However, high-dimensionality and ubiquitous modality missingness present significant analytical challenges. Existing methods for incomplete multi-omics data are scarce, do not fully leverage both modality-specific and shared information, and produce task-biased representations. We propose JASMINE, a self-supervised representation learning method for incomplete multi-omics data that preserves both modality-specific and joint information and enhances sample similarity structure. JASMINE produces embeddings that achieve superior performance across multiple tasks for two different incomplete multi-omics datasets while requiring only a single round of training per dataset.

missing data↗

PubMed2025

Target and biomarker exploration portal for drug discovery.

MOTIVATION: The discovery of novel drug targets and precision biomarkers remains a major challenge in drug development, with traditional differential expression analysis often overlooking key regulatory proteins. Here, we present a novel, web-based bioinformatics tool, the Target and Biomarker Exploration Portal (TBEP), designed to accelerate the drug discovery process by integrating large-scale biomedical data with network analysis techniques. RESULTS: TBEP harnesses machine-learning approaches to mine and combine multimodal datasets, including human genetics, functional genomics, and protein-protein interaction networks, to decode causal disease mechanisms and uncover novel therapeutic targets and precision biomarkers for specific phenotypes. A unique feature of the tool is its ability to process large-scale data in real-time, facilitated by an efficient cloud-based architecture. Additionally, the tool incorporates an integrated large language model (LLM), which assists researchers in exploring and interpreting complex biological relationships within the generated networks and multi-omics data using natural language (English). By offering an intuitive, interactive interface, the LLM enhances the exploration of biological insights, making it easier for scientists to derive actionable conclusions. This powerful integration of network analysis, multi-omics data, and LLM provides a robust framework for accelerating the identification of novel drug targets. AVAILABILITY AND IMPLEMENTATION: The tool is publicly available at https://tbep.missouri.edu. The source code, documentation and installation instructions are available at GitHub repository: https://github.com/mizzoudbl/tbep.

Drug Discovery↗

PubMed1990

The behavioral medicine service. An administrative model for biopsychosocial medical care, teaching, and research.

The biopsychosocial model has been promoted for its advantages in many important health problems. However, the lack of administrative practice models specifically designed to develop and promote the model hinder the development of systematic clinical applications. This article describes a successful clinical teaching and research practice, The Behavioral Medicine Service, that was conceptualized and developed based on the biopsychosocial model. Systematic planning enabled the service to take advantage of clinical, research, and teaching opportunities and to reduce the constraints imposed on development from within psychiatry and the institution. Advantages of the organizational model include multidisciplinary treatment teams, a systematic method of biopsychosocial assessment and management planning, and continuity of care between several settings: the medical-surgical wards of a general hospital; a behavioral medicine inpatient unit; and outpatient subspecialty clinics for chronic pain, chronic medical illness, anxiety and stress-related disorders, and drug and alcohol abuse. The Human Behavioral Pharmacology Laboratory forms the research arm of the service. Referrals of a diversity of medical and psychiatric problems create a unique learning opportunity for residents. Billing for multimodal team treatment, training residents, establishing clinical research, and managing a plethora of referrals were developmental challenges addressed by the service.

Behavioral Medicine↗

PubMed2026

Livestock Multi-Omics Integration: A Systematic Framework From Statistical Association to Causal Interpretation.

Livestock multi-omics integration is key to unraveling complex trait regulation, yet systematic, livestock-specific strategies remain scarce. This review traces the progression from single-omics accumulation to multi-dimensional integration, highlighting how large-scale genomic, epigenomic, and transcriptomic projects lay the foundation for functional dissection. We identify core impediments: extreme species diversity, marked data heterogeneity, limited sample sizes, and a pervasive reduction of multi-omics data to simplistic differential screens, resulting in low translational efficiency. We critically appraise four common pitfalls-overinterpreting correlation as causation, relegating proteomics to corroborating transcriptomics, incomplete microbiome-host integration lacking environmental context, and systematic neglect of metabolic fluxomics-and show how exposomics and fluxomics add necessary causal and dynamic dimensions. To address these, we propose a livestock-adapted three-tier analytical framework: (1) statistical association of cross-omics covariation patterns; (2) machine learning-driven feature mining and integrative modeling; and (3) causal interpretation encompassing Mendelian randomization, prior-knowledge-guided network inference, and physical causal evidence via fluxomics and metabolic control analysis. We further discuss how multimodal sequencing (single-cell, spatial, temporal) and generative AI can fundamentally mitigate heterogeneity and strengthen causal evidence. Finally, we outline future priorities in database standardization, livestock-specific benchmarking, and translational pipelines, charting a path from correlation-centric reporting to mechanistic causality and precision breeding.

Animals↗

PubMed2025

Integration of single cell multiomics data by deep transfer hypergraph neural network.

Multi-omics characterization of individual cells offers remarkable potential for analyzing the dynamics and relationships of gene regulatory states across millions of cells. How to integrate multimodal data is an open problem, existing integration methods struggle with accuracy and modality-specific biological variation retention. In this paper, we present scHyper (scalable, interpretable machine learning for single cell integration), a low-code and data-efficient deep transfer model designed for integrating paired and unpaired single-cell multimodal data. We benchmark scHyper against datasets from different multimodal data. ScHyper learns a low-dimensional representation and aligns the covariance matrices of the measured modalities, achieving high accuracy even with large scale atlas-level datasets with low memory and computational time across different cell lines, shedding light on regulatory relationships between different types of omics. Altogether, we show that scHyper is a versatile and robust tool for cell-type label transfer and integration from multimodal single-cell datasets.

Single-Cell Analysis↗

PubMed2026

Survival prediction for clear cell renal cell carcinoma based on deep multimodal synergistic survival network.

Objective.To propose a deep multimodal synergistic survival analysis framework (Deep Multimodal Synergistic Survival Network, DMSSN) to achieve accurate prognostic analysis for clear cell renal cell carcinoma (ccRCC).Methods.This study (DMSSN) utilized matched multimodal data from the Cancer Genome Atlas-KIRC database, including CT imaging data, whole slide images, copy number variation (CNV) features, and clinical data. Deep Canonical Correlation Analysis was employed to map heterogeneous modalities into a shared latent space. Contrastive learning was introduced to enhance semantic consistency across multimodal features, and a gating network was utilized for the adaptive fusion of multimodal information to achieve precise survival risk prediction for patients.Results.Experimental results demonstrated that DMSSN achieved a Concordance Index (C-index) of 0.8153 ± 0.0994, with a Log-rank testp-value of 1.6553×10-11. DMSSN exhibited significant performance advantages over traditional statistical methods like Log-rank-Cox (0.7055 ± 0.0670) and machine learning methods such as Random Survival Forest (RSF) (0.6836 ± 0.1048). Furthermore, in comparison with similar deep learning approaches, DMSSN outperformed late fusion strategies (0.7493 ± 0.1211) and discrete-time survival models such as DeepHit (0.7655 ± 0.1041) and Nnet-surv (0.7694 ± 0.0635). Notably, DMSSN still achieved the best predictive performance when compared to the classic deep survival model DeepSurv (0.7919 ± 0.0978) and advanced state-of-the-art multimodal fusion frameworks like Context-Aware Transformer (0.7735 ± 0.0818) and Multimodal Co-Attention Transformer (0.8102 ± 0.0972). Ablation studies showed that removing any single modality led to a decline in performance, with the largest numerical decrease occurring after removing CT imaging features (C-index decreased to 0.7327), validating the complementarity of multimodal data and the pivotal role of radiomic features in prognostic assessment. Module ablation experiments further confirmed the effectiveness of the core components.Conclusion:By effectively integrating imaging, pathology, genomic, and clinical features, the DMSSN framework demonstrates superior performance and robustness in the survival prediction of ccRCC.

Carcinoma, Renal Cell↗

PubMed2025

Alzheimer's subtypes A supervised, unsupervised, multimodal, multilayered embedded recursive (SUMMER) AI study.

Since Alzheimer's disease (AD) is a heterogeneous disease, different subtypes may have distinct biological, genetic, and clinical characteristics, requiring tailored interventions. While several proposed subtypes of AD exist, there is still no clear consensus on a definitive classification. By leveraging complementary AI approaches, including supervised and unsupervised learning, within a recursive pipeline (SUMMER) that integrates multimodal datasets encompassing MRI measurements, phenotypes, and genetic data, our goal was to generate robust scientific evidence for identifying AD subtypes. Data was downloaded from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database and included neuroimaging data (MRI), genetics (SNPs), clinical diagnosis, and demographics. 1133 European American participants' images, aged 55-95, were included in this study. The analysis was multi-fold, where the first step involved applying an unsupervised application to a subset of the MRI sample (AD + cognitively normal (CN) aged matched groups, 100 men aged 68-85 years, and 76 women aged 68-85 years). The MRI brain gray matter was segmented into 44 regions of interest (ROIs) according to a standard atlas, and 618 features were extracted, including ROI voxel intensity measurements such as minimum, maximum, and histogram variables. Results identified a cluster of subtype AD men and a cluster of subtype AD women that were distinct from the rest of their respective samples. In the next step, the integrity of the identified subtype AD clusters was investigated using the XGBoost supervised machine learning application with genetic features (SNPs, N=36,724) and labels: the identified subtype AD cluster vs. the rest of the sample, stratified by sex. A significant AD subtype men model (accuracy=0.85, F1=0.72, AUC=0.83) and a significant women AD subtype model (accuracy=0.81, F1=0.81, AUC=0.81) were built, confirming the homogeneity of the isolated AD subtype clusters. Discriminative biomarkers were extracted from the significant models, including selected ROIs and SNPs. Finally, the subtype models were tested on an unseen subset of ADNI data. The genetic-based models identified clusters of AD subtype participants consisting of 34% of the men AD group and 47% of the women AD group. Phenotypic analysis indicates that lower body weight was associated with the women's AD subtype. Complex diseases like AD demand a sophisticated, multimodal approach for precise diagnosis. Effectively identifying disease subtypes enhances the potential for personalized treatment, ultimately improving patient outcomes.

Journal Article↗

PubMed2026

Advancing cancer detection and treatment using longitudinal routine clinical data.

Cancer management remains fragmented across its continuum, from late-stage diagnosis and salvage therapies to non-personalized surveillance. Here, we present Oncoformer, a unified multimodal transformer model trained on the China Oncology Multimodal Prediction and Surveillance Study (COMPASS) cohort (3.67 million individuals, 17.7 million clinical visits) and validated on independent external cohorts, including the UK Biobank. Oncoformer integrates longitudinal electronic health records with chest X-ray imaging to address multiple clinical tasks: pan-cancer diagnosis (area under the receiver operating characteristic curve [AUROC] = 0.956), future cancer prediction up to 1 year before diagnosis (AUROC = 0.869), tumor stage inference (mean AUROC > 0.90), patient-specific treatment-response forecasting, and recurrence-free survival stratification across ten cancer types (all p < 0.01). Staging predictions were independently validated against postoperative pathological endpoints and shown to converge on core cancer genomic pathways. By translating routine clinical data into a dynamic view of cancer evolution, Oncoformer provides a framework for risk-informed cancer prediction and treatment stratification using routine clinical data.

Humans↗

PubMed2024

Digital pathology and spatial omics in steatohepatitis: Clinical applications and discovery potentials.

Steatohepatitis with diverse etiologies is the most common histological manifestation in patients with liver disease. However, there are currently no specific histopathological features pathognomonic for metabolic dysfunction-associated steatotic liver disease, alcohol-associated liver disease, or metabolic dysfunction-associated steatotic liver disease with increased alcohol intake. Digitizing traditional pathology slides has created an emerging field of digital pathology, allowing for easier access, storage, sharing, and analysis of whole-slide images. Artificial intelligence (AI) algorithms have been developed for whole-slide images to enhance the accuracy and speed of the histological interpretation of steatohepatitis and are currently employed in biomarker development. Spatial biology is a novel field that enables investigators to map gene and protein expression within a specific region of interest on liver histological sections, examine disease heterogeneity within tissues, and understand the relationship between molecular changes and distinct tissue morphology. Here, we review the utility of digital pathology (using linear and nonlinear microscopy) augmented with AI analysis to improve the accuracy of histological interpretation. We will also discuss the spatial omics landscape with special emphasis on the strengths and limitations of established spatial transcriptomics and proteomics technologies and their application in steatohepatitis. We then highlight the power of multimodal integration of digital pathology augmented by machine learning (ML)algorithms with spatial biology. The review concludes with a discussion of the current gaps in knowledge, the limitations and premises of these tools and technologies, and the areas of future research.

Humans↗