PubMed Health⌕ Search

SEARCH · PubMed Health

Results for “multimodal modeling”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13Linked to original sources

A state space model for the HIV epidemic in homosexual populations and some applications.

In this paper we have developed a state space model for the HIV epidemic in homosexual populations which have been divided into subpopulations according to sexual activity levels. In this model, the stochastic dynamic system model is the stochastic model of the HIV epidemic in terms of the chain multinomial model whereas the observation model is a statistical model based on the observed AIDS incidences. This model is applied to the San Francisco homosexual population for estimating the numbers of susceptible people, infective people and AIDS cases and for estimating the probabilities of HIV transmission from infective people to susceptible people given sexual contacts. The results show that the estimated numbers of AIDS incidence trace closely the observed numbers indicating the usefulness of the model. It is observed that the estimated numbers of latent people show multimodal curves and that HIV infection takes place during the primary stage and very late stage. The results have further shown that there are significant differences between the observed AIDS incidences and the estimates by the embedded deterministic model. These results indicate that using the embedded deterministic model to estimate the HIV-infected people and to predict future AIDS cases can be very misleading in some cases.

Acquired Immunodeficiency Syndrome↗

Automatic generation of plans for biomedical image interpretation.

This paper presents a new object-centered, goal-driven planning approach to biomedical image interpretation. We describe here a prototype system which takes advantage of spatial and detectability constraints from an expert-derived model of expected anatomical structures to automatically generate plans for the interpretation of multimodality images.

Artificial Intelligence↗

Multisensory perception: beyond modularity and convergence.

Recent research on multisensory perception suggests a number of general principles for crossmodal integration and that the standard model in the field--feedforward convergence of information--must be modified to include a role for feedback projections from multimodal to unimodal brain areas.

Brain↗

NeuroOmics-Net: An interpretable multimodal deep learning framework for Alzheimer's disease diagnosis and progression prediction using neuroimaging, EEG, and genomic data.

Accurate diagnosis and progression prediction of Alzheimer's disease (AD) remain challenging due to the heterogeneous nature of the disease, which involves structural brain degeneration, electrophysiological dysfunction, and molecular dysregulation. Most existing deep learning approaches rely on a single modality or limited multimodal combinations, thereby failing to capture the complex cross-domain interactions underlying AD progression. Furthermore, the scarcity of large-scale datasets containing synchronized neuroimaging, electrophysiological, and genomic measurements restricts the development of comprehensive multimodal diagnostic systems. To address these challenges, this study proposes NeuroOmics-Net, a multimodal deep learning framework for Alzheimer's disease analysis that integrates structural magnetic resonance imaging (sMRI), electroencephalography (EEG), and gene expression data. The proposed framework combines a Hierarchical Multi-View Encoder (HME) for modality-specific feature extraction, a Cross-Omics Attention Fusion (CAF) module for adaptive integration of complementary biomarkers, and a Disease Progression Graph Learning (DPGL) module for modeling progression-related relationships across biological domains. To facilitate cross-modal integration from independent cohorts, Regularized Canonical Correlation Analysis (RCCA) is employed to align heterogeneous feature representations within a shared latent space. Experiments were conducted using publicly available datasets from ADNI, PhysioNet, and GEO repositories comprising 1120 diagnosis-aligned samples. The proposed framework achieved 94.3% classification accuracy and an AUC of 0.975 for distinguishing normal controls (NC), mild cognitive impairment (MCI), and Alzheimer's disease subjects, while attaining 93.7% accuracy for predicting conversion from stable mild cognitive impairment (sMCI) to progressive mild cognitive impairment (pMCI). However, a fairness sensitivity analysis using stratified demographic reweighting revealed accuracy ranging from 90.8% (low-education, high-comorbidity proxy subgroup) to 96.1% (low-risk, high-reserve proxy subgroup), a demographic parity gap of 5.3 percentage points, indicating that overall accuracy reflects a performance ceiling in a relatively homogeneous research cohort rather than a realistic estimate for demographically diverse clinical populations. Comparative evaluations demonstrated consistent improvements over state-of-the-art unimodal and multimodal deep learning models. Interpretability analysis further identified clinically relevant biomarkers, including hippocampal and entorhinal atrophy, theta-alpha EEG alterations, and APOE-associated molecular pathways. Because sMRI, EEG, and gene expression data were sourced from separate, unpaired cohorts with no subjects possessing all three synchronized measurements, all reported cross-modal associations reflect population-level statistical correspondence across diagnosis-matched groups rather than within-subject physiological coupling; no claim of intra-individual causal cross-modal interaction is made. These findings demonstrate that NeuroOmics-Net provides an effective computer-aided framework for multimodal biomedical data processing and Alzheimer's disease analysis. By integrating neuroimaging, electrophysiological, and genomic information, the proposed approach enables accurate diagnosis, progression prediction, and biologically interpretable decision support for clinical and translational applications.

Humans↗

P53 expression, p53 and Ha-ras mutation and telomerase activation during nitrosamine-mediated hamster pouch carcinogenesis.

Squamous cell carcinomas (SCC) induced in hamster buccal pouch (HBP) by 22 weeks of topical N-methyl-N-benzylnitrosamine (MBN) treatment (twice-weekly, 10 mg MBN/ml propylene glycol) were evaluated for: (i) altered expression of p53 using immunohistochemistry (IHC); (ii) mutations in Ha-ras and p53 using PCR/single strand conformation polymorphism (SSCP); (iii) telomerase activity using the telomerase repeat amplification protocol (TRAP). Precancerous lesions were also evaluated using p53 IHC. Hamsters were killed for lesion analysis at either 3 days (group A, eight hamsters, 89 carcinomas) or 7 weeks (group B, six hamsters, 105 carcinomas) following the final MBN application. Between 3 days and 7 weeks post-treatment the proportion of tumors exhibiting p53 IHC activity (at least 10% of nuclei stained using D07 antibodies for detection of both mutant and wild-type p53) fell from 91 to 50%. However, during this same post-treatment period the frequency of tumors analyzed exhibiting confirmed sequence alterations in the conserved exons (E5-E8) of p53 remained constant (5/15 = 33% in group A versus 14/45 = 31% in group B). Heightened expression of wild-type p53 resulting from DNA damage in the immediate post-treatment period is likely to have contributed to the high proportion of group A tumors exhibiting p53 IHC activity. Nearly 80% of the identified p53 mutations were G-->A and C-->T transitions. The identified p53 point mutations occurred at or near (within three codons) of the corresponding hot-spot codons (175, 245, 248 and 273) of human oral SCC. The proportion of group A and group B tumors analyzed exhibiting Ha-ras mutations was 1/15 (7%) and 7/45 (16%), respectively. Only four of the observed eight Ha-ras mutations occurred in codons known to result in activation of this gene. Telomerase activation was demonstrated in 11 of 13 group A tumors (85%) and in 23 of 24 (96%) group B tumors analyzed. The alterations in p53, Ha-ras and telomerase activity observed in this HBP-MBN model are similar in many respects to those observed in the analogous human lesions of the head and neck. This model may be particularly useful for development of cancer chemoprevention regimens and multimodality cancer therapies.

Animals↗

Integrating quantitative and qualitative methodologies for the assessment of health care systems: emergency medicine in post-conflict Serbia.

BACKGROUND: Due to the complexity of health system reform in the post-conflict, post-disaster, and development settings, attempts to restructure health services are fraught with pitfalls that are often unanticipated because of inadequate preliminary assessments. Our proposed Integrated Multimodal Assessment - combining quantitative and qualitative methodologies - may provide a more robust mechanism for identifying programmatic priorities and critical barriers for appropriate and sustainable health system interventions. The purpose of this study is to describe this novel multimodal assessment using emergency medicine in post-conflict Serbia as a model. METHODS: Integrated quantitative and qualitative methodologies--system characterization and observation, focus group discussions, free-response questionnaires, and by-person factor analysis--were used to identify needs, problems, and potential barriers to the development of emergency medicine in Serbia. Participants included emergency and pre-hospital personnel from all emergency medical institutions in Belgrade. RESULTS: Demographic data indicate a loosely ordered network of part-time emergency departments supported by 24-hour pre-hospital services and an academic emergency center. Focus groups and questionnaires reveal significant impediments to delivery of care and suggest development priorities. By-person factor analysis subsequently divides respondents into distinctive attitudinal types, compares participant opinions, and identifies programmatic priorities. CONCLUSIONS: By combining quantitative and qualitative methodologies, our Integrated Multimodal Assessment identified critical needs and barriers to emergency medicine development in Serbia and may serve as a model for future health system assessments in post-conflict, post-disaster, and development settings.

Attitude of Health Personnel↗

A multimodal approach to managing the symptoms of multiple sclerosis.

Multiple sclerosis (MS) is a disease of the CNS with a challenging clinical course characterized by heterogeneous symptoms related to inflammation and demyelination. Disease-modifying agents (DMAs) are used to treat the related neuronal degradation. Certain symptoms occur regularly, although with variable frequency, regardless of treatment with DMAs. Because there is no cure for MS at this time, symptom management is critically important to quality of life. Symptoms commonly seen are spasticity, fatigue, sexual dysfunction, bladder dysfunction, pain, and cognitive dysfunction. Other symptoms include depression, bowel dysfunction, paroxysmal symptoms, and weakness. The symptom management model that provides optimal results for patients with MS is a multimodal approach using effective communication, patient education, physical modalities and activities, occupational and other therapies, and pharmacologic interventions. Individualizing treatment for each patient involves gaining control of symptoms as early as possible to prevent cycles of symptoms from developing.

Cognition Disorders↗

Survival prediction for clear cell renal cell carcinoma based on deep multimodal synergistic survival network.

Objective.To propose a deep multimodal synergistic survival analysis framework (Deep Multimodal Synergistic Survival Network, DMSSN) to achieve accurate prognostic analysis for clear cell renal cell carcinoma (ccRCC).Methods.This study (DMSSN) utilized matched multimodal data from the Cancer Genome Atlas-KIRC database, including CT imaging data, whole slide images, copy number variation (CNV) features, and clinical data. Deep Canonical Correlation Analysis was employed to map heterogeneous modalities into a shared latent space. Contrastive learning was introduced to enhance semantic consistency across multimodal features, and a gating network was utilized for the adaptive fusion of multimodal information to achieve precise survival risk prediction for patients.Results.Experimental results demonstrated that DMSSN achieved a Concordance Index (C-index) of 0.8153 ± 0.0994, with a Log-rank testp-value of 1.6553×10-11. DMSSN exhibited significant performance advantages over traditional statistical methods like Log-rank-Cox (0.7055 ± 0.0670) and machine learning methods such as Random Survival Forest (RSF) (0.6836 ± 0.1048). Furthermore, in comparison with similar deep learning approaches, DMSSN outperformed late fusion strategies (0.7493 ± 0.1211) and discrete-time survival models such as DeepHit (0.7655 ± 0.1041) and Nnet-surv (0.7694 ± 0.0635). Notably, DMSSN still achieved the best predictive performance when compared to the classic deep survival model DeepSurv (0.7919 ± 0.0978) and advanced state-of-the-art multimodal fusion frameworks like Context-Aware Transformer (0.7735 ± 0.0818) and Multimodal Co-Attention Transformer (0.8102 ± 0.0972). Ablation studies showed that removing any single modality led to a decline in performance, with the largest numerical decrease occurring after removing CT imaging features (C-index decreased to 0.7327), validating the complementarity of multimodal data and the pivotal role of radiomic features in prognostic assessment. Module ablation experiments further confirmed the effectiveness of the core components.Conclusion:By effectively integrating imaging, pathology, genomic, and clinical features, the DMSSN framework demonstrates superior performance and robustness in the survival prediction of ccRCC.

Carcinoma, Renal Cell↗

Sequential Monte Carlo for Bayesian matching of objects with occlusions.

We consider the problem of locating instances of a known object in a novel scene by matching the fiducial features of the object. The appearance of the features and the shape of the object are modeled separately and combined in a Bayesian framework. In this paper, we present a novel matching scheme based on Sequential Monte Carlo, in which the features are matched sequentially, utilizing the information about the locations of previously matched features to constrain the task. The particle representation of hypotheses about the object position allow matching in multimodal and cluttered environments, where batch algorithms may have convergence difficulties. The proposed method requires no initialization or predetermined matching order, as the sequence can be started from any feature. We also utilize a Bayesian model to deal with features that are not detected due to occlusions or abnormal appearance. In our experiments, the proposed matching system shows promising results, with performance equal to batch approaches when the target distribution is unimodal, while surpassing traditional methods under multimodal conditions. Using the occlusion model, the object can be localized from only a few visible features, with the nonvisible parts predicted from the conditional prior model.

Algorithms↗

Multimodal integration in the feeding behaviors of predatory teleost fishes.

The search for useful model systems to study sensory processing in vertebrate nervous systems has resulted in many neuroethological studies investigating the roles played by a single sensory modality in a given behavior. However, animals behaving in a complex, three-dimensional environment receive a large amount of information from external and internal receptor arrays. Clearly, the integration of sensory afference arising from different modalities into a coherent 'gestalt' of the world is essential to the behaviors of most animals. Over the past several years researchers in my laboratory have examined the roles played by the visual and lateral line sensory systems in organizing the feeding behavior of two species of predatory teleost fishes, the largemouth bass, Micropterus salmoides and the muskellunge, Esox masquinongy. The free-field feeding behaviors of these fishes was studied quantitatively in intact animals and compared to animals in which the lateral line and visual systems had been selectively suppressed. The data show that both bass and muskie employ similar approach and strike behaviors. Vision is crucial to the initial detection of, and orientation to, prey. Lateral line and vision together determine the optimum distance and angular deviation for the initiation of a rapid strike toward the prey. Blinded animals are able to strike accurately at prey at very close ranges and small angular deviations, indicating that this modality presents sufficient information to direct the behavior during the final phases of the strike. The results demonstrate that there is a hierarchy of senses involved in feeding behavior, with different modalities playing critical roles in succeeding phases.

Animals↗

Data fusion in medical imaging: merging multimodal and multipatient images, identification of structures and 3D display aspects.

Data fusion in medical imaging can be seen into two ways (i) multisensors fusion of anatomical and functional information and (ii) interpatient data fusion by means of warping models. These two aspects set the methodological framework necessary to perform anatomical modelling especially when concerning the modelling of brain structures. The major relevance of the work presented here concerns the interpretation of multimodal 3D neuro-anatomical data bases. Three types of data fusion problems are considered in this paper. The first one concerns the problem of data combination which includes multimodal registration (multisensor fusion applied to CT, MRI, DSA, PET, SPECT, or MEG). In particular, the problem of warping patient data to an anatomical atlas is reviewed and a solution is proposed. The second problem of data fusion addressed in this paper is the identification of anatomical structures by means of image analysis methods. Two techniques have been developed. The first one deals with the analysis of image geometrical features to end up with the determination of a fuzzy mask to label the structure of interest. The second technique consists of labelling major cerebral structures by means of statistical image features associated with relaxation techniques. Finally, the paper presents a review of up to date 3D display techniques with a special emphasis on volume rendering and 3D display of combined data.

Algorithms↗

19. Correlation between FDG PET data and EEG dipole modeling.

Purpose: FDG-PET images and EEG dipole modelling were used to localization of interictal epileptogenic foci. A multimodality approach with the analysis of FDG-PET images, EEG dipoles and anatomical images (MRI) were applied to patients with drug-resistant epilepsy.Methods: Source location was determined using Brain Electromagnetic Source Analysis (BESA) program. The dipole location provided by BESA was then transformed into PET co-ordinates using the patient's MRI previously registered with the PET image. As a difference with other methods, no external markers are necessary.Results: The study group includes ten drug-resistant epileptic patients. FDG hypometabolism was found in all patients. Abnormalities in glucose uptake were always ipsilateral to the EEG dipole. However, quantitative analysis of FDG-PET within hypometabolic areas showed no significant correlation between decrease glucose uptake and location of EEG-dipole source. The comparison of the results using both methods shows that the dipole location matches the FDG-PET hypometabolic area for all the patients. Combining the spatial localization on FDG-PET with the temporal accuracy of EEG dipole source aids in the exact localization of the epileptogenic focus.Conclusion: As a conclusion, the results show that projection of EEG dipole data onto FDG-PET may play a key role in the indication of surgery for the treatment of drug-resistant epileptic patients, provided it is simple and easy to perform.

Journal Article↗

Electrotonic architecture of hippocampal CA1 pyramidal neurons based on three-dimensional reconstructions.

1. The spread of electrical signals in pyramidal neurons from the CA1 field of rat hippocampus was investigated through multicompartmental modeling based on three-dimensional morphometric reconstructions of four of these cells. These models were used to dissect the electrotonic architecture of these neurons, and to evaluate the equivalent cylinder approach that this laboratory and others have previously applied to them. Robustness of results was verified by the use of wide ranges of values of specific membrane resistance (Rm) and cytoplasmic resistivity. 2. The anatomy exhibited extreme departures from a key assumption of the equivalent cylinder model, the so-called "3/2 power law." 3. The compartmental models showed that the frequency distribution of steady-state electrotonic distances between the soma and the dendritic terminations was multimodal, with a large range and a sizeable coefficient of variation. This violated another central assumption of the equivalent cylinder model, namely, that all terminations are electronically equidistant from the soma. This finding, which was observed both for "centrifugal" (away from the soma) and "centripetal" (toward the soma) spread of electrical signals, indicates that the concept of an equivalent electrotonic length for the whole dendritic tree is not appropriate for these neurons. 4. The multiple peaks in the electrotonic distance distributions, whether for centrifugal or centripetal voltage transfer, were clearly related to the laminar organization of synaptic afferents in the CA1 region. 5. The results in the three preceding paragraphs reveal how little of the electrotonic architecture of these neurons is captured by a simple equivalent cylinder model. The multicompartmental model is more appropriate for exploring synaptic signaling and transient events in CA1 pyramidal neurons. 6. There was significant attenuation of synaptic potential, current, and charge as they spread from the dendritic tree to the soma. Charge suffered the least and voltage suffered the most attenuation. Attenuation depended weakly on Rm and strongly on synaptic location. Delay of time to peak was more distorted for voltage than for current and was more affected by Rm. 7. Adequate space clamp is not possible for most of the synapses on these cells. Application of a somatic voltage clamp had no significant effect on voltage transients in the subsynaptic membrane. 8. The possible existence of steep voltage gradients within the dendritic tree is consistent with the idea that there can be some degree of local processing and that different regions of the neuron may function semiautonomously. These spatial gradients are potentially relevant to synaptic plasticity in the hippocampus, and they also suggest caution in interpreting some neurophysiological results.

Animals↗

Circular permutation analysis of phage T4 DNA by electron microscopy.

Phage T4 is known to have a linear duplex chromosome that is circularly permuted and terminally repeated. We found, by denaturation and self-reannealing experiments, that circular permutation in T4 native DNA is not random. Their multimodal distribution of permutation is compatible with the "headful packaging" model with the additional specifications that the encapsulation of DNA starts at several sites and these are not random distributed.

DNA, Circular↗

The treatment of Wilms tumor. Results of the National Wilms Tumor Studies.

The study of Wilms tumor has provided a model for the evaluation of the histopathology, prognostic factors, and multimodality therapy regimens in children and adults with rare malignant tumors. The NWTS Group has provided the scientific and administrative framework for the assessment of various treatment schemes in children with renal tumors, a research goal that could not have been realized using the resources available to either a single or a small group of institutions. These studies have yielded higher survival rates while demonstrating that therapy can be individualized according to well-defined risk factors most accurately identified in non-pretreated tumors. Thus, it has been found that treatments associated with significant acute and/or long-term morbidity are not necessary for patients with stage I or II, favorable histology Wilms tumor. Future studies may allow more accurate identification of those patients with stage III and IV/favorable histology Wilms tumor for whom treatment with an anthracycline is not necessary. If successful, this would decrease the size of the group of survivors at risk for cardiac dysfunction, which can follow the administration of even modest cumulative dosages of these agents.

Child↗

The role of circulating tumor DNA (ctDNA) to detect minimal residual disease in locally advanced gastroesophageal carcinoma: the BUTTERFLY study.

BACKGROUND: Despite advances in perioperative and neoadjuvant strategies, patients with locally advanced gastroesophageal cancers remain at high risk of recurrence after curative intent treatment. No validated biomarkers are available to detect minimal residual disease (MRD) or to guide post-operative risk-adapted management. Circulating tumor DNA (ctDNA) has emerged as a noninvasive tool for disease monitoring; single-parameter or tumor-informed assays, however, may lack sensitivity in low-tumor burden settings. Multimodal, tumor-agnostic approaches may overcome these limitations. METHODS: The BUTTERFLY study is a prospective, multicenter observational study enrolling patients with stage II-III gastric, gastroesophageal junction, or esophageal cancer treated with perioperative chemotherapy or neoadjuvant chemoradiotherapy followed by surgery. It evaluates the diagnostic performance and prognostic value of an academic, tumor-agnostic, multimodal ctDNA assay for MRD detection and prognostic stratification. Serial plasma samples are collected from baseline through post-operative follow-up and at relapse. Cell-free DNA is analyzed using the Agnostic Liquid Biopsy Multimodal Advancement (ALMA) platform, integrating tumor fraction estimation, somatic copy number alterations, fragmentomic features, single-nucleotide variants, and whole-genome methylation profiling. Multimodal features are combined with clinical variables using machine learning-based models to enhance MRD detection and relapse risk stratification. The primary endpoint includes sensitivity and specificity of ALMA-defined ctDNA/MRD status at the 4-8 weeks after surgery landmark, whereas secondary endpoints assess diagnostic performance at other time points and associations between ctDNA status and dynamics with disease-free survival, overall survival, treatment response, and lead time to recurrence. FUTURE PERSPECTIVES: If validated, this tumor-agnostic, multimodal ctDNA approach may enable earlier molecular relapse detection and support personalized post-operative management strategies.

circulating tumor DNA (ctDNA)↗

Task-based imaging of colon cancer in the Apc(Min/+) mouse model.

Optical coherence tomography (OCT), laser-induced fluorescence (LIF), and laser-scanning confocal microscopy (LSCM) were used for the task of multimodal study of healthy and adenomatous mouse colon. The results from each modality were compared with histology, which served as the gold standard. The Apc(Min/+) genetic mouse model of colon cancer was compared with wild-type mice. In addition, a special diet was used for the task of studying the origins of a 680 nm autofluorescent signal that was previously observed in colon. The study found close agreement among each of the modalities and with histology. All four modalities were capable of identifying diseased tissue accurately. The OCT and LSCM images provided complementary structural information about the tissue, while the autofluorescence signal measured by LIF and LSCM provided biochemical information. OCT and LIF were performed in vivo and nondestructively, while the LSCM and histology required extraction of the tissue. The magnitude of the 680 nm signal correlates with chlorophyll content in the mouse diet, suggesting that the autofluorescent compound is a dietary metabolite.

Adenomatous Polyposis Coli Protein↗

TCRspec: A Recognition Interface-Informed Multimodal Method for TCR-pMHC Specificity Prediction.

Specific recognition between T-cell receptors (TCRs) and peptide-major histocompatibility complexes (pMHCs) is central to adaptive immunity, yet accurate prediction of TCR-pMHC specificity remains challenging. Existing models mainly rely on sequence features or isolated molecular structures, limiting their ability to capture interface-level determinants within the ternary recognition complex. Here, we constructed the multimodal TCR-pMHC ternary complex (MM-TCR) data set, integrating paired TCR-pMHC sequences, V/J gene annotations, and modeled TCR-pMHC complex structures refined by short molecular dynamics-based relaxation. Based on MM-TCR, we developed TCRspec, an interpretable multimodal framework combining sequence embeddings, gene-usage features, and complex-level structural representations. Under a stringent CD-HIT TCR-cluster-disjoint split, TCRspec achieved an average AUROC of 0.896 and AUPRC of 0.882 across seven antigen-specific test data sets, outperforming representative baseline models. Cross-validation and ablation analyses confirmed the contribution of ternary complex structural information and MD-refined structures. In independent OOD peptide-TCR systems, TCRspec retained discriminative performance and identified model-inferred peptide positions associated with TCR recognition, providing a structure-informed framework for TCR specificity prediction.

Receptors, Antigen, T-Cell↗