PubMed HealthSearch

SEARCH · PubMed Health

Results for “Multimodal learning”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5Linked to original sources

Biological Foundation Models for Complex Disease Research and Clinical Translation.

Complex diseases, including cancer, rare genetic disorders, neurodevelopmental and psychiatric conditions, and neurodegenerative diseases, arise from interactions among genetic variation, gene regulation, and cellular states that are difficult to capture using a single data type or biological scale. Biological foundation models address this challenge by treating nucleotides and genes as tokens and learning representations that can be transferred to downstream biomedical and clinical tasks. In this review, we examine two major model classes, genomic sequence foundation models and cell foundation models, and compare their tokenization strategies, model architectures, pretraining objectives, and adaptation methods. We summarize their emerging applications in regulatory variant interpretation, disease-associated cell-state analysis, drug-response prediction, and therapeutic target discovery across complex diseases. We distinguish applications supported by experimental or retrospective validation from those that remain primarily computational or conceptual. We further discuss key challenges to clinical translation, including multimodal data integration, model interpretability, benchmarking, patient-specific prediction, and privacy protection. We highlight future opportunities to integrate biological foundation models with emerging frameworks of medical digital twins, agentic AI, and federated learning. By linking model design to translational goals, this review provides a practical framework for evaluating biological foundation models and their readiness for complex disease research and clinical use.

biological foundation model

AI-Supported, Integrative Prediction of Postoperative Delirium: Protocol for the CONFUSED Study.

BACKGROUND: Postoperative delirium (POD) is a frequent and serious complication in older surgical patients, characterized by acute cognitive dysfunction and fluctuating levels of consciousness. POD is associated with prolonged hospitalization, long-term cognitive decline, reduced quality of life, and increased mortality. Despite its clinical relevance, the underlying pathophysiological mechanisms remain poorly understood, and reliable biomarkers for early prediction and prevention are lacking. OBJECTIVE: The CONFUSED study aims to identify molecular and clinical predictors of POD by integrating clinical data with proteomic, transcriptomic, and epigenetic analyses. The primary objective is to develop predictive models for POD using multimodal data. Secondary objectives include the identification of delirium-associated genes, proteins, and epigenetic signatures, as well as the exploration of patient subgroups at increased risk for POD. METHODS: CONFUSED is a prospective observational cohort study conducted at a German university hospital. Adult patients undergoing major surgery under general anesthesia will be enrolled until 100 cases of POD have been observed, which is expected to require a total sample size of approximately 200 to 300 patients. Blood samples are collected at 4 predefined time points: before premedication, immediately after surgery, and on postoperative days 2 and 5. Samples undergo comprehensive proteomic profiling, transcriptomic analysis using RNA microarrays, DNA methylation analysis, and genotyping of selected polymorphisms. Clinical data, including demographics, comorbidities, perioperative variables, medications, and delirium assessments using the Confusion Assessment Method (CAM) and CAM for the intensive care unit, are systematically recorded. Statistical analyses include univariate and multivariate methods, as well as machine learning approaches such as random forests and support vector machines, to identify relevant biomarkers and develop predictive models. The study protocol follows STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) and TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) guidelines and was approved by the responsible ethics committees. RESULTS: The study was registered in the German Clinical Trials Register (DRKS00033854) on March 18, 2024. Recruitment started in January 2024 and is ongoing at the time of manuscript submission. As of now, 135 patients have been enrolled. Sample collection and laboratory analyses are ongoing. Data analysis began in January 2026, with first results anticipated in July 2026. Final data lock is anticipated after the completion of recruitment. CONCLUSIONS: By integrating multimodal molecular data with clinical parameters and applying advanced machine learning techniques, the CONFUSED study aims to improve the prediction and understanding of POD. The results are expected to support the development of personalized preventive strategies and contribute to improved perioperative care for patients at risk of POD.

Humans

Amygdala role in conditioned associative learning.

Amygdala role in emotion was reviewed in reference to recent amygdala lesion studies and neuronal responses in the rat amygdala to conditioned stimuli. Extensive lesion studies suggest that the amygdala is crucial in various kinds of motivated and emotional behavior, and related autonomic responses. These amygdala functions critically depend on learning and memory. Amygdala lesions, both before and after training of conditioned associative learning, impaired emotional expression without simple sensory-motor deficits. Pharmacological experiments indicated neurotransmission in the amygdala is mediated through NMDA and AMPA receptors. These results strongly suggest the amygdala involvement in acquiring and storing associative memory (i.e. stimulus-affect association), by which animals recognize and evaluate the biological significance of a stimulus. This information is then transferred to the brainstem executing system. In the neurophysiological experiments, there were topographic distributions of sensory-responsive neurons within the amygdala, which were well correlated to anatomical data. The responses of rat amygdala neurons changed plastically during learning. Furthermore, more sensory-responsive neurons were encountered in the amygdala of rats trained to associate the sensory stimuli with a reinforcement than in the amygdala of rats that were not trained. In trained rats, multimodal neurons that responded to conditioned and unconditioned stimuli were frequently found in the basolateral and central nuclei of the amygdala. The results suggest that basolateral and central nuclei are foci where various sensory modalities converge, and which might perform critical functions in acquiring and storing long-term associative memory to link between sensory information and affective significance.

Amygdala

The psychology of polio as prelude to post-polio sequelae: behavior modification and psychotherapy.

Even as the physical causes and treatments for post-polio sequelae (PPS) are being identified, psychological symptoms--chronic stress, anxiety, depression, and compulsive, Type A behavior--are becoming evident in polio survivors. Importantly, these symptoms are not only causing marked distress but are preventing patients from making the lifestyle changes necessary to treat their PPS. Neither clinicians nor polio survivors have paid sufficient attention to the acute polio experience, its conditioning of life-long patterns of behavior, its relationship to the development of PPS, and its effect on the ability of individuals to cope with and treat their new symptoms. We describe the acute polio and post-polio experiences on the basis of patient histories, relate the experience of polio to the development of compulsive, Type A behavior, link these behaviors to the physical and psychological symptoms reported in the National Post-Polio Surveys, and present a multimodal behavioral approach to treatment.

Activities of Daily Living

Behavioural and neurohistological changes in aging Sepia.

The life cycle of the cuttlefish in the English Channel is characterized by a succession of homogeneous population cohorts. These conditions provide an excellent opportunity for the study of aging in this cephalopod. In a first longitudinal study, we considered the oldest animals and compared their success rates at the first capture attempt. During the first weeks of the study, the results remained constant and then, during the weeks immediately preceding the natural death, a dramatic drop was observed. This deterioration may be due to defects of visuomotor coordination. In a second study, we used an associative learning protocol with negative reinforcement and the performances of young and old animals were compared. The most striking results showed that the performances of the oldest animals during the retention test were very mediocre. Such results suggest that the long-term memory process is affected. Finally, a modification of the Fink-Heimer silver stain enabled us to draw a map of spontaneous terminal degeneration in the central nervous system of the oldest animals. The structures which are characterized by the presence of multimodal inputs (the spine of the peduncle lobe and the basal lobes) present the most obvious signs of degeneration.

Aging

Concepts of a Web-based open distributed textbook for the multimodal diagnostics of gastrointestinal tumours with MRI, CT and video-endoscopy addressing students of medicine and students of medical informatics as two different target groups.

Multimodal diagnostics of gastrointestinal tumours with MRI, CT and video-endoscopy is a rapidly changing domain. The education at our universities should overcome the obstacles of traditional learning based on paper media and oral lectures with retention rates of 10-30% only. The paper presents the objectives and the results of the design phase of the project ODITEB1-Open Distributed TExt Book, for Computer-Assisted Instruction in the domain mentioned above. The main objective is to produce an electronic interactive textbook in order to shift education to more efficient learning settings with higher retention rates. The main concepts are 1) three-layer architecture (dynamic case layer, intermediate query layer, static instruction layer) 2) case pool distribution 3) active learners experience (interactive exploration of original image data).

Computer-Assisted Instruction

Academic underachievement, attention deficits, and aggression: comorbidity and implications for intervention.

Although comorbidity with specific learning disabilities is less frequent than commonly reported, externalizing behavior disorders--particularly attention-deficit hyperactivity disorder (ADHD)--often overlap with various indices of academic underachievement during childhood. Furthermore, by adolescence, delinquency is clearly associated with school failure. Because the link between behavioral and learning problems often appears before formal schooling, and because the co-morbid problems predict a negative course, early intervention is a necessity. Controlled treatment investigations with youngsters who show these combined problems are rare, and such studies present a host of methodologic and practical problems. I discuss issues surrounding multimodality treatment programs and the potential for long-term interventions to break cycles of school failure and externalizing behavior.

Adolescent

Episodic retrieval activates the precuneus irrespective of the imagery content of word pair associates. A PET study.

The aim of this study was to evaluate further the role of the precuneus in episodic memory retrieval. The specific hypothesis addressed was that the precuneus is involved in episodic memory retrieval irrespective of the imagery content. Two groups of six right-handed normal male volunteers took part in the study. Each subject underwent six [15O]butanol-PET scans. In each of the six trials, the memory task began with the injection of a bolus of 1500 MBq of [15O]butanol. For Group 1, 12 word pair associates were presented visually, for Group 2 auditorily. The subjects of each group had to learn and retrieve two sets of 12 word pairs each. One set consisted of highly imaginable words and another one of abstract words. Words of both sets were not related semantically, representing 'hard' associations. The presentations of nonsense words served as reference conditions. We demonstrate that the precuneus shows consistent activation during episodic memory retrieval. Precuneus activation occurred in visual and auditory presentation modalities and for both highly imaginable and abstract words. The present study therefore provides further evidence that the precuneus has a specific function in episodic memory retrieval as a multimodal association area.

Acoustic Stimulation

Robotic multimodality stereotactic brain tissue identification: work in progress.

Real-time identification of tissue would improve procedures such as stereotactic brain biopsy (SBX), functional and implantation neurosurgery, and brain tumor excision. To standard SBX equipment has been added: (1) computer-controlled stepper motors to drive the biopsy needle/probe precisely; (2) multiple microprobes to track tissue density, detect blood vessels and changes in blood flow, and distinguish the various tissues being penetrated; (3) neural net learning programs to allow real-time comparisons of current data with a normative data bank; (4) three-dimensional graphic displays to follow the probe as it traverses brain tissue. The probe can differentiate substances such as pig brain, differing consistencies of the 'brain-like' foodstuff tofu, and gels made to simulate brain, as well as detect blood vessels imbedded in these substances. Multimodality probes should improve the safety, efficacy, and diagnostic accuracy of SBX and other neurosurgical procedures.

Animals

Do infants see emotional expressions in static faces?

To determine whether young infants discriminate photographs of different emotions on an affect-relevant basis or on the basis of isolated features unrelated to emotion, groups of 17-, 23-, and 29-week-olds were habituated to slides of 8 women posing either Toothy Angry, Nontoothy Angry, or Nontoothy Smiling facial expressions and were then shown 2 new women in the familiarized expression and in a novel Toothy Smiling expression. At all 3 ages, recovery to the novel Toothy Smiling faces occurred only after habituation to Nontoothy faces (whether smiling or angry), not after habituation to Toothy Angry faces, indicating that infants had been responsive to nonspecific features of the photographs (presence or absence of bared teeth) rather than to affectively relevant configurations of features. In a second experiment, 2 older age groups (35 and 41 weeks) also proved to be insensitive to affect-related aspects of still faces, though more so for angry than for happy expressions. It is suggested that the young infant's difficulty in extracting emotional information from static stimuli may be attributable to the absence of the critical invariants (dynamic, multimodally specified) that characterize naturalistic expressions of emotion.

Anger

Inferring Metabolic States from Single Cell Transcriptomic Data via Geometric Deep Learning.

The ability to measure gene expression at single-cell resolution has elevated our understanding of how biological features emerge from complex and interdependent networks at molecular, cellular, and tissue scales. As technologies have evolved that complement scRNAseq measurements with things like single-cell proteomic, epigenomic, and genomic information, it becomes increasingly apparent how much biology exists as a product of multimodal regulation. Biological processes such as transcription, translation, and post-translational or epigenetic modification impose both energetic and specific molecular demands on a cell and are therefore implicitly constrained by the metabolic state of the cell. While metabolomics is crucial for defining a holistic model of any biological process, the chemical heterogeneity of the metabolome makes it particularly difficult to measure, and technologies capable of doing this at single-cell resolution are far behind other multiomics modalities. To address these challenges, we present GEFMAP (Gene Expression-based Flux Mapping and Metabolic Pathway Prediction), a method based on geometric deep learning for predicting flux through reactions in a global metabolic network using transcriptomics data, which we ultimately apply to scRNAseq. GEFMAP leverages the natural graph structure of metabolic networks to learn both a biological objective for each cell and estimate a mass-balanced relative flux rate for each reaction in each cell using novel deep learning models.

Preprint

Multimodal biofeedback in the treatment of migraine.

The purpose of this study was twofold: (a) to compare the effects of three behavioral strategies for the relief of migraine, and (b) to examine different combinations of the treatments to assess the effectiveness of multimodal biofeedback with this problem. Twenty-four volunteer migraine sufferers not on medication, and with at least weekly occurrence of headaches, participated in the study. Results indicated that (a) subjects who learned temporal cooling, frontalis relaxation, and progressive muscular relaxation exhibited the best success with headache relief; (b) control subjects, who did not show the same psychophysiological changes as experimental subjects, reported no headache relief; and (c) subjects in the group with only relaxation exercises performed similarly to control subjects and reported no headache relief.

Biofeedback, Psychology

Disorders of attention and vigilance.

Attention deficit hyperactivity disorder has gone through multiple name and criterion changes over the last 10 years. It is still viewed as the most common neurobehavioral disorder in the pediatric age group. Diagnostic criteria, as outlined by the DSM III-R, have, as primary diagnostic features of ADHD, inattention, impulsivity, and hyperactivity. There are no diagnostic tests that will absolutely confirm this disorder. Many questionnaires have been devised to help organize and systematize the evaluation. Vision and hearing screening, physical examination, and the neurologic examination are important to exclude other causes for hyperactivity and impulsivity. There are no routine laboratory or specialized tests that are useful in the diagnosis of ADHD. When psychologic testing is deemed necessary, specific areas to study would include discrepancies in the performance and verbal IQ (possibly indicating further testing needed to rule out learning disabilities) and difficulties in the coding portion of the examination, which might point to attentional difficulties. Educational testing can further complete an evaluation when needed. Management of children with ADHD involves multimodal treatment, including educational, behavioral, and possibly pharmacologic intervention. Pharmacotherapy can be a useful adjunct in treating these children. The psycho-stimulants continue to be the first-line medication in treatment, although investigators have studied other drugs. Once the diagnosis has been made and a child has been placed on the multimodal treatment, he should be carefully followed. This disorder does not disappear at the time of puberty, as once was thought, but instead continues to cause problems through adolescence.

Attention Deficit Disorder with Hyperactivity

Unifying multimodal single-cell data with a mixture-of-experts β-variational autoencoder framework.

Multimodal single-cell assays profile complementary layers of cell state, but integration is complicated by modality mismatch, sparsity, and uneven cohort coverage. Here, we present Unified Variational Inference (UniVI), a scalable mixture-of-experts β-variational autoencoder that learns a shared latent space while preserving modality-specific structure. UniVI couples modality-specific encoders/decoders with a shared latent prior and a symmetric cross-modal alignment objective, enabling consistent integration of paired measurements without curated feature-link graphs or preannotated reference atlases; optional supervised heads can be added when labels are available. Across paired RNA-protein (CITE-seq) and RNA-chromatin (10x Genomics Multiome, SHARE-seq) data spanning human PBMCs and mouse back skin-a nonhematopoietic tissue with continuous differentiation hierarchies-UniVI produces coherent embeddings, improves label transfer, and enables cross-modal reconstruction and denoising. Extending to trimodal measurements, UniVI maintains robust three-way alignment among RNA, chromatin accessibility, and surface proteins (TEA-seq), and accommodates DNA methylation in a paired scNMT-seq mouse gastrulation proof-of-concept under beta-binomial likelihoods. Performance degrades gracefully under severe cell type imbalance and in the presence of modality-exclusive populations. In an acute myeloid leukemia mosaic design, a paired RNA-protein bridge anchors independent RNA-only and protein+genotype cohorts, revealing genotype-associated neighborhoods that sharpen with mutation-aware fine-tuning. UniVI thus provides a flexible, interpretable framework for multimodal integration across paired, trimodal, and mosaic study designs and supports practical reference-to-query projection in partially observed studies.

Journal Article

Recovery of a temporally based visual discrimination after visual cortex lesion in the rat.

In Expt. 1, rats were conditioned to emit a shock avoidance response when the pulse rate of a light was increased. Then, after bilateral visual cortex lesions, the rats were given 10, 20, or 40 days recovery before relearning the discrimination. While all rats were able to relearn the discrimination response, lesion rats had a performance deficit after all recovery periods. Expt. 2 compared the effects of postoperative visual pulse rate training to those of auditory pulse rate training on relearning of the photic pulse-rate discrimination 10 days after visual decortication. Recovery of discrimination responding was better after auditory pulse rate training than after visual training. These data suggest that visual cortex lesions in the rat disrupt perceptual or associational functions involving the temporal features of a visual stimulus. In addition, generalization of relational properties during cross-modal training through multimodal CNS structures appears to enhance recovery of behavior after brain insult.

Acoustic Stimulation

Visual and song nuclei correlate with courtship skills in brown-headed cowbirds.

We assessed courtship success in 14 adult male brown-headed cowbirds, Molothrus ater. Volumes of song control nuclei and visual nuclei were measured in Nissl stained tissue. Variation in courtship success was found to be related to variation in two areas of the avian brain: the song control nucleus, area X, and the thalamic visual area, nucleus rotundus. Volume of area X was negatively correlated with song potency, as assessed by female playback, and with rate of vocalizing. Volume of nucleus rotundus was positively correlated with song potency, vocalizing to females and courtship persistence. These data are the first to implicate a visual nucleus in the use of song. The data also complement previous findings with cowbirds suggesting that song learning involves visual attention to females. Taken as a whole, these results suggest that use of song depends on integration of auditory, vocal and visual information. In that female songbirds in many species assess multimodal performance of song, these findings with cowbirds suggest that future studies of brain and behaviour include a broader view of possible behavioural and neural correlates. Copyright 1998 The Association for the Study of Animal Behaviour.

Journal Article

Synaptic protein CSF levels relate to memory scores in individuals without dementia.

BACKGROUND: We investigated how cerebrospinal fluid levels of synaptic proteins associate with memory function in normal cognition (CN) and mild cognitive impairment (MCI), and investigated the effect of amyloid positivity on these associations. METHODS: We included 242 CN (105(43%) abnormal amyloid), and 278 MCI individuals (183(66%) abnormal amyloid) from the European Medical Information Framework for Alzheimer's Disease Multimodal Biomarker Discovery (EMIF-AD MBD) and the Alzheimer's Disease Neuroimaging Initiative (ADNI). For 181 (EMIF-AD MBD) and 36 (ADNI) proteins with a synaptic annotation in SynGO, associations with word learning recall were analysed with linear models. RESULTS: Subsets of synaptic proteins showed lower levels with worse recall in preclinical AD (EMIF-AD MBD: 7, ADNI: 5 proteins, none overlapping), prodromal AD (EMIF-AD MBD only, 27 proteins) and non-AD MCI (EMIF-AD MBD: 1, ADNI: 7 proteins). The majority of these associations were specific to these clinical groups. CONCLUSIONS: Synaptic disturbance-related memory impairment occurred very early in AD, indicating it may be relevant to develop therapies targeting the synapse early in the disease.

Humans

Mushroom bodies of the cockroach: their participation in place memory.

Insects and other arthropods use visual landmarks to remember the location of their nest, or its equivalent. However, so far, only olfactory learning and memory have been claimed to be mediated by any particular brain region, notably the mushroom bodies. Here we describe the results of experiments that demonstrate that the mushroom bodies of the cockroach (Periplaneta americana), already shown to be involved in multimodal sensory processing, play a crucial role in place memory. Behavioral tests, based on paradigms similar to those originally used to demonstrate place memory in rats, demonstrate a rapid improvement in the ability of individual cockroaches to locate a hidden target when its position is provided by distant visual cues. Bilateral lesions of selected areas of the mushroom bodies abolish this ability but leave unimpaired the ability to locate a visible target. The present results demonstrate that the integrity of the pedunculus and medial lobe of a single mushroom body is required for place memory. The results are comparable to the results obtained from hippocampal lesions in rats and are relevant to recent studies on the effects of ablations of Drosophila mushroom bodies on locomotion.

Animals