SEARCH · PubMed Health

Results for “Multimodal learning”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6Linked to original sources

PubMed1997

Future directions in lung cancer research and therapeutics.

Although immune and genetic approaches do not yet have substantial clinical exposure in NSCLC, it is hoped that the future will bring less toxic, more specific, and more effective methods to prevent the development of end-stage NSCLC. The principles of multitargeted and multimodality therapy to overcome tumor heterogeneity and resistance, and drug delivery issues have been developed over years of effort using chemotherapy. The lessons learned should not be forgotten, because they will remain applicable to newer biologic and genetic modalities.

Antineoplastic Agents↗

PubMed2026

Beyond antibiotics: artificial intelligence-enabled anti-infective ecosystems for next-generation precision therapeutics against antimicrobial resistance.

The rapid global expansion of antimicrobial resistance (AMR) threatens to undermine decades of progress in infectious disease management and highlights the limitations of conventional antibiotic-centered therapeutic strategies. Although emerging technologies-including antimicrobial peptides, bacteriophage therapy, CRISPR-based antimicrobials, microbiome therapeutics, anti-virulence approaches, nanotechnology-enabled drug delivery, and artificial intelligence (AI)-have individually demonstrated considerable promise, they are predominantly being developed as independent interventions rather than as coordinated components of an integrated therapeutic strategy. This Perspective proposes the Intelligent Anti-Infective Ecosystem (IAIE) as a conceptual systems-level framework that computationally integrates multimodal diagnostics, pathogen genomics, microbiome profiling, AI-assisted decision support, programmable precision therapeutics, ecological monitoring, and longitudinal clinical feedback within a continuously learning dynamically optimized workflow. Unlike existing paradigms that primarily optimize individual technologies or therapeutic decisions, IAIE emphasizes closed-loop coordination among complementary antimicrobial approaches to support precision-guided infection management while preserving microbiome integrity and mitigating resistance selection pressure. We further outline the core components, operational principles, translational challenges, and technology readiness of the major therapeutic platforms that could contribute to such an ecosystem, while distinguishing clinically established interventions from emerging experimental strategies. Importantly, IAIE should be interpreted as a prospective conceptual architecture rather than an existing clinical platform. Its proposed clinical value remains to be established through sequential computational, preclinical, and prospective clinical investigations using standardized microbiological, ecological, and patient-centered outcome measures. By framing antimicrobial innovation within an responsive systems perspective, IAIE provides a roadmap for future multidisciplinary research aimed at integrating artificial intelligence and systems microbiology to enable sustainable management of antimicrobial resistance.

Humans↗

PubMed1992

Evaluation of coma and vegetative states.

The Coma/Near-Coma (CNC) scale was designed to measure small clinical changes in patients with severe traumatic and nontraumatic brain injuries who were functioning at very low levels characteristic of near-vegetative and vegetative states. In 20 patients followed for 16 weeks the scale identified 25% who ultimately showed modest improvement. Interrater reliability was high (r = .95); validity was supported by significant correlations between CNC- and brain-multimodality evoked potential abnormality scores as well as between scores on the CNC and the Disability Rating Scale. The CNC scale was easily learned and it could be completed quickly and cost effectively. Staff found it useful in recognizing among relatively homogeneous low-level patients those most likely to respond to further rehabilitation care. The CNC appears to be useful for justifying ongoing intensive rehabilitation and for preventing premature transfer to lower levels of care.

Adolescent↗

PubMed2026

Biological Foundation Models for Complex Disease Research and Clinical Translation.

Complex diseases, including cancer, rare genetic disorders, neurodevelopmental and psychiatric conditions, and neurodegenerative diseases, arise from interactions among genetic variation, gene regulation, and cellular states that are difficult to capture using a single data type or biological scale. Biological foundation models address this challenge by treating nucleotides and genes as tokens and learning representations that can be transferred to downstream biomedical and clinical tasks. In this review, we examine two major model classes, genomic sequence foundation models and cell foundation models, and compare their tokenization strategies, model architectures, pretraining objectives, and adaptation methods. We summarize their emerging applications in regulatory variant interpretation, disease-associated cell-state analysis, drug-response prediction, and therapeutic target discovery across complex diseases. We distinguish applications supported by experimental or retrospective validation from those that remain primarily computational or conceptual. We further discuss key challenges to clinical translation, including multimodal data integration, model interpretability, benchmarking, patient-specific prediction, and privacy protection. We highlight future opportunities to integrate biological foundation models with emerging frameworks of medical digital twins, agentic AI, and federated learning. By linking model design to translational goals, this review provides a practical framework for evaluating biological foundation models and their readiness for complex disease research and clinical use.

biological foundation model↗

PubMed2026

AI-Supported, Integrative Prediction of Postoperative Delirium: Protocol for the CONFUSED Study.

BACKGROUND: Postoperative delirium (POD) is a frequent and serious complication in older surgical patients, characterized by acute cognitive dysfunction and fluctuating levels of consciousness. POD is associated with prolonged hospitalization, long-term cognitive decline, reduced quality of life, and increased mortality. Despite its clinical relevance, the underlying pathophysiological mechanisms remain poorly understood, and reliable biomarkers for early prediction and prevention are lacking. OBJECTIVE: The CONFUSED study aims to identify molecular and clinical predictors of POD by integrating clinical data with proteomic, transcriptomic, and epigenetic analyses. The primary objective is to develop predictive models for POD using multimodal data. Secondary objectives include the identification of delirium-associated genes, proteins, and epigenetic signatures, as well as the exploration of patient subgroups at increased risk for POD. METHODS: CONFUSED is a prospective observational cohort study conducted at a German university hospital. Adult patients undergoing major surgery under general anesthesia will be enrolled until 100 cases of POD have been observed, which is expected to require a total sample size of approximately 200 to 300 patients. Blood samples are collected at 4 predefined time points: before premedication, immediately after surgery, and on postoperative days 2 and 5. Samples undergo comprehensive proteomic profiling, transcriptomic analysis using RNA microarrays, DNA methylation analysis, and genotyping of selected polymorphisms. Clinical data, including demographics, comorbidities, perioperative variables, medications, and delirium assessments using the Confusion Assessment Method (CAM) and CAM for the intensive care unit, are systematically recorded. Statistical analyses include univariate and multivariate methods, as well as machine learning approaches such as random forests and support vector machines, to identify relevant biomarkers and develop predictive models. The study protocol follows STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) and TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) guidelines and was approved by the responsible ethics committees. RESULTS: The study was registered in the German Clinical Trials Register (DRKS00033854) on March 18, 2024. Recruitment started in January 2024 and is ongoing at the time of manuscript submission. As of now, 135 patients have been enrolled. Sample collection and laboratory analyses are ongoing. Data analysis began in January 2026, with first results anticipated in July 2026. Final data lock is anticipated after the completion of recruitment. CONCLUSIONS: By integrating multimodal molecular data with clinical parameters and applying advanced machine learning techniques, the CONFUSED study aims to improve the prediction and understanding of POD. The results are expected to support the development of personalized preventive strategies and contribute to improved perioperative care for patients at risk of POD.

Humans↗

PubMed1995

Amygdala role in conditioned associative learning.

Amygdala role in emotion was reviewed in reference to recent amygdala lesion studies and neuronal responses in the rat amygdala to conditioned stimuli. Extensive lesion studies suggest that the amygdala is crucial in various kinds of motivated and emotional behavior, and related autonomic responses. These amygdala functions critically depend on learning and memory. Amygdala lesions, both before and after training of conditioned associative learning, impaired emotional expression without simple sensory-motor deficits. Pharmacological experiments indicated neurotransmission in the amygdala is mediated through NMDA and AMPA receptors. These results strongly suggest the amygdala involvement in acquiring and storing associative memory (i.e. stimulus-affect association), by which animals recognize and evaluate the biological significance of a stimulus. This information is then transferred to the brainstem executing system. In the neurophysiological experiments, there were topographic distributions of sensory-responsive neurons within the amygdala, which were well correlated to anatomical data. The responses of rat amygdala neurons changed plastically during learning. Furthermore, more sensory-responsive neurons were encountered in the amygdala of rats trained to associate the sensory stimuli with a reinforcement than in the amygdala of rats that were not trained. In trained rats, multimodal neurons that responded to conditioned and unconditioned stimuli were frequently found in the basolateral and central nuclei of the amygdala. The results suggest that basolateral and central nuclei are foci where various sensory modalities converge, and which might perform critical functions in acquiring and storing long-term associative memory to link between sensory information and affective significance.

Amygdala↗

PubMed2003

Dreaming and episodic memory: a functional dissociation?

The activity that takes place in memory systems during sleep is likely to be related to the role of sleep in memory consolidation and learning, as well as to the generation of dream hallucinations. This study addressed the often-stated hypothesis that replay of whole episodic memories contributes to the multimodal hallucinations of sleep. Over a period of 14 days, 29 subjects kept a log of daytime activities, events, and concerns, wrote down any recalled dreams, and scored the dreams for incorporation of any waking experiences. While 65% of a total of 299 sleep mentation reports were judged to reflect aspects of recent waking life experiences, the episodic replay of waking events was found in no more than 1-2% of the dream reports. This finding has implications for understanding the unique memory processing that takes place during the night and is consistent with evidence that sleep has no role in episodic memory consolidation.

Consciousness↗

PubMed1991

The psychology of polio as prelude to post-polio sequelae: behavior modification and psychotherapy.

Even as the physical causes and treatments for post-polio sequelae (PPS) are being identified, psychological symptoms--chronic stress, anxiety, depression, and compulsive, Type A behavior--are becoming evident in polio survivors. Importantly, these symptoms are not only causing marked distress but are preventing patients from making the lifestyle changes necessary to treat their PPS. Neither clinicians nor polio survivors have paid sufficient attention to the acute polio experience, its conditioning of life-long patterns of behavior, its relationship to the development of PPS, and its effect on the ability of individuals to cope with and treat their new symptoms. We describe the acute polio and post-polio experiences on the basis of patient histories, relate the experience of polio to the development of compulsive, Type A behavior, link these behaviors to the physical and psychological symptoms reported in the National Post-Polio Surveys, and present a multimodal behavioral approach to treatment.

Activities of Daily Living↗

PubMed1992

Behavioural and neurohistological changes in aging Sepia.

The life cycle of the cuttlefish in the English Channel is characterized by a succession of homogeneous population cohorts. These conditions provide an excellent opportunity for the study of aging in this cephalopod. In a first longitudinal study, we considered the oldest animals and compared their success rates at the first capture attempt. During the first weeks of the study, the results remained constant and then, during the weeks immediately preceding the natural death, a dramatic drop was observed. This deterioration may be due to defects of visuomotor coordination. In a second study, we used an associative learning protocol with negative reinforcement and the performances of young and old animals were compared. The most striking results showed that the performances of the oldest animals during the retention test were very mediocre. Such results suggest that the long-term memory process is affected. Finally, a modification of the Fink-Heimer silver stain enabled us to draw a map of spontaneous terminal degeneration in the central nervous system of the oldest animals. The structures which are characterized by the presence of multimodal inputs (the spine of the peduncle lobe and the basal lobes) present the most obvious signs of degeneration.

Aging↗

PubMed2000

Traces of learning in the auditory localization pathway.

One of the fascinating properties of the central nervous system is its ability to learn: the ability to alter its functional properties adaptively as a consequence of the interactions of an animal with the environment. The auditory localization pathway provides an opportunity to observe such adaptive changes and to study the cellular mechanisms that underlie them. The midbrain localization pathway creates a multimodal map of space that represents the nervous system's associations of auditory cues with locations in visual space. Various manipulations of auditory or visual experience, especially during early life, that change the relationship between auditory cues and locations in space lead to adaptive changes in auditory localization behavior and to corresponding changes in the functional and anatomical properties of this pathway. Traces of this early learning persist into adulthood, enabling adults to reacquire patterns of connectivity that were learned initially during the juvenile period.

Animals↗

PubMed2000

Nonlinear autoassociation is not equivalent to PCA.

A common misperception within the neural network community is that even with nonlinearities in their hidden layer, autoassociators trained with backpropagation are equivalent to linear methods such as principal component analysis (PCA). Our purpose is to demonstrate that nonlinear autoassociators actually behave differently from linear methods and that they can outperform these methods when used for latent extraction, projection, and classification. While linear autoassociators emulate PCA, and thus exhibit a flat or unimodal reconstruction error surface, autoassociators with nonlinearities in their hidden layer learn domains by building error reconstruction surfaces that, depending on the task, contain multiple local valleys. This interpolation bias allows nonlinear autoassociators to represent appropriate classifications of nonlinear multimodal domains, in contrast to linear autoassociators, which are inappropriate for such tasks. In fact, autoassociators with hidden unit nonlinearities can be shown to perform nonlinear classification and nonlinear recognition.

Linear Models↗

PubMed1998

Concepts of a Web-based open distributed textbook for the multimodal diagnostics of gastrointestinal tumours with MRI, CT and video-endoscopy addressing students of medicine and students of medical informatics as two different target groups.

Multimodal diagnostics of gastrointestinal tumours with MRI, CT and video-endoscopy is a rapidly changing domain. The education at our universities should overcome the obstacles of traditional learning based on paper media and oral lectures with retention rates of 10-30% only. The paper presents the objectives and the results of the design phase of the project ODITEB1-Open Distributed TExt Book, for Computer-Assisted Instruction in the domain mentioned above. The main objective is to produce an electronic interactive textbook in order to shift education to more efficient learning settings with higher retention rates. The main concepts are 1) three-layer architecture (dynamic case layer, intermediate query layer, static instruction layer) 2) case pool distribution 3) active learners experience (interactive exploration of original image data).

Computer-Assisted Instruction↗

PubMedDate not supplied

A study of multimodal motherese: the role of temporal synchrony between verbal labels and gestures.

This study examined European American and Hispanic American mothers' multimodal communication to their infants (N = 24). The infants were from three age groups representing three levels of lexical-mapping development: prelexical (5 to 8 months), early-lexical (9 to 17 months), and advanced-lexical (21 to 30 months). Mothers taught their infants four target (novel) words by using distinct objects during a semistructured play episode. Recent research suggests that young infants rely on temporal synchrony to learn syllable-object relations, but later, the role of synchrony diminishes. Thus, mothers' target and nontarget naming were coded for synchrony and other communication styles. The results indicated that mothers used target words more often than nontarget words in synchrony with object motion and sometimes touch. Thus, "multimodal motherese" likely highlights target word-referent relations for infants. Further, mothers tailored their communication to infants' level of lexical-mapping development. Mothers of prelexical infants used target words in synchrony with object motion more often than mothers of early- and advanced-lexical infants. Mothers' decreasing use of synchrony across age parallels infants' decreasing reliance on synchrony, suggesting a dynamical and reciprocal environment-organismic relation.

Adult↗

PubMed1992

Academic underachievement, attention deficits, and aggression: comorbidity and implications for intervention.

Although comorbidity with specific learning disabilities is less frequent than commonly reported, externalizing behavior disorders--particularly attention-deficit hyperactivity disorder (ADHD)--often overlap with various indices of academic underachievement during childhood. Furthermore, by adolescence, delinquency is clearly associated with school failure. Because the link between behavioral and learning problems often appears before formal schooling, and because the co-morbid problems predict a negative course, early intervention is a necessity. Controlled treatment investigations with youngsters who show these combined problems are rare, and such studies present a host of methodologic and practical problems. I discuss issues surrounding multimodality treatment programs and the potential for long-term interventions to break cycles of school failure and externalizing behavior.

Adolescent↗

PubMed1999

Episodic retrieval activates the precuneus irrespective of the imagery content of word pair associates. A PET study.

The aim of this study was to evaluate further the role of the precuneus in episodic memory retrieval. The specific hypothesis addressed was that the precuneus is involved in episodic memory retrieval irrespective of the imagery content. Two groups of six right-handed normal male volunteers took part in the study. Each subject underwent six [15O]butanol-PET scans. In each of the six trials, the memory task began with the injection of a bolus of 1500 MBq of [15O]butanol. For Group 1, 12 word pair associates were presented visually, for Group 2 auditorily. The subjects of each group had to learn and retrieve two sets of 12 word pairs each. One set consisted of highly imaginable words and another one of abstract words. Words of both sets were not related semantically, representing 'hard' associations. The presentations of nonsense words served as reference conditions. We demonstrate that the precuneus shows consistent activation during episodic memory retrieval. Precuneus activation occurred in visual and auditory presentation modalities and for both highly imaginable and abstract words. The present study therefore provides further evidence that the precuneus has a specific function in episodic memory retrieval as a multimodal association area.

Acoustic Stimulation↗

PubMed1997

Robotic multimodality stereotactic brain tissue identification: work in progress.

Real-time identification of tissue would improve procedures such as stereotactic brain biopsy (SBX), functional and implantation neurosurgery, and brain tumor excision. To standard SBX equipment has been added: (1) computer-controlled stepper motors to drive the biopsy needle/probe precisely; (2) multiple microprobes to track tissue density, detect blood vessels and changes in blood flow, and distinguish the various tissues being penetrated; (3) neural net learning programs to allow real-time comparisons of current data with a normative data bank; (4) three-dimensional graphic displays to follow the probe as it traverses brain tissue. The probe can differentiate substances such as pig brain, differing consistencies of the 'brain-like' foodstuff tofu, and gels made to simulate brain, as well as detect blood vessels imbedded in these substances. Multimodality probes should improve the safety, efficacy, and diagnostic accuracy of SBX and other neurosurgical procedures.

Animals↗

PubMed1985

Do infants see emotional expressions in static faces?

To determine whether young infants discriminate photographs of different emotions on an affect-relevant basis or on the basis of isolated features unrelated to emotion, groups of 17-, 23-, and 29-week-olds were habituated to slides of 8 women posing either Toothy Angry, Nontoothy Angry, or Nontoothy Smiling facial expressions and were then shown 2 new women in the familiarized expression and in a novel Toothy Smiling expression. At all 3 ages, recovery to the novel Toothy Smiling faces occurred only after habituation to Nontoothy faces (whether smiling or angry), not after habituation to Toothy Angry faces, indicating that infants had been responsive to nonspecific features of the photographs (presence or absence of bared teeth) rather than to affectively relevant configurations of features. In a second experiment, 2 older age groups (35 and 41 weeks) also proved to be insensitive to affect-related aspects of still faces, though more so for angry than for happy expressions. It is suggested that the young infant's difficulty in extracting emotional information from static stimuli may be attributable to the absence of the critical invariants (dynamic, multimodally specified) that characterize naturalistic expressions of emotion.

Anger↗

PubMed2023

Inferring Metabolic States from Single Cell Transcriptomic Data via Geometric Deep Learning.

The ability to measure gene expression at single-cell resolution has elevated our understanding of how biological features emerge from complex and interdependent networks at molecular, cellular, and tissue scales. As technologies have evolved that complement scRNAseq measurements with things like single-cell proteomic, epigenomic, and genomic information, it becomes increasingly apparent how much biology exists as a product of multimodal regulation. Biological processes such as transcription, translation, and post-translational or epigenetic modification impose both energetic and specific molecular demands on a cell and are therefore implicitly constrained by the metabolic state of the cell. While metabolomics is crucial for defining a holistic model of any biological process, the chemical heterogeneity of the metabolome makes it particularly difficult to measure, and technologies capable of doing this at single-cell resolution are far behind other multiomics modalities. To address these challenges, we present GEFMAP (Gene Expression-based Flux Mapping and Metabolic Pathway Prediction), a method based on geometric deep learning for predicting flux through reactions in a global metabolic network using transcriptomics data, which we ultimately apply to scRNAseq. GEFMAP leverages the natural graph structure of metabolic networks to learn both a biological objective for each cell and estimate a mass-balanced relative flux rate for each reaction in each cell using novel deep learning models.

Preprint↗