SEARCH · PubMed Health

Results for “Multimodal learning”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9Linked to original sources

PubMedDate not supplied

A study of multimodal motherese: the role of temporal synchrony between verbal labels and gestures.

This study examined European American and Hispanic American mothers' multimodal communication to their infants (N = 24). The infants were from three age groups representing three levels of lexical-mapping development: prelexical (5 to 8 months), early-lexical (9 to 17 months), and advanced-lexical (21 to 30 months). Mothers taught their infants four target (novel) words by using distinct objects during a semistructured play episode. Recent research suggests that young infants rely on temporal synchrony to learn syllable-object relations, but later, the role of synchrony diminishes. Thus, mothers' target and nontarget naming were coded for synchrony and other communication styles. The results indicated that mothers used target words more often than nontarget words in synchrony with object motion and sometimes touch. Thus, "multimodal motherese" likely highlights target word-referent relations for infants. Further, mothers tailored their communication to infants' level of lexical-mapping development. Mothers of prelexical infants used target words in synchrony with object motion more often than mothers of early- and advanced-lexical infants. Mothers' decreasing use of synchrony across age parallels infants' decreasing reliance on synchrony, suggesting a dynamical and reciprocal environment-organismic relation.

Adult↗

PubMed1992

Academic underachievement, attention deficits, and aggression: comorbidity and implications for intervention.

Although comorbidity with specific learning disabilities is less frequent than commonly reported, externalizing behavior disorders--particularly attention-deficit hyperactivity disorder (ADHD)--often overlap with various indices of academic underachievement during childhood. Furthermore, by adolescence, delinquency is clearly associated with school failure. Because the link between behavioral and learning problems often appears before formal schooling, and because the co-morbid problems predict a negative course, early intervention is a necessity. Controlled treatment investigations with youngsters who show these combined problems are rare, and such studies present a host of methodologic and practical problems. I discuss issues surrounding multimodality treatment programs and the potential for long-term interventions to break cycles of school failure and externalizing behavior.

Adolescent↗

PubMed1999

Episodic retrieval activates the precuneus irrespective of the imagery content of word pair associates. A PET study.

The aim of this study was to evaluate further the role of the precuneus in episodic memory retrieval. The specific hypothesis addressed was that the precuneus is involved in episodic memory retrieval irrespective of the imagery content. Two groups of six right-handed normal male volunteers took part in the study. Each subject underwent six [15O]butanol-PET scans. In each of the six trials, the memory task began with the injection of a bolus of 1500 MBq of [15O]butanol. For Group 1, 12 word pair associates were presented visually, for Group 2 auditorily. The subjects of each group had to learn and retrieve two sets of 12 word pairs each. One set consisted of highly imaginable words and another one of abstract words. Words of both sets were not related semantically, representing 'hard' associations. The presentations of nonsense words served as reference conditions. We demonstrate that the precuneus shows consistent activation during episodic memory retrieval. Precuneus activation occurred in visual and auditory presentation modalities and for both highly imaginable and abstract words. The present study therefore provides further evidence that the precuneus has a specific function in episodic memory retrieval as a multimodal association area.

Acoustic Stimulation↗

PubMed2006

Mother-infant bonding and the evolution of mammalian social relationships.

A wide variety of maternal, social and sexual bonding strategies have been described across mammalian species, including humans. Many of the neural and hormonal mechanisms that underpin the formation and maintenance of these bonds demonstrate a considerable degree of evolutionary conservation across a representative range of these species. However, there is also a considerable degree of diversity in both the way these mechanisms are activated and in the behavioural responses that result. In the majority of small-brained mammals (including rodents), the formation of a maternal or partner preference bond requires individual recognition by olfactory cues, activation of neural mechanisms concerned with social reward by these cues and gender-specific hormonal priming for behavioural output. With the evolutionary increase of neocortex seen in monkeys and apes, there has been a corresponding increase in the complexity of social relationships and bonding strategies together with a significant redundancy in hormonal priming for motivated behaviour. Olfactory recognition and olfactory inputs to areas of the brain concerned with social reward are downregulated and recognition is based on integration of multimodal sensory cues requiring an expanded neocortex, particularly the association cortex. This emancipation from olfactory and hormonal determinants of bonding has been succeeded by the increased importance of social learning that is necessitated by living in a complex social world and, especially in humans, a world that is dominated by cultural inheritance.

Animals↗

PubMed2004

Walk and Talk: an intervention for behaviorally challenged youths.

This qualitative research explored the question: Do preadolescent and adolescent youths with behavioral challenges benefit from a multimodal intervention of walking outdoors while engaging in counseling? The objective of the Walk and Talk intervention is to help the youth feel better, explore alternative behavioral choices, and learn new coping strategies and life skills by engaging in a counseling process that includes the benefits of mild aerobic exercise, and that nurtures a connection to the outdoors. The intervention utilizes a strong therapeutic alliance based on the Rogerian technique of unconditional positive regard, which is grounded and guided by the principles of attachment theory. For eight weeks, eight students (aged 9 to 13 years) from a middle school in Alberta, Canada, participated weekly in the Walk and Talk intervention. Students' self-reports indicated that they benefited from the intervention. Research triangulation with involved adults supported findings that indicated the students were making prosocial choices in behavior, and were experiencing more feelings of self-efficacy and well-being. Limitations, new research directions, and subsequent longitudinal research possibilities are discussed.

Adaptation, Psychological↗

PubMed1997

Robotic multimodality stereotactic brain tissue identification: work in progress.

Real-time identification of tissue would improve procedures such as stereotactic brain biopsy (SBX), functional and implantation neurosurgery, and brain tumor excision. To standard SBX equipment has been added: (1) computer-controlled stepper motors to drive the biopsy needle/probe precisely; (2) multiple microprobes to track tissue density, detect blood vessels and changes in blood flow, and distinguish the various tissues being penetrated; (3) neural net learning programs to allow real-time comparisons of current data with a normative data bank; (4) three-dimensional graphic displays to follow the probe as it traverses brain tissue. The probe can differentiate substances such as pig brain, differing consistencies of the 'brain-like' foodstuff tofu, and gels made to simulate brain, as well as detect blood vessels imbedded in these substances. Multimodality probes should improve the safety, efficacy, and diagnostic accuracy of SBX and other neurosurgical procedures.

Animals↗

PubMed2004

Four types of novelty-familiarity responses in associative recognition memory of humans.

Animal studies show that, like inferior temporal neurons, dorsolateral prefrontal and parietal neurons often respond more strongly to individual novel than to individual familiar stimuli. It is currently unclear whether the novelty preference of prefrontal and parietal neurons extends to associative memory. We used electromagnetic recordings (MEG/EEG) and functional magnetic resonance imaging in two groups of healthy young adults to identify neural populations outside the inferior temporal cortex that exhibit associative novelty (stronger responses for new than for old configurations of two familiar items), and to distinguish them from associative familiarity (stronger responses for old than for new configurations of two familiar items). Subjects were required to learn and were later tested for associations based on the spatial configurations of two stimuli (a face and a tool). At test, learned (old) and rearranged (new) spatial stimulus configurations had to be discriminated. This recognition memory test could only be solved through the associative relationship between individual items because all component items of the stimulus configurations were equally familiar. In both imaging modalities, right dorsolateral prefrontal cortex and right parietal cortex showed an associative novelty response, whereas the right superior temporal cortex showed an associative familiarity response. With EEG/MEG only, the right extrastriate cortex showed an early associative familiarity and a late associative novelty response, whereas the opposite pattern emerged in bilateral frontopolar cortex. Thus, through a multimodal approach, it was possible to identify four types of associative novelty/familiarity responses outside the inferior temporal cortex.

Adult↗

PubMed1985

Do infants see emotional expressions in static faces?

To determine whether young infants discriminate photographs of different emotions on an affect-relevant basis or on the basis of isolated features unrelated to emotion, groups of 17-, 23-, and 29-week-olds were habituated to slides of 8 women posing either Toothy Angry, Nontoothy Angry, or Nontoothy Smiling facial expressions and were then shown 2 new women in the familiarized expression and in a novel Toothy Smiling expression. At all 3 ages, recovery to the novel Toothy Smiling faces occurred only after habituation to Nontoothy faces (whether smiling or angry), not after habituation to Toothy Angry faces, indicating that infants had been responsive to nonspecific features of the photographs (presence or absence of bared teeth) rather than to affectively relevant configurations of features. In a second experiment, 2 older age groups (35 and 41 weeks) also proved to be insensitive to affect-related aspects of still faces, though more so for angry than for happy expressions. It is suggested that the young infant's difficulty in extracting emotional information from static stimuli may be attributable to the absence of the critical invariants (dynamic, multimodally specified) that characterize naturalistic expressions of emotion.

Anger↗

PubMed2023

Inferring Metabolic States from Single Cell Transcriptomic Data via Geometric Deep Learning.

The ability to measure gene expression at single-cell resolution has elevated our understanding of how biological features emerge from complex and interdependent networks at molecular, cellular, and tissue scales. As technologies have evolved that complement scRNAseq measurements with things like single-cell proteomic, epigenomic, and genomic information, it becomes increasingly apparent how much biology exists as a product of multimodal regulation. Biological processes such as transcription, translation, and post-translational or epigenetic modification impose both energetic and specific molecular demands on a cell and are therefore implicitly constrained by the metabolic state of the cell. While metabolomics is crucial for defining a holistic model of any biological process, the chemical heterogeneity of the metabolome makes it particularly difficult to measure, and technologies capable of doing this at single-cell resolution are far behind other multiomics modalities. To address these challenges, we present GEFMAP (Gene Expression-based Flux Mapping and Metabolic Pathway Prediction), a method based on geometric deep learning for predicting flux through reactions in a global metabolic network using transcriptomics data, which we ultimately apply to scRNAseq. GEFMAP leverages the natural graph structure of metabolic networks to learn both a biological objective for each cell and estimate a mass-balanced relative flux rate for each reaction in each cell using novel deep learning models.

Preprint↗

PubMed1983

Multimodal biofeedback in the treatment of migraine.

The purpose of this study was twofold: (a) to compare the effects of three behavioral strategies for the relief of migraine, and (b) to examine different combinations of the treatments to assess the effectiveness of multimodal biofeedback with this problem. Twenty-four volunteer migraine sufferers not on medication, and with at least weekly occurrence of headaches, participated in the study. Results indicated that (a) subjects who learned temporal cooling, frontalis relaxation, and progressive muscular relaxation exhibited the best success with headache relief; (b) control subjects, who did not show the same psychophysiological changes as experimental subjects, reported no headache relief; and (c) subjects in the group with only relaxation exercises performed similarly to control subjects and reported no headache relief.

Biofeedback, Psychology↗

PubMed1988

Disorders of attention and vigilance.

Attention deficit hyperactivity disorder has gone through multiple name and criterion changes over the last 10 years. It is still viewed as the most common neurobehavioral disorder in the pediatric age group. Diagnostic criteria, as outlined by the DSM III-R, have, as primary diagnostic features of ADHD, inattention, impulsivity, and hyperactivity. There are no diagnostic tests that will absolutely confirm this disorder. Many questionnaires have been devised to help organize and systematize the evaluation. Vision and hearing screening, physical examination, and the neurologic examination are important to exclude other causes for hyperactivity and impulsivity. There are no routine laboratory or specialized tests that are useful in the diagnosis of ADHD. When psychologic testing is deemed necessary, specific areas to study would include discrepancies in the performance and verbal IQ (possibly indicating further testing needed to rule out learning disabilities) and difficulties in the coding portion of the examination, which might point to attentional difficulties. Educational testing can further complete an evaluation when needed. Management of children with ADHD involves multimodal treatment, including educational, behavioral, and possibly pharmacologic intervention. Pharmacotherapy can be a useful adjunct in treating these children. The psycho-stimulants continue to be the first-line medication in treatment, although investigators have studied other drugs. Once the diagnosis has been made and a child has been placed on the multimodal treatment, he should be carefully followed. This disorder does not disappear at the time of puberty, as once was thought, but instead continues to cause problems through adolescence.

Attention Deficit Disorder with Hyperactivity↗

PubMed2026

Unifying multimodal single-cell data with a mixture-of-experts β-variational autoencoder framework.

Multimodal single-cell assays profile complementary layers of cell state, but integration is complicated by modality mismatch, sparsity, and uneven cohort coverage. Here, we present Unified Variational Inference (UniVI), a scalable mixture-of-experts β-variational autoencoder that learns a shared latent space while preserving modality-specific structure. UniVI couples modality-specific encoders/decoders with a shared latent prior and a symmetric cross-modal alignment objective, enabling consistent integration of paired measurements without curated feature-link graphs or preannotated reference atlases; optional supervised heads can be added when labels are available. Across paired RNA-protein (CITE-seq) and RNA-chromatin (10x Genomics Multiome, SHARE-seq) data spanning human PBMCs and mouse back skin-a nonhematopoietic tissue with continuous differentiation hierarchies-UniVI produces coherent embeddings, improves label transfer, and enables cross-modal reconstruction and denoising. Extending to trimodal measurements, UniVI maintains robust three-way alignment among RNA, chromatin accessibility, and surface proteins (TEA-seq), and accommodates DNA methylation in a paired scNMT-seq mouse gastrulation proof-of-concept under beta-binomial likelihoods. Performance degrades gracefully under severe cell type imbalance and in the presence of modality-exclusive populations. In an acute myeloid leukemia mosaic design, a paired RNA-protein bridge anchors independent RNA-only and protein+genotype cohorts, revealing genotype-associated neighborhoods that sharpen with mutation-aware fine-tuning. UniVI thus provides a flexible, interpretable framework for multimodal integration across paired, trimodal, and mosaic study designs and supports practical reference-to-query projection in partially observed studies.

Journal Article↗

PubMed1993

Recovery of a temporally based visual discrimination after visual cortex lesion in the rat.

In Expt. 1, rats were conditioned to emit a shock avoidance response when the pulse rate of a light was increased. Then, after bilateral visual cortex lesions, the rats were given 10, 20, or 40 days recovery before relearning the discrimination. While all rats were able to relearn the discrimination response, lesion rats had a performance deficit after all recovery periods. Expt. 2 compared the effects of postoperative visual pulse rate training to those of auditory pulse rate training on relearning of the photic pulse-rate discrimination 10 days after visual decortication. Recovery of discrimination responding was better after auditory pulse rate training than after visual training. These data suggest that visual cortex lesions in the rat disrupt perceptual or associational functions involving the temporal features of a visual stimulus. In addition, generalization of relational properties during cross-modal training through multimodal CNS structures appears to enhance recovery of behavior after brain insult.

Acoustic Stimulation↗

PubMed1998

Visual and song nuclei correlate with courtship skills in brown-headed cowbirds.

We assessed courtship success in 14 adult male brown-headed cowbirds, Molothrus ater. Volumes of song control nuclei and visual nuclei were measured in Nissl stained tissue. Variation in courtship success was found to be related to variation in two areas of the avian brain: the song control nucleus, area X, and the thalamic visual area, nucleus rotundus. Volume of area X was negatively correlated with song potency, as assessed by female playback, and with rate of vocalizing. Volume of nucleus rotundus was positively correlated with song potency, vocalizing to females and courtship persistence. These data are the first to implicate a visual nucleus in the use of song. The data also complement previous findings with cowbirds suggesting that song learning involves visual attention to females. Taken as a whole, these results suggest that use of song depends on integration of auditory, vocal and visual information. In that female songbirds in many species assess multimodal performance of song, these findings with cowbirds suggest that future studies of brain and behaviour include a broader view of possible behavioural and neural correlates. Copyright 1998 The Association for the Study of Animal Behaviour.

Journal Article↗

PubMed2025

Synaptic protein CSF levels relate to memory scores in individuals without dementia.

BACKGROUND: We investigated how cerebrospinal fluid levels of synaptic proteins associate with memory function in normal cognition (CN) and mild cognitive impairment (MCI), and investigated the effect of amyloid positivity on these associations. METHODS: We included 242 CN (105(43%) abnormal amyloid), and 278 MCI individuals (183(66%) abnormal amyloid) from the European Medical Information Framework for Alzheimer's Disease Multimodal Biomarker Discovery (EMIF-AD MBD) and the Alzheimer's Disease Neuroimaging Initiative (ADNI). For 181 (EMIF-AD MBD) and 36 (ADNI) proteins with a synaptic annotation in SynGO, associations with word learning recall were analysed with linear models. RESULTS: Subsets of synaptic proteins showed lower levels with worse recall in preclinical AD (EMIF-AD MBD: 7, ADNI: 5 proteins, none overlapping), prodromal AD (EMIF-AD MBD only, 27 proteins) and non-AD MCI (EMIF-AD MBD: 1, ADNI: 7 proteins). The majority of these associations were specific to these clinical groups. CONCLUSIONS: Synaptic disturbance-related memory impairment occurred very early in AD, indicating it may be relevant to develop therapies targeting the synapse early in the disease.

Humans↗

PubMed1998

Mushroom bodies of the cockroach: their participation in place memory.

Insects and other arthropods use visual landmarks to remember the location of their nest, or its equivalent. However, so far, only olfactory learning and memory have been claimed to be mediated by any particular brain region, notably the mushroom bodies. Here we describe the results of experiments that demonstrate that the mushroom bodies of the cockroach (Periplaneta americana), already shown to be involved in multimodal sensory processing, play a crucial role in place memory. Behavioral tests, based on paradigms similar to those originally used to demonstrate place memory in rats, demonstrate a rapid improvement in the ability of individual cockroaches to locate a hidden target when its position is provided by distant visual cues. Bilateral lesions of selected areas of the mushroom bodies abolish this ability but leave unimpaired the ability to locate a visible target. The present results demonstrate that the integrity of the pedunculus and medial lobe of a single mushroom body is required for place memory. The results are comparable to the results obtained from hippocampal lesions in rats and are relevant to recent studies on the effects of ablations of Drosophila mushroom bodies on locomotion.

Animals↗

PubMed2000

The role of movement and object in action memory: a comparative study between blind, blindfolded and sighted subjects.

Two experiments systematically compared four SPT conditions involving real/imaginary movement and real/imaginary object with one VT condition involving no enactment and no object. To test the effect of visual information on SPT memory, sighted subjects were compared with blindfolded subjects (in Experiment 1) and blind subjects (in Experiment 2). All subjects learned all SPTs and VTs. Free recall data showed no difference between the SPT conditions and between the groups of subjects; only blind subjects were found to be limited in the use of visualization strategy. All SPTs were recalled better than VTs, indicating that the enactment effect is not determined by either movement or object alone, rather both have an effective role and are equally involved for obtaining the enactment effect. The results provide no support for the motor encoding and multimodality views of SPTs, but are in line with the episodic integration view which assumes that neither movement nor object are of special importance, rather both have contribution in the enactment effect.

Adult↗

PubMed1994

Effects of learning modalities on melodic and rhythmic retention and on vocal pitch-matching by preschool children.

To assess whether melodic and rhythmic retention as well as pitch-matching ability could be improved through use of learning modalities, 61 children ages 4 and 5 years were presented music instruction in one of four ways, visually (seeing visual aids with the music), auditorily (singing and listening), kinesthetically (moving to music), or through multimodal presentations. Analysis indicated that preschool children receiving the auditory and multimodal treatments scored significantly higher on both the melodic and rhythmic posttests than on pretests. Children receiving kinesthetic treatment scored significantly lower on both the melodic and rhythmic posttests than the other three classes. Further, children in the auditory and multimodal classes matched pitch significantly better at posttest than children in either the visual or kinesthetic classes.

Attention↗