SEARCH · PubMed Health

Results for “multimodal integration”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32Linked to original sources

PubMed1996

Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading.

Mechanisms of human attention allow selective processing of just the relevant events among the many stimuli bombarding our senses. Most laboratory studies examine attention within just a single sense, but in the real world many important events are specified multimodally, as in verbal communication. Speech comprises visual lip movements as well as sounds, and lip-reading contributes to speech perception, even for listeners with good hearing, by a process of audiovisual integration. Such examples raise the problem of how we coordinate our spatial attention across the sensory modalities, to select sights and sounds from a common source for further processing. Here we show that this problem is alleviated by allowing some cross-modal matching before attentional selection is completed. Cross-modal matching can lead to an illusion, whereby sounds are mislocated at their apparent visual source; this crossmodal illusion can enhance selective spatial attention to speech sounds.

Attention↗

PubMed2006

Iomazenil-single-photon emission computed tomography reveals selective neuronal loss in magnetic resonance-defined mismatch areas.

BACKGROUND AND PURPOSE: The mismatch of hypoperfused tissue on perfusion imaging and ischemic tissue on diffusion-weighted imaging is used as a surrogate marker for thrombolytic therapy in the extended time window. Mismatch tissue may recover completely, progress toward infarction, or proceed toward incomplete infarction with selective loss of cortical neurons. We used [(123)I]iomazenil-single-photon emission computed tomography (IMZ-SPECT) to characterize the neuronal integrity of reperfused "tissue at risk of infarction" that appeared morphologically intact on follow-up magnetic resonance imaging (MRI). METHODS: Twelve patients with acute stroke with striatocapsular (SC) infarctions were examined with multimodal MRI at days 0, 1, and 7; IMZ-SPECT was performed at days 5 to 15. The PI at day 0, fluid-attenuated inversion recovery (FLAIR) image at day 7, and IMZ-SPECT were coregistered and stereotactically normalized. The mismatch volume of interest (VOI) was defined as the initial PI lesion subtracted by the FLAIR lesion at day 7. An asymmetry ratio (AR) was computed by dividing the mean IMZ uptake of the mismatch VOI by the unaffected mirror VOI. The same AR was computed for signal intensity on FLAIR images at day 7. Three patients with cortical infarctions were included for calibration of the AR. In this group, the VOI consisted of the FLAIR lesion at day 7. RESULTS: All patients with SC infarctions had a large mismatch of initially hypoperfused (112+/-31 mL; mean+/-SD) and finally infarcted tissue (19+/-14 mL). Mean AR of cortical IMZ uptake was 0.85+/-0.01 in cortical infarctions and 0.95+/-0.03 in SC infarctions; thereby AR showed a continuous distribution from clearly reduced (0.89) to normal (1.01) in SC infarctions. Mean AR for FLAIR signal intensity was 1.84+/-0.14 for cortical infarctions and normal (1.01+0.03) for SC infarctions. CONCLUSIONS: IMZ-SPECT detected a selective loss of cortical neurons in patients with SC infarctions in transient hypoperfused tissue, which was morphologically intact on MRI.

Adult↗

PubMed2005

[Individual therapists in inpatient psychotherapy -- a pilot study on their importance for therapeutic outcome].

OBJECTIVE: How much does the individual psychotherapist influence therapy outcome in a multimodal inpatient setting with several therapeutic relationships and the additional influence of group processes. METHODS: We studied the outcome of n = 293 psychotherapy inpatients (mean treatment duration: 11 weeks) who were treated in an integrated inpatient concept by 9 mostly psychodynamically oriented psychotherapists (professional experience 0.5 to 7 yrs, 13 to 56 patients per therapist). Patients had an extensive evaluation of process and outcome variables (e. g. SCL 90R, IIP, Global outcome rating, Helping Alliance Questionnaire - HAQ). Differences in outcome between patients of different therapists were calculated. RESULTS: Patients assigned to the participating therapists did not differ in their basic clinical characteristics, except in the clinical severity rating carried out by therapists themselves. Outcome in the SCL90R - GSI residual score and global rating differed significantly between the therapists (d (SCL-pooled SD) = 0.43 to 1.10, explained variance: 10 %). The same holds true for the therapeutic alliance (HAQ, explained variance: 45 %). Assessment of therapeutic relationship shows good correlation between patients and therapists, outcome assessment does not. Therapists differed in the degree of matching between their own and their patients' evaluations. CONCLUSIONS: We found significant therapist differences in outcome and alliance. Despite the multifaceted inpatient setting the individual therapist is an important determinant of therapeutic success.

Adult↗

PubMed1991

Multimodal therapy for stones in pelvic kidneys.

Stones located in pelvic kidneys can be successfully treated by extracorporeal lithotripsy, either alone or in combination with endourology and open surgery. A multimodal approach was used in 16 patients with pelvic kidney stone disease and a 87.5% stone-free rate was achieved. The proper positioning of the patient on the lithotripter apparatus and the correct integration of the different therapeutic procedures were the real clues for obtaining a remarkable success rate with a minimally invasive approach.

Adult↗

PubMed1989

Speech perception using combinations of auditory, visual, and tactile information.

Four normally-hearing subjects were trained and tested with all combinations of a highly-degraded auditory input, a visual input via lipreading, and a tactile input using a multichannel electrotactile speech processor. The speech perception of the subjects was assessed with closed sets of vowels, consonants, and multisyllabic words; with open sets of words and sentences, and with speech tracking. When the visual input was added to any combination of other inputs, a significant improvement occurred for every test. Similarly, the auditory input produced a significant improvement for all tests except closed-set vowel recognition. The tactile input produced scores that were significantly greater than chance in isolation, but combined less effectively with the other modalities. The addition of the tactile input did produce significant improvements for vowel recognition in the auditory-tactile condition, for consonant recognition in the auditory-tactile and visual-tactile conditions, and in open-set word recognition in the visual-tactile condition. Information transmission analysis of the features of vowels and consonants indicated that the information from auditory and visual inputs were integrated much more effectively than information from the tactile input. The less effective combination might be due to lack of training with the tactile input, or to more fundamental limitations in the processing of multimodal stimuli.

Adult↗

PubMed1984

The role of connexon aggregate fusion in gap junction growth.

Gap junctions are thought to grow in size by the incorporation of single connexons and by the fusion of connexon aggregates but the relative importance of these two growth mechanisms, particularly in the formation of large gap junctions, remains unclear. We have used a quantitative approach to distinguish between these two growth mechanisms and present evidence here that gap junctions in Drosophila melanogaster wing discs have a periodic, (approximately equal to 0.025 micron 2), multimodal areal distribution which suggests that gap junctions grow by aggregate fusion. Previously published gap junction areal frequency distributions from another invertebrate, Manduca sexta, and from several vertebrate animals also show periodic (approximately equal to 0.030 micron 2), multimodal areal distributions. We suggest that the interpeak spacings on the frequency distributions represent average areas of 'unit aggregates' of connexons and that the fusion of such unit aggregates with one another and with integral multiples of unit aggregates may be a general mechanism for gap junction growth. Unit aggregates in mammals and insects are similar in surface area rather than in connexon number suggesting that a common molecular mechanism, perhaps operating at the level of the plasma membrane, may be involved in their formation and stabilization.

Animals↗

PubMed2003

3-d source localization of epileptic foci integrating EEG and MRI data.

This study evaluates the utility of 3-D localization of interictal spike activity on the electroencephalographs (EEG) superimposed on magnetic resonance imagery (MRI) in a pediatric population with extra-temporal lesional epileptic foci. 3-D software programming based on the CURRY platform (a multimodal neuro-imaging software) was adapted for analyzing scalp EEG data and reconstructing superimposed images in 10 children who underwent extensive pre-surgical evaluation for intractable partial seizures. The results of 3-D spike source localization were assessed in relationship to focal lesions evident on the patient's MRI scans. Calculated spike sources were closest to the lesions during intervals corresponding to the spike peaks. The information was useful in surgical planning in six children that underwent successful resections.

Action Potentials↗

PubMed2006

Integrating treatment and education for mood disorders: an adolescent case report.

This case study illustrates one successful outcome of an intensive, outpatient, treatment project for adolescents with mood disorders. An 18-year-old female with symptoms across several DSM-IV Axis I classifications, including a depressive disorder, and her parents participated in a year-long, multimodal intervention that included mood-focused psychoeducation and coaching designed to impact on her, her family, school, and community systems. Self-report, clinician-driven, and ecologically valid measures were used to assess treatment effects on psychiatric symptoms and psychosocial functioning. Results on the Child and Adolescent Functional Assessment Scale demonstrated considerable gains in the following areas: Home, school/work, social behavior, self-harm, thinking/communication, and substance use. During the intervention, she went from failing several of her classes to graduating from high school. In addition, she made the Honours' List in her first semester at a local community college. A discussion of intervention pluses and pitfalls specific to the case highlight the necessity to influence the various spheres of the young person's life.

Achievement↗

PubMed2006

Multimodal visualization interface for data management, self-learning and data presentation.

A multimodal visualization software, called the Data Manager (DM), has been developed to increase interdisciplinary communication around the topic of visualization and modeling of various aspects of the human anatomy. Numerous tools used in Radiology are integrated in the interface that runs on standard personal computers. The available tools, combined to hierarchical data management and custom layouts, allow analyzing of medical imaging data using advanced features outside radiological premises (for example, for patient review, conference presentation or tutorial preparation). The system is free, and based on an open-source software development architecture, and therefore updates of the system for custom applications are possible.

Anatomy↗

PubMed1999

Computerized dynamic posturography and seasickness susceptibility.

OBJECTIVE/HYPOTHESIS: The neural mismatch theory emphasizes the role of conflicting multimodal sensory interactions in producing both motion sickness and the rearrangement process that finally leads to habituation to the adverse motion conditions. If this theory is, indeed, correct, the patterns of the response to the integrated signal from simultaneous multisensory stimulation, characterized by unusual relationships between the senses responsible for spatial orientation, should differ according to motion sickness susceptibility. Computerized dynamic posturography (CDP) provides the opportunity to simultaneously change the interactions between visual, somatosensory, and vestibular inputs, thus giving an indication of the relative importance of these senses in maintaining balance. The objective was to investigate balance strategies in naval crew members with differing susceptibility to sea conditions using CDP. STUDY DESIGN: Cross-sectional, parallel-group design. METHODS: Twenty subjects susceptible to seasickness (SS) and 20 nonsusceptible subjects (NSS), healthy male volunteers aged 18 to 25, were tested using the EquiTest system (NeuroCom, Inc., Clackamas, OR). RESULTS: The SS group exhibited significantly less stability than the NSS group in condition 5 of the sensory organization test (SOT). The ratio of the SOT scores of conditions 5 to 1 (the vestibular organization pattern) was also found to be significantly lower in the SS group. CONCLUSIONS: The results suggest that SS might be more dependent on somatosensory and visual inputs and less on vestibular inputs for maintenance of balance compared with NSS. Higher susceptibility to seasickness might reflect abnormal weighting of sensory modalities during the integration process. This would result in disruption of the integration process required to maintain balance and a sense of orientation in space in conditions producing conflicting sensory inputs.

Adolescent↗

PubMed2002

Neoadjuvant chemotherapy in non-small cell lung cancer.

Non-small cell lung cancer (NSCLC) is a systemic illness. More than half of those patients who present with stage I-IIIA disease and are resected will experience distant relapse. Postoperative adjuvant chemotherapy has been evaluated in several randomized trials but the results of these trials have been inconclusive with increased survival reported in few trials. In resectable stage IIIA NSCLC the findings of three randomized trials have indicated that the survival of these patients is better with neoadjuvant chemotherapy plus surgical resection than with resection alone. Phase II trials using preoperative concurrent chemoradiotherapy have been carried out with encouraging results. The majority of patients with stage IIIA NSCLC require multimodality therapy if they are to achieve a 5-year survival. Combined modality treatment in locally advanced NSCLC continues to evolve and is a subject of ongoing research. One focus for present research is to integrate new active agents into the neoadjuvant setting. Another challenge is to find better treatment approaches in earlier stages of disease. Some data suggest that induction chemotherapy in stage I-II is feasible, does not appear to compromise surgery and yields high response rates. A further aim is to use molecular biological markers of malignancy to identify patients at highest risk of metastatic relapse.

Carcinoma, Non-Small-Cell Lung↗

PubMed1994

Expanding the use and effectiveness of dose-volume histograms for 3-D treatment planning. I: Integration of 3-D dose-display.

PURPOSE: A technique is presented for overcoming a major deficiency of histogram analysis in three-dimensional (3-D) radiotherapy treatment planning; the lack of spatial information. METHODS AND MATERIALS: In this technique, histogram data and anatomic images are displayed in a side-by-side fashion. The histogram curve is used as a guide to interactively probe the nature of the corresponding 3-D dose distribution. Regions of dose that contribute to a specific dose bin or range of bins are interactively highlighted on the anatomic display as a window-style cursor is positioned along the dose-axis of the histogram display. This dose range highlighting can be applied to two-dimensional (2-D) images and to 3-D views which contain anatomic surfaces, multimodality image data, and representations of radiation beams and beam modifiers. Additionally, as a range of histogram bins is specified, dose and volume statistics for the range are continually updated and displayed. RESULTS: The implementation of these techniques is presented and their use illustrated for a nonaxial three field treatment of a hepatic tumor. CONCLUSION: By integrating displays of 3-D doses and the corresponding histogram data, it is possible to recover the positional information inherently lost in the calculation of a histogram. Important questions such as the size and location of hot spots in normal tissues and cold spots within target volumes can be more easily uncovered, making the iterative improvement of treatment plans more efficient.

Humans↗

PubMed1987

Perceiving similarity and comprehending metaphor.

We conducted a series of 3 experiments to assess the comprehension of 4 types of cross-modal (synesthetic) similarities in nearly 500 3.5-13.5-year-old children and more than 100 adults. We tested both perceptual and verbal (metaphoric) modes. Children of all ages and adults matched pitch to brightness and loudness to brightness, thereby showing that even very young children recognize perceptual similarities between hearing and vision. Children did not consistently recognize similarity between pitch and size until about age 11. This difference in developmental timetables is compatible with the view that pitch-brightness and loudness-brightness similarities are intrinsic characteristics of perception (characteristics based, perhaps, on common sensory codes), whereas pitch-size similarity may be learned (perhaps through association of size with resonance properties). In a parallel verbal task, even 4-year-old children showed at least some capacity to translate meanings metaphorically from one modality to another (e.g., rating "low pitched" as dim and "high pitched" as bright). But not all literal meanings produced metaphoric equivalents in the youngest children (e.g., rating "sunlight" brighter but not louder than "moonlight"). Improvements with age in making metaphoric translations of synesthetic expressions paralleled increasing differentiation of meanings along literal dimensions and increasing capacity to integrate meanings of components in compound expressions. We postulate that perceptual knowledge about objects and events is represented in terms of locations in a multidimensional space; cross-modal similarities imply that the space is also multimodal. Verbal processes later gain access to this graded perceptual knowledge, thus permitting the interpretation of synesthetic metaphors according to the rules of cross-modal perception.

Acoustic Stimulation↗

PubMed2001

Quantum trajectory analysis of multimode subsystem-bath dynamics.

The dynamics of a swarm of quantum trajectories is investigated for systems involving the interaction of an active mode (the subsystem) with an M-mode harmonic reservoir (the bath). Equations of motion for the position, velocity, and action function for elements of the probability fluid are integrated in the Lagrangian (moving with the fluid) picture of quantum hydrodynamics. These fluid elements are coupled through the Bohm quantum potential and as a result evolve as a correlated ensemble. Wave function synthesis along the trajectories permits an exact description of the quantum dynamics for the evolving probability fluid. The approach is fully quantum mechanical and does not involve classical or semiclassical approximations. Computational results are presented for three systems involving the interaction on an active mode with M=1, 10, and 15 bath modes. These results include configuration space trajectory evolution, flux analysis of the evolving ensemble, wave function synthesis along trajectories, and energy partitioning along specific trajectories. These results demonstrate the feasibility of using a small number of quantum trajectories to obtain accurate quantum results on some types of open quantum systems that are not amenable to standard quantum approaches involving basis set expansions or Eulerian space-fixed grids.

Journal Article↗

PubMed2001

Dynamic and multimodal responses of gustatory cortical neurons in awake rats.

To investigate the dynamic aspects of gustatory activity, we recorded the responses of small ensembles of cortical neurons to tastants administered to awake rats. Multiple trials of each tastant were delivered during recordings made in oral somatosensory (SI) and gustatory cortex (GC). When integrated tastant responses (firing rates averaged across 2.5 sec) were compared with water responses, 14.4% (13/90) of the GC neurons responded in a taste-specific manner. When time was considered as a source of information, however, the incidence of taste-specific firing increased: as many as 41% (37/90) of the recorded GC neurons exhibited taste-specific patterns of response. For 17% of the neurons identified as responding with taste-specific patterns, the stimulus that caused the most significant response was a function of the time since stimulus delivery. That is, a single neuron might respond most strongly to one tastant in the first 500 msec of a response and then respond most strongly to another tastant later in the response. Further analysis of the time courses of GC and SI cortical neural responses revealed that modulations of GC firing rate arose from three separable processes: early somatosensory input (less than approximately 0.2 sec post-stimulus), later chemosensory input ( approximately 0.2-1 sec), and delayed somatosensory input related to orofacial responses (more than approximately 1.0 sec). These data demonstrate that sensory information is available in the time course of GC responses and suggest the viability of views of gustatory processing that treat the temporal structure of cortical responses as an integral part of the neural code.

Action Potentials↗

PubMed2024

Multiome Perturb-seq unlocks scalable discovery of integrated perturbation effects on the transcriptome and epigenome.

Single-cell CRISPR screens link genetic perturbations to transcriptional states, but high-throughput methods connecting these induced changes to their regulatory foundations are limited. Here, we introduce Multiome Perturb-seq, extending single-cell CRISPR screens to simultaneously measure perturbation-induced changes in gene expression and chromatin accessibility. We apply Multiome Perturb-seq in a CRISPRi screen of 13 chromatin remodelers in human RPE-1 cells, achieving efficient assignment of sgRNA identities to single nuclei via an improved method for capturing barcode transcripts from nuclear RNA. We organize expression and accessibility measurements into coherent programs describing the integrated effects of perturbations on cell state, finding that ARID1A and SUZ12 knockdowns induce programs enriched for developmental features. Modeling of perturbation-induced heterogeneity connects accessibility changes to changes in gene expression, highlighting the value of multimodal profiling. Overall, our method provides a scalable and simply implemented system to dissect the regulatory logic underpinning cell state. A record of this paper's transparent peer review process is included in the supplemental information.

Humans↗

PubMed1984

Mapping the body representation in the SI cortex of anesthetized and awake rats.

We have used single unit recording techniques to map the representation of cutaneous and joint somatosensory modalities in the primary somatosensory (SI) cortex of both anesthetized and awake rats. The cytoarchitectonic zones within the rat SI were divided into the following main categories: (1) granular zones (GZs)--areas exhibiting koniocortical cytoarchitecture (i.e., containing dense aggregates of layer IV granule cells), (2) perigranular zones (PGZs)--narrow strips of less granular cortex surrounding the GZs, and (3) dysgranular zones (DZs)--large areas of dysgranular cortex enclosed within the SI. The narrow strip between the SI and the rostrally adjacent frontal agranular cortex was termed the "transitional zone" (TZ). Initial computer-based studies of the properties of cutaneous receptive fields (RFs) in SI showed that, although there were differences in response threshold, adaptability, frequency response, and overall RF size and shape of adjacent neurons, the size and location of the "centers" of the RFs were quite constant and were similar to those seen in multiple unit recordings. The same was true of RFs of single neurons recorded through different anesthetic states. The body representation in SI was first mapped by determining single unit and unit cluster RFs within a total of 2,170 microelectrode penetrations in barbiturate-anesthetized rats. Cutaneous RFs in the GZs were quite discrete. Thus, a single, finely detailed, continuous map of the cutaneous periphery was definable within the GZs themselves. Only the forepaw had a double representation. RFs in the PGZs were larger and more diffuse, but since they covered roughly the same skin areas as the RFs in the most closely adjacent GZs, they fit into the same body map. Neurons in the DZs were unresponsive to any sensory stimuli in the anesthetized animal. In chronically implanted, freely moving, awake animals cutaneous RFs were larger and more volatile than in the anesthetized, but the accuracy of the map was clearly preserved by the fact that the locations of the RF centers (which often must be defined quantitatively) were unchanged. The PGZs and DZs in the awake animals exhibited a multimodal convergence of cutaneous and joint movement RFs within single vertical penetrations, or even on single neurons. Directionally specific and bilateral cutaneous RFs were also observed in the DZs. It was concluded the DZs are more associational or integrative areas within the SI, but they could not be shown to comprise a distinct and separate body representation.(ABSTRACT TRUNCATED AT 400 WORDS)

Anesthesia, General↗

PubMed2026

Deep generative models in biological sequence and structure analysis and design.

Deep generative models have transformed biological sequence modeling from predictive analysis toward increasingly controllable design. Early biological applications of Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) established latent representation learning and sequence synthesis, while recent advances in transformer-based language models, discrete diffusion, flow-matching, and multimodal generative frameworks have substantially expanded the scope of biological design. This review examines generative models for DNA, RNA, and protein sequence design, emphasizing how different model classes represent biological constraints, operate over discrete and continuous spaces, and integrate sequence, structure, and function. We compare VAEs, GANs, autoregressive and masked language models, diffusion models, and flow-based approaches across genomics, transcriptomics, and proteomics, with particular attention to controllability, long-range dependency modeling, structural grounding, generalization, and experimental utility. We further examine evaluation strategies, out-of-distribution generalization, and closed-loop design-build-test-learn workflows that connect in silico generation with empirical validation. We distinguish fundamental modality-dependent constraints including sequence discreteness, context length, structural coupling, and physical or thermodynamic requirements from architecture-dependent advantages that reflect the current state of the field. Current studies suggest that long-context models are particularly useful for genome-scale representation and sequence modeling, whereas structure-aware diffusion, flow-based, and inverse-folding approaches provide better frameworks for geometry-constrained RNA and protein design. This perspective provides a critical framework for understanding the present capabilities, limitations, and convergence of generative approaches toward reliable and experimentally grounded biological design.

Biological sequence analysis↗