SEARCH · PubMed Health

Results for “Multimodal Fusion”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3Linked to original sources

PubMed1999

[The multimodal integration, correlation and fusion of morphology and function: the methods and initial clinical applications].

Besides the pure visual diagnosis of morphology in comparison with function, digital multimodal image analysis is steadily gaining in importance. Digital image processing is used in integration, correlation and fusion of topographically identical image data. After defining these terms and describing the acquisition techniques and influence parameters, this article reviews the methods of multimodal image processing, with emphasis on correlation methods. It also gives a short description of special methods in nuclear medicine. The clinical part briefly reviews the clinical use as well as the progress achieved and the benefits expected for diagnostic applications.

Diagnostic Imaging↗

PubMed1993

Data fusion in medical imaging: merging multimodal and multipatient images, identification of structures and 3D display aspects.

Data fusion in medical imaging can be seen into two ways (i) multisensors fusion of anatomical and functional information and (ii) interpatient data fusion by means of warping models. These two aspects set the methodological framework necessary to perform anatomical modelling especially when concerning the modelling of brain structures. The major relevance of the work presented here concerns the interpretation of multimodal 3D neuro-anatomical data bases. Three types of data fusion problems are considered in this paper. The first one concerns the problem of data combination which includes multimodal registration (multisensor fusion applied to CT, MRI, DSA, PET, SPECT, or MEG). In particular, the problem of warping patient data to an anatomical atlas is reviewed and a solution is proposed. The second problem of data fusion addressed in this paper is the identification of anatomical structures by means of image analysis methods. Two techniques have been developed. The first one deals with the analysis of image geometrical features to end up with the determination of a fuzzy mask to label the structure of interest. The second technique consists of labelling major cerebral structures by means of statistical image features associated with relaxation techniques. Finally, the paper presents a review of up to date 3D display techniques with a special emphasis on volume rendering and 3D display of combined data.

Algorithms↗

PubMed1999

MRI-SPECT fusion for the synthesis of high resolution 3D functional brain images: a preliminary study.

Medical imaging being a fast-expanding field, multimodal data fusion appears more and more as a key element for the optimal use of images. By fusion, we mean the combination of several information sources (in particular images), with the aim of providing either more condensed or more pertinent information. The long term scope of this work would be to improve the interpretation of 3D brain images, providing extra elements for the diagnosis and patient follow up. This preliminary study is part of a wider context: the medical follow up of patients suffering from probable Alzheimer disease observed in single photon emission tomography by fusion after registration with magnetic resonance images. Several information combination techniques based on the possibility theory are presented. A new operator, more specifically adapted to the fusion of anatomical and functional images, as well as a high resolution functional image synthesis technique are proposed. A first comparative study of fusion techniques is then proposed. Although no thorough test protocol has been defined, these preliminary results are encouraging, giving access to a wide field of potential clinical applications.

Aged↗

PubMed2005

Is susceptibility to perceptual migration and fusion modality-specific or multimodal?

A previous paper reported high susceptibility to spatial migration (allochiria) of tactile stimuli in about 25% of healthy individuals (High Error subjects). When synchronous stimuli touched the two hands, if the unattended stimulus was temporally modulated when the attended one was not (and was thus more salient than the latter), it "migrated" to and fused with or replaced the stimulus on the attended hand. When subjects rated similarity of the attended stimulus accompanied by a distractor to each stimulus alone, scaling distributions tested against a sampling model showed most High Error subjects experienced fused stimuli, others experienced replacement and Low Error subjects experienced neither. We argued that these migrations are equivalent to allochiria and that this underlies neglect and extinction. This study assessed whether the individual difference is modality-specific or not. In auditory and visual equivalents of the tactile rating experiment, the difference between High and Low Error subjects was replicated in audition, but no migration occurred in vision. However, when two words were briefly presented visually before a mask with cued report of one, letter migrations to equivalent locations did occur and the individual difference was reproduced. This constitutes the first report of individual differences in auditory fusion and visual letter migration. Migration occurred in egocentric coordinates but apparently preserved structural homology. Different migration rates between the modalities paralleled relative salience of the unattended to the attended stimulus. The multimodality of the individual difference suggests that its source is supramodal, in deficient binding of perceptual content to location.

Adult↗

PubMed2026

Multimodal computational framework resolves B cell maturation in autoimmunity and ageing.

Identification of the origin of pathogenic immune cells is crucial for therapeutic interventions and diagnosis but pseudotime methods struggle to trace immune cells accurately. Current trajectory inference methods for B cell development and response in health and disease either ignore or underutilize antigen receptor sequence information, limiting their ability to resolve developmental pathways, particularly for pathogenic populations. Widely used methods such as Monocle 3 reconstruct developmental paths from transcriptomic similarity alone, discarding the features from immune receptors. Dandelion has combined the immune receptor features with transcriptomics but it struggles to simulate the trajectory path of B cells. Here we present ClonoTrace, a computational framework that integrates BCR sequence features with transcriptomic trajectory inference through gated fusion of multimodal embeddings. In fetal B cell development and germinal centre development, ClonoTrace demonstrates closer concordance with the canonical reference ordering than Monocle 3 and Dandelion. Applied to systemic lupus erythematosus, ClonoTrace indicates a memory B cell extrafollicular maturation route alongside the naïve B cell route, accompanied by induction of ZEB2 with a concomitant decline of BACH2 along the trajectory, as a candidate alternative route to pathogenic double negative 2 B cells (DN2) in systemic lupus erythematosus (SLE) patients. In healthy ageing, ClonoTrace resolved three candidate age-related B cell maturation routes, from naïve, IgM+ memory and switched-memory B cells, each passing through a DN2-associated transcriptional state that is ordered before age-associated B cells along the inferred trajectory. ClonoTrace's fate probability algorithm indicated that IgM+ memory B cell to ABC transition as the leading candidate age-associated transition, which may be distinct from SLE DN2 maturation. ClonoTrace provides a generalizable framework for receptor-informed trajectory inference, describing candidate developmental routes of pathogenic B cell populations in autoimmunity and ageing.

Humans↗

PubMed2006

Advanced approach for intraoperative MRI guidance and potential benefit for neurosurgical applications.

PURPOSE: To present an advanced approach for intraoperative image guidance in an open 0.5 T MRI and to evaluate its effectiveness for neurosurgical interventions by comparison with a dynamic scan-guided localization technique. MATERIALS AND METHODS: The built-in scan guidance mode relied on successive interactive MRI scans. The additional advanced mode provided real-time navigation based on reformatted high-quality, intraoperatively acquired MR reference data, allowed multimodal image fusion, and used the successive scans of the built-in mode for quick verification of the position only. Analysis involved tumor resections and biopsies in either scan guidance (N = 36) or advanced mode (N = 59) by the same three neurosurgeons. Technical, surgical, and workflow aspects were compared. RESULTS: The image quality and hand-eye coordination of the advanced approach were improved. While the average extent of resection, neurologic outcome after functional MRI (fMRI) integration, and diagnostic yield appeared to be slightly better under advanced guidance, particularly for the main surgeon, statistical analysis revealed no significant differences. Resection times were comparable, while biopsies took around 30 minutes longer. CONCLUSION: The presented approach is safe and provides more detailed images and higher navigation speed at the expense of actuality. The surgical outcome achieved with advanced guidance is (at least) as good as that obtained with dynamic scan guidance.

Adult↗

PubMed2006

Image-guided brachytherapy for prostate cancer.

Prostate brachytherapy offers and effective treatment for organ-confined prostate carcinoma. It is rapidly delivered compared with external beam radiation therapy or surgery and well tolerated by patients. Volumetric imaging and image guidance play critical roles in patient selection, treatment planning, treatment delivery, and postimplant assessment. Costs, availability and ease of use often dictate the local and regional differences in imaging approach, whether ultrasound, CT, or MR. Future volumetric image developments may permit multimodality image fusion to integrate tumor-specific imaging such as MR spectroscopy or positron emission tomography/CT into real-time ultrasound, CT, or MR.

Brachytherapy↗

PubMed1998

A knowledge-based approach to 3-D reconstruction of human cerebral vasculature.

Good visualisation of the vasculature is essential in the diagnosis and treatment of a variety of brain disorders. This paper presents a new approach to 3-D reconstruction of vascular structures using knowledge-based image processing and multimodal image fusion. The task is to reconstruct the human cerebro-vascular system from the partial information collected from a variety of medical imaging instruments and then recombine these limited models into an anatomically accurate one.

Algorithms↗

PubMed2006

A simplified method to integrate metabolic images in stereotactic procedures using a PET/CT scanner.

We have developed a method that needs only the computed tomography (CT) indicator box to coregister positron emission tomography (PET) images and integrates this information with magnetic resonance imaging. The study was performed using a PET/CT scanner. A standard CT bed adapter was attached to the scanner couch. Then, the patient, with the Leksell G frame fixed, was positioned into the scanner with the CT indicator box. PET images were acquired using either [18F]fluorodeoxyglucose or [11C]choline as radioisotopes. After acquisition, CT and PET images were exported in DICOM 3 standard and transferred to a dedicated workstation via data link. Homemade software was implemented for multimodal image fusion. PET images were overwritten to their corresponding CT point values using a threshold algorithm, maintaining the stereotactic CT markers. The use of a CT indicator simplifies the procedure, because there is no need for a radioactive solution filling the indicator box. This method was tested first using a phantom and then in patients. The localization accuracy of the PET images is limited only by the slice thickness.

Aged↗

PubMed1994

Imaging techniques and computers.

Imaging technology has been the pacesetter of stereotactic technique since its introduction in clinical practice. Quite recently, the extraordinary quality of diagnostic neuroimages and the growing availability of computing power has widened the field of applications of stereotaxy suggesting, at the same time, substantial evolution of its instrumentation. Fusion of multimodal digital images, integration of morphological and functional data, and dimensional rendering techniques have become powerful 'navigational aids' for tissue sampling, functional procedures and image-guided endoscopic surgery. Radiosurgical dose planning systems integrated with image processing and computer graphics routines allow efficient interactive evaluation of tissue-dose volumes superimposition in brachytherapy and external beam focused irradiation. Volumetric guided open surgery, assisted by intraoperative acquisition of stereotactic echographic and microscopic images, allows satisfactory treatment of deep-seated cerebral lesions. Further improvement is to be expected with the implementation of the ongoing development of localizing solutions totally integrated with the microsurgical instrumentation.

Brain↗

PubMed1994

A bioimaging integration system implemented for neurological applications.

A system aimed at the management and fusion of multimodal biomedical images, including X-ray computed tomography, magnetic resonance imaging, positron emission tomography, and single photon emission computed tomography, has been implemented for neurological applications. This bioimaging integration system (BIS) consists of a network for image transmission from acquisition machines to dedicated image processing workstations, a software library for image standardization, and an image registration technique to project multimodal volumetric images into a common reference space. The registration procedure was evaluated in MRI/PET correlation studies, in which misalignment errors of 2.6 mm in the xy transaxial plane and 3.4 mm along the z axis were found. BIS has been validated for the anatomical-functional correlation analysis of MRI and PET images in neurological research protocols and clinical studies.

Brain Diseases↗

PubMed1997

Advances in brain imaging: a new ethical challenge.

Technical advances in the past 25 years permitted substantial advances in the neuroimaging field, expanding the diagnostic and research potentials and significantly reducing the use of old invasive imaging techniques for research purposes. The safer procedures now available allow acquisition of reference data, morphological assessment and functional characterisation from healthy volunteers. However, enrollment of volunteers is still a sensitive ethical issue. Ethical problems related to informed consent, for both research and diagnostic procedures, in patients with neuropsychiatric disorders represent an additional crucial issue. Furthermore, with both functional and structural neuroimaging studies, there is a theoretical risk of violation of individual privacy. Research in the neuroimaging field should tend to increase the amount of information obtained through appropriate post-processing procedures, including multimodality image fusion, and to limit stress and discomfort.

Brain↗

PubMed2004

A hierarchical parametric algorithm for deformable multimodal image registration.

Image fusion is of utmost importance for many applications in image analysis. Particularly in medical imaging, images of different modalities are necessary because they provide complementary information that must be merged for an optimal use. The fusion of these images, which can be achieved through a registration process, makes it possible to superimpose all available information on the same frame. In many cases, a rigid transformation is sufficient to align correctly the images. However, there are cases where a non-rigid transformation is needed: geometrical distortions present in one image, non-rigid motion, etc. The purpose of this paper is to propose a generic method to account for these deformations in case of multimodal images. We have applied the algorithm in the particular context of 3D medical images and present results on simulated and real data.

Algorithms↗

PubMed1998

A system for medical consultation and education using multimodal human/machine communication.

Recent developments in networking and computing have enabled collaborative biomedical engineering research by geographically separated participants. One of the most promising goals is to use these technologies to extend human intellectual capabilities in medical decision making. These emerging technologies are poised to drastically reduce healthcare cost by providing service at remote locations. This also increases diagnosis capacity since information is made available to experts at any location. In this paper, we propose a novel application of a recently developed interactive and distributed system in medical consultation and education. Our approach builds on the notion that interactive and distributive capabilities of the system are crucial for medical consultation and education. The presented application uses a multiuser, collaborative environment with multimodal human/machine communication in the dimensions of sight, sound, and touch. The experimental setup, consisting of two user stations, and the multimodal interfaces, including sight (eye-tracking), sound (automatic speech), and touch (microbeam pen), were tested and evaluated. The system uses a collaborative workspace as a common visualization space. Users communicate with the application through a fusion agent by eye-tracking, speech, and microbeam pen. The audio/video teleconferencing is also included to help the radiologists to communicate with each other simultaneously while they are working on the mammograms. The system used in this study has three software agents: a fusion agent, a conversational agent, and an analytic agent. The fusion agent interprets multimodal commands by integrating the multimodal inputs. The conversational agent answers the user's questions and detects human-related or semantic errors and notifies the user about the results of the image analysis. The analytic agent enhances the digitized images using the wavelet denoising algorithm if requested by the user. To show how well the system performs in practice, we used the system for medical consultation on mammograms. Results also show that the relevant information about the region of interest (ROI) of the mammograms chosen by the users is extracted automatically and used to enhance the mammograms.

Education, Distance↗

PubMed2001

Multimodality image registration quality assurance for conformal three-dimensional treatment planning.

PURPOSE: We present a quality assurance methodology to determine the accuracy of multimodality image registration and fusion for the purpose of conformal three-dimensional and intensity-modulated radiation therapy treatment planning. Registration and fusion accuracy between any combination of computed tomography (CT), magnetic resonance (MR), and positron emission computed tomography (PET) imaging studies can be evaluated. METHODS AND MATERIALS: A commercial anthropomorphic head phantom filled with water and containing CT, MR, and PET visible targets was modified to evaluate the accuracy of multimodality image registration and fusion software. For MR and PET imaging, the water inside the phantom was doped with CuNO(3) and 18F-fluorodeoxyglucose (18F-FDG), respectively. Targets consisting of plastic spheres and pins were distributed throughout the cranium section of the phantom. Each target sphere had a conical-shaped bore with its apex at the center of the sphere. The pins had a conical extension or indentation at the free end. The contours of the spheres, sphere centers, and pin tips were used as anatomic landmark models for image registration, which was performed using affine coordinate-transformation tools provided in a commercial multimodality image registration/fusion software package. Four sets of phantom image studies were obtained: primary CT, secondary CT with different phantom immobilization, MR, and PET study. A novel CT, MR, and PET external fiducial marking system was also tested. RESULTS: The registration of CT/CT, CT/MR, and CT/PET images allowed correlation of anatomic landmarks to within 2 mm, verifying the accuracy of the registration software and spatial fidelity of the four multimodality image sets. CONCLUSIONS: This straightforward phantom-based quality assurance of the image registration and fusion process can be used in a routine clinical setting or for providing a working image set for development of the image registration and fusion process and new software.

Head and Neck Neoplasms↗

PubMed1999

Implementation of three-dimensional EEG brain mapping.

The electroencephalogram (EEG) visualization software was developed containing two-dimensional (2D) and three-dimensional (3D) brain mapping modules. The input to the program is standard clinical individual patient data recorded using digital EEG and magnetic resonance imaging (MRI). The software utilizes several techniques, such as heuristic triangulation, ray casting, Gouraud shading, and image fusion to form multimodal 3D images. The program has been applied to the 3D visualization of various EEG signals, "cortical" EEG signals, and potential fields generated by a computer model. The developed program appears to operate efficiently and intuitively in PC/Windows environment.

Algorithms↗

PubMed2026

Integrating Radiogenomics and CSF-Based Liquid Biopsy Sequencing for Precision Neuro-Oncology.

Glioblastoma and diffuse gliomas pose major therapeutic challenges due to marked intratumoral heterogeneity, limited tissue accessibility, and the blood-brain barrier. Tissue-based next-generation sequencing (NGS) remains essential for WHO CNS5 molecular classification, yet it is invasive and poorly suited to serial monitoring. Two complementary non- or minimally invasive approaches have advanced rapidly: radiogenomics, which correlates multiparametric MRI features with genomic alterations, and cerebrospinal fluid (CSF) liquid biopsy sequencing, which detects circulating tumor DNA with high tissue concordance. This review examines the independent progress and synergistic integration of radiogenomics and CSF-NGS. Imaging signatures can non-invasively predict key drivers (IDH1/2, EGFR, TERT, PTEN, TP53) and molecular subtypes, while CSF-ctDNA sequencing enables real-time assessment of clonal evolution, therapy resistance (including post-temozolomide hypermutation), and residual disease. We discuss technical considerations, performance metrics, multimodal artificial-intelligence fusion, and emerging clinical applications for diagnosis, prognosis, treatment selection, and longitudinal surveillance. Critical challenges, standardization, prospective validation, and workflow integration are highlighted. By combining the spatial phenotypic information of radiogenomics with the temporal genomic resolution of CSF sequencing, this multimodal strategy offers a promising path toward precision neuro-oncology and reduced reliance on repeated invasive sampling.

Humans↗

PubMed2005

Phase I vaccination trial of SYT-SSX junction peptide in patients with disseminated synovial sarcoma.

BACKGROUND: Synovial sarcoma is a high-grade malignant tumor of soft tissue, characterized by the specific chromosomal translocation t(X;18), and its resultant SYT-SSX fusion gene. Despite intensive multimodality therapy, the majority of metastatic or relapsed diseases still remain incurable, thus suggesting a need for new therapeutic options. We previously demonstrated the antigenicity of SYT-SSX gene-derived peptides by in vitro analyses. The present study was designed to evaluate in vivo immunological property of a SYT-SSX junction peptide in selected patients with synovial sarcoma. METHODS: A 9-mer peptide (SYT-SSX B: GYDQIMPKK) spanning the SYT-SSX fusion region was synthesized. Eligible patients were those (i) who have histologically and genetically confirmed, unresectable synovial sarcoma (SYT-SSX1 or SYT-SSX2 positive), (ii) HLA-A*2402 positive, (iii) between 20 and 70 years old, (iv) ECOG performance status between 0 and 3, and (v) who gave informed consent. Vaccinations with SYT-SSX B peptide (0.1 mg or 1.0 mg) were given subcutaneously six times at 14-day intervals. These patients were evaluated for DTH skin test, adverse events, tumor size, tetramer staining, and peptide-specific CTL induction. RESULTS: A total of 16 vaccinations were carried out in six patients. The results were (i) no serious adverse effects or DTH reactions, (ii) suppression of tumor progression in one patient, (iii) increases in the frequency of peptide-specific CTLs in three patients and a decrease in one patient, and (iv) successful induction of peptide-specific CTLs from four patients. CONCLUSIONS: Our findings indicate the safety of the SYT-SSX junction peptide in the use of vaccination and also give support to the property of the peptide to evoke in vivo immunological responses. Modification of both the peptide itself and the related protocol is required to further improve the therapeutic efficacy.

Journal Article↗