SEARCH · PubMed Health

Results for “Multimodal Fusion”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5Linked to original sources

PubMed1983

Periodic, multimodal distribution of granule volumes in mast cells.

The areas of 2327 mast cell granules in transmission electron micrographs of sections of peritoneal mast cells from adult rats were measured by digitized planimetry. A histogram constructed using equivalent volumes calculated from the measured areas assuming approximation of the granules to spheres showed a periodic multimodal distribution in which the modes fell at volumes that were successively larger integral multiples of the volume at the first mode. Application of a moving-bin technique to the data confirmed the presence of the modes. We propose a mechanism of fusion of unit sized granules to account for the multimodal distribution. The presence of pear- and dumbbell-shaped granules in mast cells is consistent with this mechanism.

Animals↗

PubMed2005

Image fusion analysis of (99m)Tc-HYNIC-Tyr(3)-octreotide SPECT and diagnostic CT using an immobilisation device with external markers in patients with endocrine tumours.

PURPOSE: The aim of this study was to assess the value of multimodality imaging using a novel repositioning device with external markers for fusion of single-photon emission computed tomography (SPECT) and computed tomography (CT) images. The additional benefit derived from this methodological approach was analysed in comparison with SPECT and diagnostic CT alone in terms of detection rate, reliability and anatomical assignment of abnormal findings with SPECT. METHODS: Fifty-three patients (30 males, 23 females) with known or suspected endocrine tumours were studied. Clinical indications for somatostatin receptor (SSTR) scintigraphy (SPECT/CT image fusion) included staging of newly diagnosed tumours (n=14) and detection of unknown primary tumour in the presence of clinical and/or biochemical suspicion of neuroendocrine malignancy (n=20). Follow-up studies after therapy were performed in 19 patients. A mean activity of 400 MBq of (99m)Tc-EDDA/HYNIC-Tyr(3)-octreotide was given intravenously. SPECT using a dual-detector scintillation camera and diagnostic multi-detector CT were sequentially performed. To ensure reproducible positioning, patients were fixed in an individualised vacuum mattress with modality-specific external markers for co-registration. SPECT and CT data were initially interpreted separately and the fused images were interpreted jointly in consensus by nuclear medicine and diagnostic radiology physicians. RESULTS: SPECT was true-positive (TP) in 18 patients, true-negative (TN) in 16, false-negative (FN) in ten and false-positive (FP) in nine; CT was TP in 18 patients, TN in 21, FP in ten and FN in four. With image fusion (SPECT and CT), the scan result was TP in 27 patients (50.9%), TN in 25 patients (47.2%) and FN in one patient, this FN result being caused by multiple small liver metastases; sensitivity was 95% and specificity, 100%. The difference between SPECT and SPECT/CT was statistically as significant as the difference between CT and SPECT/CT image fusion (P<0.001). Twenty-seven abnormal SPECT findings in 17 patients could not be initially assigned to organs, but were clearly delineated after image fusion. In 21 patients (40%), clinically relevant information was obtained by image fusion as compared with SPECT alone. CONCLUSION: Co-registration of SPECT and diagnostic CT using a cost-effective immobilisation device provides excellent accuracy for tumour detection of endocrine malignancies and is superior to SPECT and CT alone. Image fusion reduces false positive results and can detect additional lesions. Anatomical information provided by CT enables precise localisation of abnormalities observed in SPECT.

Adult↗

PubMed1984

The role of connexon aggregate fusion in gap junction growth.

Gap junctions are thought to grow in size by the incorporation of single connexons and by the fusion of connexon aggregates but the relative importance of these two growth mechanisms, particularly in the formation of large gap junctions, remains unclear. We have used a quantitative approach to distinguish between these two growth mechanisms and present evidence here that gap junctions in Drosophila melanogaster wing discs have a periodic, (approximately equal to 0.025 micron 2), multimodal areal distribution which suggests that gap junctions grow by aggregate fusion. Previously published gap junction areal frequency distributions from another invertebrate, Manduca sexta, and from several vertebrate animals also show periodic (approximately equal to 0.030 micron 2), multimodal areal distributions. We suggest that the interpeak spacings on the frequency distributions represent average areas of 'unit aggregates' of connexons and that the fusion of such unit aggregates with one another and with integral multiples of unit aggregates may be a general mechanism for gap junction growth. Unit aggregates in mammals and insects are similar in surface area rather than in connexon number suggesting that a common molecular mechanism, perhaps operating at the level of the plasma membrane, may be involved in their formation and stabilization.

Animals↗

PubMed2002

Comparison of mutual information-based warping accuracy for fusing body CT and PET by 2 methods: CT mapped onto PET emission scan versus CT mapped onto PET transmission scan.

UNLABELLED: This article assesses the resulting accuracies of 2 registration methods using the same multimodal mutual information registration algorithm. In the indirect, fusion method, the CT dataset is warped onto the PET transmission scan, and then the patient's attenuation-corrected emission dataset is substituted for the transmission dataset. In the direct, fusion method, the CT is warped directly onto the attenuation-corrected emission dataset. METHODS: CT and (18)F-FDG PET image datasets from 14 subjects with malignant lesions in the thorax were registered. In both CT and PET imaging acquisitions, the patient's arms were at the patient's side, resting on the scanning couch in a manner similar to that of routine PET acquisition procedures. The accuracy of the 2 warping registrations was assessed by measuring the distance between lesion centroids on CT and PET emission after fusion. RESULTS: The indirect method has a statistically smaller mean error, 6.2 mm, than the direct method, 10.6 mm. CONCLUSION: The indirect method appears to be the more accurate/reliable choice for fusing body CT and FDG PET.

Algorithms↗

PubMed2002

Image fusion between 18FDG-PET and MRI/CT for radiotherapy planning of oropharyngeal and nasopharyngeal carcinomas.

PURPOSE: Accurate diagnosis of tumor extent is important in three-dimensional conformal radiotherapy. This study reports the use of image fusion between (18)F-fluoro-2-deoxy-D-glucose positron emission tomography (18FDG-PET) and magnetic resonance imaging/computed tomography (MRI/CT) for better targets delineation in radiotherapy planning of head-and-neck cancers. METHODS AND MATERIALS: The subjects consisted of 12 patients with oropharyngeal carcinoma and 9 patients with nasopharyngeal carcinoma (NPC) who were treated with radical radiotherapy between July 1999 and February 2001. Image fusion between 18FDG-PET and MRI/CT was performed using an automatic multimodality image registration algorithm, which used the brain as an internal reference for registration. Gross tumor volume (GTV) was determined based on clinical examination and 18FDG uptake on the fusion images. Clinical target volume (CTV) was determined following the usual pattern of lymph node spread for each disease entity along with the clinical presentation of each patient. RESULTS: Except for 3 cases with superficial tumors, all the other primary tumors were detected by 18FDG-PET. The GTV volumes for primary tumors were not changed by image fusion in 19 cases (89%), increased by 49% in one NPC, and decreased by 45% in another NPC. Normal tissue sparing was more easily performed based on clearer GTV and CTV determination on the fusion images. In particular, parotid sparing became possible in 15 patients (71%) whose upper neck areas near the parotid glands were tumor-free by 18FDG-PET. Within a mean follow-up period of 18 months, no recurrence occurred in the areas defined as CTV, which was treated prophylactically, except for 1 patient who experienced nodal recurrence in the CTV and simultaneous primary site recurrence. CONCLUSION: This preliminary study showed that image fusion between 18FDG-PET and MRI/CT was useful in GTV and CTV determination in conformal RT, thus sparing normal tissues.

Adult↗

PubMed1997

[Image fusion of different tomographic methods (PET/CT/MRI) effectively contribute to therapy planning].

Image fusion (and image registration) is a fundamental topic of three-dimensional (3D) medical image processing. The need to register different imaging modalities arises for several reasons: some anatomical details, especially soft tissues, are more easily seen in MRI than CT images, but the bony structures are better visualised by CT. The visualisation of any morphological volume with a functional data set which may come from SPECT or PET is a very important method both for research purposes and for routine diagnostics. Our developed programs allowing us to solve image fusion tasks also contained some free available softwares. These packages helped us to start additional software development for more advanced applications having elaborated this way the informatical background of multimodality image processing. This paper presents three oncological applications of the image fusion: PET-CT-MRI registration for 3D radiotherapy planning, PET-MRI registration in planning surgery and metastasis localization. All these registration processes were solved by the landmark-matching registration method.

Decision Making↗

PubMed2005

Large-scale evaluation of multimodal biometric authentication using state-of-the-art systems.

We examine the performance of multimodal biometric authentication systems using state-of-the-art Commercial Off-the-Shelf (COTS) fingerprint and face biometric systems on a population approaching 1,000 individuals. The majority of prior studies of multimodal biometrics have been limited to relatively low accuracy non-COTS systems and populations of a few hundred users. Our work is the first to demonstrate that multimodal fingerprint and face biometric systems can achieve significant accuracy gains over either biometric alone, even when using highly accurate COTS systems on a relatively large-scale population. In addition to examining well-known multimodal methods, we introduce new methods of normalization and fusion that further improve the accuracy.

Algorithms↗

PubMed1997

Normal fusion for three-dimensional integrated visualization of SPECT and magnetic resonance brain images.

UNLABELLED: Multimodality visualization aims at efficiently presenting integrated information obtained from different modalities, usually combining a functional modality (SPECT, PET, functional magnetic resonance imaging) with an anatomical modality [CT, magnetic resonance imaging (MRI)]. This paper presents a technique for three-dimensional integrated visualization of SPECT and magnetic resonance brain images, where MRI is used as a framework of reference for the display of the SPECT data. METHODS: A novel technique for three-dimensional integrated visualization of functional and anatomical information, called normal fusion, is presented. With this technique, local functional information is projected onto an anatomic structure. RESULTS: The normal fusion technique is applied to three cases of SPECT/MRI integration. The results are presented, discussed and evaluated for clinical relevance. CONCLUSION: The results for three-dimensional integrated display of SPECT and MR brain images indicate that the normal fusion technique provides a potentially comprehensive and diagnostically valuable presentation of cerebral blood perfusion in relation to the anatomy of the brain.

Autistic Disorder↗

PubMedDate not supplied

New avenues in 3-D computerised (stereopathological) imaging of breast cancer (review).

Multimodal methods of three-dimensional (3-D) imaging of breast cancer are described. These involve scanning confocal microscopy, using 50 MHz acoustic or near-infrared images, four-view (tetrahedral) radiography and x-ray projection microscopy. Computerised volume data from these techniques can be used to produce three-dimensional images of tissue ranging from 500 microns to approximately 4 mm in thickness. Preliminary findings indicate that stereoscopic images or 3-D computerised reconstructions are capable of advancing the understanding of the structure of ductal carcinoma in situ, lesions simulating microinvasive breast carcinoma, surgical clearance of high-grade calcifying ductal carcinoma in situ, and the 3-D growth patterns of invasive forms of breast carcinoma. In the future computerised image fusion techniques seem likely to be able to take advantage of multimodal imaging of breast cancers, thus correcting primary imaging artefacts, improving robustness, and combining complementary information. In addition, the use of computerised tetrahedral radiography may change the intraoperative assessment of breast cancers, which mostly depend at present upon subsequent laboratory procedures that take days to perform.

Breast Neoplasms↗

PubMed2006

Minimally invasive transforaminal lumbar interbody fusion with unilateral pedicle screw fixation.

OBJECT: Posterior lumbar interbody fusion (PLIF) has been shown to be effective in the treatment of axial low-back pain. Minimally invasive spine surgery for arthrodesis has several advantages, including quicker patient recovery, less postoperative pain, and less destruction of adjacent tissue. The purpose of this paper is to evaluate the clinical outcomes after PLIF procedures in which unilateral pedicle screw fixation was used. METHODS: Prospective data were collected in 34 patients undergoing a one-level minimally invasive transforaminal lumbar interbody fusion (TLIF) in 2003. Conservative therapy, including physical therapy and aggressive multimodality pain management, had failed in all patients. Selection was based on magnetic resonance imaging studies demonstrating degenerative disc disease. All patients underwent a unilateral TLIF procedure in conjunction with posterior unilateral pedicle screw fixation. Twenty patients in whom the follow-up duration was longer than 6 months were included in this study. The follow-up duration in all patients ranged from 6 to 12 months. Seventeen (85%) of 20 patients had a good result, which was defined as a greater than 20-point reduction in the Oswestry Disability Index (ODI) score. The other three patients had no improvement. The mean preoperative ODI score of 57 improved to 25 after surgery (p < 0.005). In the 17 patients who demonstrated improvement, the mean ODI score improved from 57 to 18. The patients' visual analog scale pain scores improved from 8.3 to 1.4 (p < 0.005) after surgery. In patients who received Workers' Compensation, three (75%) of four improved. Follow-up computerized tomography scans were obtained in all 20 patients at 6 months. At that time, 13 of the patients demonstrated some degree of fusion, and no symptomatic pseudarthrosis was noted. CONCLUSIONS: Minimally invasive TLIF in conjunction with unilateral pedicle screw instrumentation is an effective treatment for axial low-back pain in appropriately selected patients.

Adult↗

PubMed2005

Multiport N x N multimode air-clad holey fiber coupler for high-power combiner and splitter.

We report 2 x 2 and 4 x 4 multimode air-clad holey fiber couplers fabricated by a novel fusion and tapering technique. The devices showed excellent port-to-port coupling uniformity over a wide spectral range of 800-1650 nm. The 4 x 4 coupler showed a low insertion loss of 9.9 dB and excess loss of 3.9 dB at 1310 nm. Because of its unique air cladding and the high numerical aperture of the holey fiber structure, the device showed strong potential in high-power applications.

Journal Article↗

PubMed2005

[A medical image fusion method based on fuzzy mathematics].

In this paper, a method based on fuzzy mathematics to fuse multimodality medical images was presented. The improved FCM algorithm was adopted to segment images, and automatic fuzzy redistribution algorithm to define the subject degree. 16 different combinations of image tissues and 16 context relations, 256 models altogether, were considered. The result showed that the method had the great ability of anti-error and anti-segmentation interference and had the characteristics of robustness, quickness, and accuracy.

Algorithms↗

PubMedDate not supplied

[Nuclear medicine approach for the detection and characterization of liver metastasis].

Nuclear Medicine procedures include in vitro techniques able to detect and characterize liver metastases. Among the in vitro applications, circulating tumour markers level determination provides information about the response to therapy and the presence of active disease. While after a successful antineoplastic treatment the serum biological marker level usually falls, a progressive increase represents an alarm signal of occult/residual tumour or metastatic dissemination. However, this method is not able to identify the site of the lesion, therefore complementary imaging techniques are needed to confirm or exclude the diagnostic suspicion. The main advantages of the in vitro methods are the low cost and, consequently, the possibility of periodically checking of tumour activity. The in vivo nuclear medicine techniques are usually performed as second level test. In fact, they can play a role in detecting and characterizing hepatic metastases when radiological methods are inconclusive or when selected tumour seeking agents (i.e. radiolabelled antibodies, radioiodinated mIBG, Octreotide, etc.) can be used. Moreover, the introduction of PET procedures has provided physicians with a useful tool for the evaluation of tumour behaviour (glucose consumption, proteic, synthesis, receptor expression). A further significant diagnostic improvement is represented by the recent introduction of the multimodality co-registration of images (including CT scan, MRI, SPET, PET). The fusion imaging allows a superimposition of the functional and metabolic information to the morphological one.

Biomarkers, Tumor↗

PubMed2025

Multimodal CustOmics: A unified and interpretable multi-task deep learning framework for multimodal integrative data analysis in oncology.

Characterizing cancer presents a delicate challenge as it involves deciphering complex biological interactions within the tumor's microenvironment. Clinical trials often provide histology images and molecular profiling of tumors, which can help understand these interactions. Despite recent advances in representing multimodal data for weakly supervised tasks in the medical domain, achieving a coherent and interpretable fusion of whole slide images and multi-omics data is still a challenge. Each modality operates at distinct biological levels, introducing substantial correlations between and within data sources. In response to these challenges, we propose a novel deep-learning-based approach designed to represent multi-omics & histopathology data for precision medicine in a readily interpretable manner. While our approach demonstrates superior performance compared to state-of-the-art methods across multiple test cases, it also deals with incomplete and missing data in a robust manner. It extracts various scores characterizing the activity of each modality and their interactions at the pathway and gene levels. The strength of our method lies in its capacity to unravel pathway activation through multimodal relationships and to extend enrichment analysis to spatial data for supervised tasks. We showcase its predictive capacity and interpretation scores by extensively exploring multiple TCGA datasets and validation cohorts. The method opens new perspectives in understanding the complex relationships between multimodal pathological genomic data in different cancer types and is publicly available on Github.

Deep Learning↗

PubMed2001

Image-guided endoscopic spine surgery: Part I. A feasibility study.

STUDY DESIGN: A feasibility study was performed to determine the efficacy of computer assistance in endoscopic spine surgery. OBJECTIVES: To assess a new method for computer assistance based on image guidance during thoracoscopic or any endoscopic spine procedure. To evaluate the reproducibility, the sensitivity and the reliability of the technique first in vitro and second in clinical use. SUMMARY OF BACKGROUND DATA: The computer-based, image-guided surgery is now a routine tool used in open spine surgery. Exposure of the anatomy of the vertebra is needed for registration. This methodology is inapplicable in endoscopic approach. Fluoroscopic-based navigation combines the technology of image-guided surgery and C-arm fluoroscopy. The navigation is based on the fluoroscopic images acquired before surgery. This technology is applicable to endoscopic surgery but the navigation is based on fluoroscopic image. The computed tomography images are not exploited. There are no published data on a technique that allows image-guided surgery based on computed tomography and magnetic resonance imaging. METHOD: A laboratory study was performed on a thoracic human spine. One vertebra was marked on the right lateral side of the body with five titanium marks. A percutaneous reference frame was specifically designed to be placed in the pedicle of the same marked vertebrae. The reference frame acted as a 3D localizer and a registration tool. The spine model was scanned including the reference frame. A standard Stealth station treatment guidance platform (Medtronic, Sofamor Danek, Memphis, TN) was used for simulation. The registration was obtained using the reference frame. Twenty navigation procedure trials were done and the error was recorded based on the distance between the anatomical point and the corresponding virtual one. RESULTS: Registration was always possible using the stealth station and a standard spine navigational software (spine 3, Medtronic Sofamor Danek, Memphis, TN). The mean error after registration given by the computer was 0.96 mm. The mean error recorded during the navigation simulation was 1.6 mm. CONCLUSIONS: This technique allows the possibility of computed tomography and magnetic resonance imaging-based, image-guided endoscopic surgery. It is probable that in the near future, as image fusion technology improves, the fluoronavigation based on fluoroscopic images would enable to navigate on multimodal images. Otherwise the technique described in this article is the only reproducible one that allows computed-tomography-based computer assistance during endoscopic procedures.

Endoscopy↗

PubMed2026

Data-centric, robust, and explainable multimodal deep learning for clinical decision support: A systematic review.

PURPOSE: Multimodal deep learning is increasingly proposed for clinical decision support (CDS) under a "data-centric" framing that prioritizes label quality, missing-modality robustness, distribution shift, calibration, and explainability. Prior reviews have examined multimodal medical AI, CDS, and data-centric methods separately, but none address their intersection. We mapped the modalities, fusion strategies, and data-centric and explainability techniques used in this recent literature, quantified how often each is implemented rather than merely mentioned, assessed deployment-relevant evidence (external validation, clinical-outcome measurement, equity), and formally appraised study-level risk of bias. METHODS: Following the PRISMA 2020 statement (PROSPERO CRD420261427815; registered retrospectively), we screened 150 records and included primary, clinical, multimodal studies that applied machine or deep learning to a decision-support task and reported at least one quantitative result. Two reviewers screened and extracted data with consensus adjudication. Each study was coded against pre-specified operational definitions, separating implemented or empirically evaluated techniques from those only mentioned. Study-level risk of bias was assessed with PROBAST + AI. Synthesis was narrative. RESULTS: Thirty-one studies met inclusion; 30 (97%) were published between 2024 and 2026, with a median of three modalities (range 2-6), most commonly structured EHR (71%) and imaging (39%). Data-centric techniques were frequently reported (74-84% across label-noise, distribution-shift, calibration, missing-modality and class-imbalance handling; equity 61%). However, external validation was reported in only 4/31 studies (13%), a clinical or provider outcome in 3/31 (10%), and no study reported routine deployment. Overall risk of bias was high in 27/31 studies (87%), driven by the analysis domain. CONCLUSION: Within this recent, self-selected slice of the field, technical robustness and explainability techniques are widely reported but rarely validated out-of-distribution or against clinical outcomes, and the underlying evidence is at high risk of bias. Progress requires external multi-site validation, clinical-outcome measurement, formal bias appraisal, and adherence to AI reporting standards (e.g., TRIPOD + AI) before deployment can be justified.

Deep Learning↗

PubMed1996

Source propagation of interictal spikes in temporal lobe epilepsy. Correlations between spike dipole modelling and [18F]fluorodeoxyglucose PET data.

Source localization methods were applied to interictal spikes from scalp EEGs and correlated with metabolic (PET scan) data in eight patients suffering from drug-resistant temporal lobe epilepsy (TLE). Dipolar sources, [18F]fluorodeoxyglucose (18FDG)-PET data and anatomical images (MRI) were projected into the same three-dimensional coordinates system. Averaged spikes were adequately modelled by two or three dipolar sources with different onset time of activation but overlapping activity (mean residual variance 3.4 +/- 2.1%). Although, in all patients, spike modelling demonstrated dipolar sources in both mesial and lateral temporal cortex, dipole propagation was consistent with the early involvement of only one of these two areas (mesio-temporal, five patients; lateral and polar neocortex, three patients). Six patients showed a unilateral interictal decrease in glucose uptake, as measured with 18FDG-PET, in the temporal lobe ipsilateral to the EEG spike focus. Temporal hypometabolism was bilateral in one patient and absent in the remaining case. When projected onto PET-scan slices, the dipolar sources of these patients were always included within the hypometabolic area. However, within the hypometabolic zone, the decrease in glucose uptake was not found to be more pronounced in regions containing dipoles. Therefore the spatio-temporal spread of neuronal hyperactivity underlying interictal spiking suggests the presence of preferential epileptogenic networks inside the hypometabolic temporal lobe. Fusion of bioelectric, metabolic and anatomical data proves to be a convenient way of summarizing multimodal information from non-invasive investigations in TLE patients entering an epilepsy surgery programme, and suggests that both interictal spike dipole modelling and 18FDG-PET data might be useful, as a complement to ictal electro-clinical data, in the presurgical evaluation of such patients.

Adult↗

PubMed2002

An ectopic mediastinal parathyroid adenoma accurately located by a single-day imaging protocol of Tc-99m pertechnetate-MIBI subtraction scintigraphy and MIBI-SPECT-computed tomographic image fusion.

PURPOSE: Because ectopic parathyroid adenoma (PA) is a frequent cause of failed initial surgery, an imaging approach with accurate preoperative localization is recommended by some authors in patients with primary hyperparathyroidism (HPT). METHODS: The authors describe a 52-year-old woman in whom primary HPT was diagnosed incidentally during a screening program for osteoporosis. The peculiarity of this case is that the patient was examined before operation in a single-day multimodal imaging protocol based on the combination of high-resolution cervical ultrasound, planar Tc-99m pertechnetate-MIBI scans, and an MIBI-SPECT-computed tomographic (CT) image fusion study. An ectopic PA was accurately located in the upper middle mediastinum, close to the lower margin of the sternal notch. RESULTS: Guided by the MIBI-SPECT-CT fusion images, the surgeon performed a limited median sternotomy and easily removed the PA that was revealed before operation. To confirm the completeness of resection, a bilateral neck exploration was performed through the same incision, with identification of three normally sized parathyroid glands. CONCLUSIONS: Our experience suggests the utility of multimodality imaging procedures for the accurate preoperative localization of PAs, particularly when they are present in ectopic mediastinal locations. Such procedures, including the MIBI-SPECT-CT image fusion study, can be performed in a single day.

Adenoma↗