SEARCH · PubMed Health

Results for “Multimodal Fusion”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4Linked to original sources

PubMed2005

Fusion images of Tl-201 SPECT and FDG PET with CT in detection of cervical carcinoma with bladder invasion.

Thallium-201 single-photon emission computed tomography (Tl-201 SPECT) and F-18 fluorodeoxyglucose positron emission tomography (FDG PET) are accurate and excellent methods for detecting malignant tumors and related metabolic abnormalities. However, there are still some limitations in clinical application. The major limitation of SPECT is poor image resolution. PET is less sensitive for detecting lesions of the urinary system adjacent to the bladder because of bladder radioactivity. Both T1-201 and F-18 FDG PET often provide less information than x-ray computed tomography (CT) about the anatomic landmarks needed to precisely locate lesions. Recent development of multimodality image registration and fusion has been shown to be an excellent tool to resolve these drawbacks. The authors present a rare case of residual cervical carcinoma with bladder invasion, which was detected by fusion imaging of Tl-201 SPECT and FDG PET with CT. This noninvasive image technique may become a valuable alternative to identify bladder invasion in cervical cancer.

Adenocarcinoma↗

PubMed2026

Multimodal artificial intelligence and machine learning in oncology: from data integration to precision cancer care.

Cancer remains a major global health burden, with approximately 20 million new cases and 9.7 million cancer-related deaths reported globally in 2022. While advances in radiological imaging, molecular profiling, and clinical data have enhanced the interpretation of disease progression, the availability of multiple such modalities still does not meet the needs of a large patient population. This narrative review focuses on the role of multimodal artificial intelligence and machine learning in bridging the gap in interpreting heterogeneous modalities to improve risk prediction, prognostic assessment, and treatment decision-making in precision oncology. Multimodal frameworks such as Pathomic Fusion illustrate how complementary histopathological and genomic information can be integrated for cancer diagnosis and prognostic modeling. Multimodal models have demonstrated potential in virtual biopsy, cancer screening, prognostic prediction, radiotherapy planning, intraoperative guidance, and clinical-trial design using digital twins and synthetic control arms. The major limitations of incorporating multimodal artificial intelligence and machine learning in oncology include data heterogeneity, demographic or institutional biases, and reproducibility challenges that hinder translation. Accordingly, appropriate data-governance strategies, fairness audits, and privacy-preserving approaches such as federated learning should be considered where appropriate. Future progress will depend on the development of standardized benchmarking datasets, robust external validation, seamless integration with electronic health records and picture archiving and communication systems, and the implementation of explainable, secure, and clinically validated multimodal artificial intelligence frameworks that support precision oncology in routine clinical practice.

deep learning↗

PubMed2006

Direct quantitative in vivo comparison of calcified atherosclerotic plaque on vascular MRI and CT by multimodality image registration.

PURPOSE: To investigate direct volumetric in vivo correspondence of calcified atherosclerotic plaque lesions in MRI and CT images of the thoracic aorta by multimodality image registration and fusion. MATERIALS AND METHODS: Twelve CT (11 noncontrast and one contrast) and MRI (TruFISP, contrast T1-weighted volumetric interpolated breath-hold examination (VIBE)) data sets were co-registered by approximate segmentation of the aorta and subsequent automatic co-registration by maximization of mutual information (MI). We quantitatively assessed 22 co-registered calcified plaque lesions on CT and MRI. RESULTS: The three-dimensional registration consistency and accuracy were 1.74 +/- 1.3 mm, and 2.42 +/- 1.65 mm, respectively. The ratio of CT/MRI calcified plaque volume decreased asymptotically with MRI volume, and correlated with average CT lesion density (r = 0.72) for small lesions (<25 mm(3)). The average calcified plaque volume, circumferential extent, and maximal radial width by MRI were significantly smaller compared to CT (35%, 68%, and 53%, respectively; P < 0.05). CONCLUSION: Software co-registration allowed precise, direct, and voxel-based comparison of calcified atherosclerotic plaque lesions imaged by MRI and CT. In comparison with co-registered MRI, overestimation of calcified plaque in aortic CT due to "blooming" correlates with the average lesion density for small plaques, and is greater for small plaques.

Aged↗

PubMed2026

Advancing cancer detection and treatment using longitudinal routine clinical data.

Cancer management remains fragmented across its continuum, from late-stage diagnosis and salvage therapies to non-personalized surveillance. Here, we present Oncoformer, a unified multimodal transformer model trained on the China Oncology Multimodal Prediction and Surveillance Study (COMPASS) cohort (3.67 million individuals, 17.7 million clinical visits) and validated on independent external cohorts, including the UK Biobank. Oncoformer integrates longitudinal electronic health records with chest X-ray imaging to address multiple clinical tasks: pan-cancer diagnosis (area under the receiver operating characteristic curve [AUROC] = 0.956), future cancer prediction up to 1 year before diagnosis (AUROC = 0.869), tumor stage inference (mean AUROC > 0.90), patient-specific treatment-response forecasting, and recurrence-free survival stratification across ten cancer types (all p < 0.01). Staging predictions were independently validated against postoperative pathological endpoints and shown to converge on core cancer genomic pathways. By translating routine clinical data into a dynamic view of cancer evolution, Oncoformer provides a framework for risk-informed cancer prediction and treatment stratification using routine clinical data.

Humans↗

PubMed2004

Stereotactic radiation treatment planning and follow-up studies involving fused multimodality imaging.

OBJECT: Innovative new software solutions may enable image fusion to produce the desired data superposition for precise target definition and follow-up studies in radiosurgery/stereotactic radiotherapy in patients with intracranial lesions. The aim is to integrate the anatomical and functional information completely into the radiation treatment planning and to achieve an exact comparison for follow-up examinations. Special conditions and advantages of BrainLAB's fully automatic image fusion system are evaluated and described for this purpose. METHODS: In 458 patients, the radiation treatment planning and some follow-up studies were performed using an automatic image fusion technique involving the use of different imaging modalities. Each fusion was visually checked and corrected as necessary. The computerized tomography (CT) scans for radiation treatment planning (slice thickness 1.25 mm), as well as stereotactic angiography for arteriovenous malformations, were acquired using head fixation with stereotactic arc or, in the case of stereotactic radiotherapy, with a relocatable stereotactic mask. Different magnetic resonance (MR) imaging sequences (T1, T2, and fluid-attenuated inversion-recovery images) and positron emission tomography (PET) scans were obtained without head fixation. Fusion results and the effects on radiation treatment planning and follow-up studies were analyzed. The precision level of the results of the automatic fusion depended primarily on the image quality, especially the slice thickness and the field homogeneity when using MR images, as well as on patient movement during data acquisition. Fully automated image fusion of different MR, CT, and PET studies was performed for each patient. Only in a few cases was it necessary to correct the fusion manually after visual evaluation. These corrections were minor and did not materially affect treatment planning. High-quality fusion of thin slices of a region of interest with a complete head data set could be performed easily. The target volume for radiation treatment planning could be accurately delineated using multimodal information provided by CT, MR, angiography, and PET studies. The fusion of follow-up image data sets yielded results that could be successfully compared and quantitatively evaluated. CONCLUSIONS: Depending on the quality of the originally acquired image, automated image fusion can be a very valuable tool, allowing for fast (approximately 1-2 minute) and precise fusion of all relevant data sets. Fused multimodality imaging improves the target volume definition for radiation treatment planning. High-quality follow-up image data sets should be acquired for image fusion to provide exactly comparable slices and volumetric results that will contribute to quality contol.

Adult↗

PubMed2001

Image fusion in open-architecture PACS-environment.

Multimodal digital imaging is common in many fields of diagnosis and therapy planning - there is great interest in matching globally, fusing or registering data from the same part of the body. In practice, there are still difficulties in customizing image fusion in hospitals. Efficient routine use of image fusion requires, among others, an image management infrastructure - a picture archiving and communication system (PACS) - to provide storage of image data in a standard digital format, intelligent image management and fault-tolerant high-speed image networking. In order to customize image fusion, advances in both fusion software and hardware are also needed. The algorithms should be automatic, fast and accurate enough. Registration of multimodal data also creates a need for different display techniques and user-friendly interfaces. Image fusion has been impractical and too tedious to be performed in routine work, but in the future, fused images will be used in clinical practice - even in teleradiological consultation.

Humans↗

PubMedDate not supplied

Multimodality imaging of lymphocytic migration using lentiviral-based transduction of a tri-fusion reporter gene.

PURPOSE: Previous work showed quantitative imaging of T-cell migration into a tumor site by positron emission tomography (PET), using retroviral transduction of mutated thymidine kinase (sr39TK) reporter genes into immunized T-lymphocytes. PROCEDURES AND RESULTS: In order to improve the sensitivity and flexibility of the imaging analysis, lentivirus, that expressed sr39TK, was used to transduce the lymphocytes that migrated to an immunogenic sarcoma site. In comparison to retrovirally transduced lymphocytes, the lentivirally transduced lymphocytes showed enhanced PET signal when equal numbers of transduced lymphocytes were transferred. Furthermore, in order to utilize multimodality in vivo imaging capability, a tri-fusion reporter gene containing sr39TK, synthetic Renilla luciferase (hRluc), and enhanced green fluorescent protein (eGFP) was inserted into a lentiviral transfer vector. Using the adoptive transfer model, tumor-specific lymphocytic migration was detected by both microPET scan and bioluminescence imaging. CONCLUSION: The multimodal imaging strategy coupled with lentiviral reporter construct delivery demonstrated here can facilitate future molecular imaging studies.

Animals↗

PubMed1992

Integrated presentation of multimodal brain images.

This article discusses the fusion of brain images from multiple modalities as well as the presentation of the integrated image information. The paper has three parts. First, individual brain imaging modalities are compared as regards clinical appreciation, invasiveness, dimensionality, spatial resolution, temporal resolution, and cost. Next, methods to combine multiple images are briefly surveyed and collated by characteristics as accuracy, patient-friendliness, reproducibility, labour-extensiveness, feasibility of retrospective matching, and general applicability. Finally, techniques to display multimodal image information are outlined and examples of the various options for integrated presentation are shown.

Brain↗

PubMed1994

Computer-assisted technologies in lung cancer diagnosis and staging.

A paradigm for digital image processing in radiological diagnosis and an appropriate algorithmic instrumentation toolset for the implementation of image processing methods on inexpensive computers and workstations are outlined briefly. Examples of computer-assisted technologies for lung cancer differential diagnosis are given that exhibit considerable increase in diagnostic accuracy. Multimodal image processing and data fusion in lung cancer diagnosis are discussed and a 'road map' for examining lung cancer patients is suggested on the basis of clinical experience in the use of different modalities for lung cancer staging.

Algorithms↗

PubMedDate not supplied

PET-CT fusion imaging in differentiating physiologic from pathologic FDG uptake.

Interpretation of positron emission tomographic (PET) scans in the absence of correlative anatomic information can be challenging. PET-computed tomography (CT) fusion imaging is a novel multimodality technology that allows the correlation of findings from two concurrent imaging modalities in a comprehensive examination. CT demonstrates exquisite anatomic detail but does not provide functional information, whereas 2-[fluorine 18]fluoro-2-deoxy-D-glucose (FDG) PET reveals aspects of tumor function and allows metabolic measurements. Subtle findings at FDG PET that might otherwise be disregarded or interpreted as physiologic variants may lead to detection of a malignant process after being correlated with simultaneously acquired CT findings. Alternatively, equivocal CT findings, which could represent malignant tumor, reactive changes, or fibrosis, can be clarified with the help of the additional metabolic information provided by concurrent FDG PET. Accurate interpretation of FDG PET scans requires a thorough knowledge of the normal physiologic distribution of FDG and of normal variants that may reduce the accuracy of PET studies, thereby significantly affecting patient treatment. Although in rare instances PET-CT cannot help resolve the diagnostic dilemma, it is enjoying widespread acceptance in the medical imaging community, usually allowing differentiation of physiologic variants from juxtaposed or mimetic neoplastic lesions and more accurate tumor localization.

Adult↗

PubMed2001

[Fusion of MRI, fMRI and intraoperative MRI data. Methods and clinical significance exemplified by neurosurgical interventions].

The aim of this work was to realize and clinically evaluate an image fusion platform for the integration of preoperative MRI and fMRI data into the intraoperative images of an interventional MRI system with a focus on neurosurgical procedures. A vertically open 0.5 T MRI scanner was equipped with a dedicated navigation system enabling the registration of additional imaging modalities (MRI, fMRI, CT) with the intraoperatively acquired data sets. These merged image data served as the basis for interventional planning and multimodal navigation. So far, the system has been used in 70 neurosurgical interventions (13 of which involved image data fusion--requiring 15 minutes extra time). The augmented navigation system is characterized by a higher frame rate and a higher image quality as compared to the system-integrated navigation based on continuously acquired (near) real time images. Patient movement and tissue shifts can be immediately detected by monitoring the morphological differences between both navigation scenes. The multimodal image fusion allowed a refined navigation planning especially for the resection of deeply seated brain lesions or pathologies close to eloquent areas. Augmented intraoperative orientation and instrument guidance improve the safety and accuracy of neurosurgical interventions.

Adult↗

PubMed2004

The space of senses: impaired crossmodal interactions in a patient with Balint syndrome after bilateral parietal damage.

Balint syndrome after bilateral parietal damage involves a severe disturbance of space representation including impaired oculomotor behaviour, optic ataxia, and simultanagnosia. Binding of object features into a unique spatial representation can also be impaired. We report a patient with bilateral parietal lesions and Balint syndrome, showing severe spatial deficits in several visual tasks predominantly affecting the left hemispace. In particular, we tested whether a loss of spatial representation would affect crossmodal interactions between simultaneous visual and tactile events occurring at the same versus different locations. A tactile discrimination task, where spatially congruent or incongruent visual cues were delivered near the patient's hands, was used. Following stimulation of the left hand in the left side of space, we observed visuo-tactile interactions that were not modulated by spatially congruent conditions. In contrast, performance following stimulation of the right hand in the right side of space was affected in a spatially selective manner--facilitated for congruent stimuli and slowed for incongruent stimuli. To dissociate effects on somatotopic and spatiotopic coordinates, we crossed the patient's hands during unimodal tactile discriminations. Tactile performance of the left hand improved when it was positioned in the right hemispace, whereas placing the right hand in left space produced no significant changes, suggesting that left-sided tactile inputs are coded with respect to a combination of limb- and trunk-centred coordinates. These data converge with recent findings in animals and healthy humans to indicate a critical role of the posterior parietal cortex in multimodal spatial integration, and in the fusion of different coordinates into a unified representation of space.

Analysis of Variance↗

PubMed2026

An Integrative Morphological and Genomic Analysis With a Refined Fluorescence In Situ Hybridization (FISH) Threshold and Novel Kinase Fusions in a Large Asian Cohort of Spitzoid Neoplasms.

Differentiating atypical Spitz tumors (ASTs) from true Spitz melanomas (SMs) and conventional melanomas with spitzoid features (MSFs) remains a formidable diagnostic challenge. Because current molecular epidemiological data are overwhelmingly derived from Caucasian cohorts, the genomic landscape of Asian populations remains largely unexplored. To elucidate the molecular progression landscape and refine the diagnostic criteria, we performed a comprehensive multimodal analysis-integrating histomorphology, immunohistochemistry, multiprobe fluorescence in situ hybridization (FISH), and targeted RNA/DNA-based next-generation sequencing (NGS)-on a cohort of 140 spitzoid neoplasms. This cohort, comprising 126 ASTs, 8 SMs, and 6 MSFs, represents the largest Asian cohort to date. Malignant phenotype strongly correlated with lesional asymmetry, deep atypical mitoses, a sheet-like growth pattern, diffuse preferentially expressed antigen of melanoma positivity, and significant loss of p16 expression (64.3% in SM/MSF vs 9.5% in ASTs; P < .0001). Building upon the established melanoma FISH criteria, we optimized a prognostic threshold of ≥2 FISH abnormalities specifically tailored for spitzoid neoplasms. We demonstrated that isolated single chromosomal aberrations (particularly MYB loss) are relatively stable events that are frequent in indolent ASTs, whereas our refined ≥2 threshold yielded 100% sensitivity and 92.5% specificity for predicting regional lymph node metastasis/local recurrence. Molecularly, NGS identified mutually exclusive initiating driver alterations (comprising kinase fusions and HRAS mutations) in 89.9% of true Spitz neoplasms, a remarkably high prevalence suggesting a distinct genetic background in Asian populations. We also characterized 5 entirely novel kinase fusions (ZNF24::ROS1, PCBP1::ROS1, NUMA1::RET, CBWD1::ALK, and TPR::NTRK1). Furthermore, NGS definitively segregated true Spitz neoplasms from morphological mimics (MSF), which lacked fusions and were driven by canonical genomic alterations of the conventional melanoma pathway. Integrating these genomic landscapes validated a stepwise progression model. Although isolated kinase fusions drove indolent ASTs, malignant SM invariably harbored concurrent pathogenic secondary alterations, demonstrating a profound reliance on CDKN2A/B, TP53, and CDK4 aberrations. Ultimately, we propose an integrated diagnostic algorithm combining morphological evaluation, the refined FISH threshold, and comprehensive NGS profiling, providing a precise, evidence-based framework for pathway classification and clinical management of spitzoid neoplasms.

fluorescence in situ hybridization↗

PubMed2000

Postoperative analgesic effects of celecoxib or rofecoxib after spinal fusion surgery.

UNLABELLED: Nonsteroidal antiinflammatory drugs are recommended for the multimodal management of postoperative pain and may have a significant opioid-sparing effect after major surgery. The analgesic efficacy of the cyclooxygenase-2 nonsteroidal antiinflammatory drugs, celecoxib and rofecoxib, have not been evaluated after major orthopedic surgery. This study was designed to determine whether the administration of a preoperative dose of celecoxib or rofecoxib to patients who have undergone spinal stabilization would decrease patient-controlled analgesia (PCA) morphine use and/or enhance analgesia. We evaluated 60 inpatients undergoing spine stabilization by one surgeon. All patients received PCA morphine. The patients were divided into three groups. Preoperatively, they were given oral celecoxib 200 mg, rofecoxib 50 mg, or placebo. The outcome measures included pain scores and 24-h morphine use at six times during the first 24 postoperative h. The total dose of morphine and the cumulative doses for each of the six time periods were significantly more in the placebo group than in the other two groups. The morphine dose was significantly less in five of the six time intervals in the rofecoxib group compared with the celecoxib group. The pain scores were significantly less in the rofecoxib group than in the other two groups at two of the six intervals, and less than the placebo group in an additional interval. Although both rofecoxib and celecoxib produce similar analgesic effects in the first 4 h after surgery, rofecoxib demonstrated an extended analgesic effect that lasted throughout the 24-h study. We thus recommend that rofecoxib be used as a preoperative component of pain management that includes PCA morphine in patients undergoing spine stabilization surgery. IMPLICATIONS: The cyclooxygenase-2-specific nonsteroidal antiinflammatory drugs, celecoxib and rofecoxib, both demonstrate an opioid-sparing effect after spinal fusion surgery. Celecoxib resulted in decreased morphine use for the first 8 h after surgery, whereas rofecoxib demonstrated less morphine use throughout the 24-h study period.

Adult↗

PubMed1993

Theoretical considerations on the formation of secretory granules in the rat pancreas.

Rat pancreatic zymogen granule sizes were determined by analysis of electron micrographs of the pancreas from adult and newborn rats. Areas of granule profiles were measured and converted to equivalent volumes. Histograms of the equivalent volumes showed integral multimodal distributions which were evaluated for goodness of fit with two models, unit addition and random fusion. Previous analyses of zymogen granule size distributions have failed to recognize the multimodality we have observed. Distributions of equivalent volumes for the two models were developed using Monte Carlo simulation. In the case of the granules from the newborn rats, the distribution of granule sizes gave a better fit with the random fusion model, whereas the granules from the adult rats had distributions with a better fit to the unit granule addition model. The estimated unit granule sizes for the two different ages were the same. Both unit addition and random fusion models propose that following formation of secretory granules from Golgi-derived material, the granules fuse with one another to create a wide dispersion of granule sizes. The present results extend the evidence for fusional growth of secretory granules, originally developed for the mast cell, to the zymogen granules of pancreas. All normal cells previously studied have yielded secretory granule distributions most consistent with unit addition. The basis for the expression of random fusion in the newborn rather than the more usual unit addition is not known.

Aging↗

PubMed1997

Integration of functional and anatomical brain images.

This article concerns the integration of functional and anatomical volumetric brain images. Integration consists of two steps: matching or registration, where the images are brought into spatial agreement, and fusion or simultaneous display where the registered multimodal image information is presented in an integrated fashion. Approaches to register multiple images are divided into extrinsic methods based on artificial markers, and intrinsic matching methods based solely on the patient related image data. The various methods are compared by a number of characteristics, which leads to a clear preference for one class of intrinsic methods, viz. voxel-based matching. Furthermore, two- and three-dimensional techniques to display multimodality image information are outlined.

Brain↗

PubMed2003

Development of a new reporter gene system--dsRed/xanthine phosphoribosyltransferase-xanthine for molecular imaging of processes behind the intact blood-brain barrier.

We report the development of a novel dual-modality fusion reporter gene system consisting of Escherichia coli xanthine phosphoribosyltransferase (XPRT) for nuclear imaging with radiolabeled xanthine and Discosoma red fluorescent protein for optical fluorescent imaging applications. The dsRed/XPRT fusion gene was successfully created and stably transduced into RG2 glioma cells, and both reporters were shown to be functional. The level of dsRed fluorescence directly correlated with XPRT enzymatic activity as measured by ribophosphorylation of [14C]-xanthine was in vitro (Ki = 0.124 +/- 0.008 vs. 0.00031 +/- 0.00005 mL/min/g in parental cell line), and [*]-xanthine octanol/water partition coefficient was 0.20 at pH = 7.4 (logP = -0.69), meeting requirements for the blood-brain barrier (BBB) penetrating tracer. In the in vivo experiment, the concentration of [14C]-xanthine in the normal brain varied from 0.20 to 0.16 + 0.05% dose/g under 0.87 + 0.24% dose/g plasma radiotracer concentration. The accumulation in vivo in the transfected flank tumor was to 2.4 +/- 0.3% dose/g, compared to 0.78 +/- 0.02% dose/g and 0.64 +/- 0.05% dose/g in the control flank tumors and intact muscle, respectively. [14C]-Xanthine appeared to be capable of specific accumulation in the transfected infiltrative brain tumor (RG2-dsRed/XPRT), which corresponded to the 585 nm fluorescent signal obtained from the adjacent cryosections. The images of endogenous gene expression with the "sensory system" have to be normalized for the transfection efficiency based on the "beacon system" image data. Such an approach requires two different "reporter genes" and two different "reporter substrates." Therefore, the novel dsRed/XPRT fusion gene can be used as a multimodality reporter system in the biological applications requiring two independent reporter genes, including the cells located behind the BBB.

Amino Acid Sequence↗

PubMed1995

[Image fusion of MRI and immunoscintigraphy with MAb-170 in ovarian tumors].

In recent years multimodality imaging achieved growing importance. It is mostly performed by means of quite expensive software and hardware solutions. In the present pilot study a simple and low-cost procedure was developed to achieve image fusion in the pelvis. The image data of immunoscintigraphy (SPECT) and MRI were transferred to a personal computer and combined by standard software for image manipulation. The results in eleven patients with space-occupying lesions in the pelvis showed that adequate anatometabolic slices could be achieved. The results show a tendency to increased specificity and precision of multimodality imaging in comparison with SPECT and MRI alone. In conclusion, the low-cost solution, as developed by us, is feasible in clinical practice. Its results are reliable in clinical decision making.

Adult↗