SEARCH · PubMed Health

Results for “Contrastive learning”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3Linked to original sources

PubMed2003

Learning foreign vowels.

Two hypotheses have recently been put forward to account for listeners' ability to distinguish and learn contrasts between speech sounds in foreign languages. First, Best's Perceptual Assimilation Model and Flege's Speech Learning Model both predict that the ease with which a listener can tell one non-native phoneme from another varies directly with the extent to which these sounds assimilate to different native phonemes (Best, 1994; also Best, McRoberts, & Goodell, 2001; Flege, 1991). Second, Logan, Lively, & Pisoni (1991) have argued that training listeners to identify non-native phonemes teaches them sets of exemplars rather than more abstract distinctive feature values. I report here the results of three sets of experiments designed to test these hypotheses, in which American English listeners were trained to categorize German nonlow vowels. The first set of experiments show that some instances of the same contrast between German vowels are more easily discriminated than others, a result incompatible with the predictions of either Best's or Flege's models, but compatible with the alternative category recognition interpretation. The second set of experiments reveals effects of contextual and speaker variation on listeners' ability to learn [tense] but not [high] contrasts between foreign vowels, and are thus at least partly compatible with an exemplar model of foreign category learning (Pisoni, Lively, & Logan, 1994; also Nosofsky, 1986). The third set of experiments compares the predictions of Nosofsky's (1986) selective attention exemplar model of category learning with those of a feature learning model in tests of listeners' learning the natural classes to which the German vowels belong. The results are mixed: listeners learned the features that define the natural classes of [+/- high] and [+/- back] vowels, but could have learned either the feature that defines the natural classes of [+/- tense] vowels or sets of [+/- tense] exemplars. Natural classes defined by abstract distinctive feature values are thus learnable, even if their membership is phonetically polymorphous.

Adult↗

PubMed2026

Scalable, generalizable and uncertainty-aware integration of spatial multiomics across diverse modalities and platforms with SCIGMA.

Recent advances in spatial omics technologies have enabled simultaneous profiling of transcriptomic, proteomic, epigenomic, metabolomic and imaging data at high spatial resolution, offering unprecedented opportunities to dissect tissue complexity. However, integrating these diverse and large-scale spatial multimodal datasets remains a major computational challenge. We present SCIGMA, a scalable and generalizable deep learning framework for spatial multiomics integration. SCIGMA introduces an uncertainty-aware contrastive learning objective and multiview graph neural networks to preserve modality-specific signals while learning biologically meaningful joint representations. Unlike previous methods, SCIGMA provides spatially resolved uncertainty estimates, interpretably identifying regions of biological or technical heterogeneity. SCIGMA supports integration of up to five modalities, and its modular framework is extensible to future technologies with even more modalities. It also scales to more than 1 million spatial locations, enabling analysis of high-resolution datasets such as Visium HD and Xenium Prime. We evaluated SCIGMA across 19 datasets spanning 8 modalities, 10 tissues and 9 platforms. On benchmarkable datasets, SCIGMA outperformed other methods in spatial domain detection, modality preservation, feature reconstruction and reproducibility. SCIGMA identifies biologically meaningful structures, refined spatial domains and modality-specific regulatory programs, providing a robust, flexible and future-ready solution for scalable spatial multimodal integration.

Multiomics↗

PubMed2004

Processing local transitions versus long-distance syntactic hierarchies.

A recent study by Fitch and Hauser reported that finite-state grammars can be learned by non-human primates, whereas phrase-structure grammars cannot. Humans, by contrast, learn both grammars easily. This species difference is taken as the critical juncture in the evolution of the human language faculty. Given the far-reaching relevance of this conclusion, the question arises as to whether the distinction between these two types of grammars finds its reflection in different neural systems within the human brain.

Animals↗

PubMed2026

ARGformer: learning on ancestral recombination graphs with transformers.

MOTIVATION: Recent advances in inference of the ancestral recombination graph (ARG), which describes how segments of chromosomes trace back through recombination and shared lineages, have made it possible to reconstruct genome-wide genealogies for large cohorts, but it remains difficult to summarize and use this information for population genetic analyses. RESULTS: We present ARGformer, an encoder-only transformer that learns context-dependent embeddings with a self-supervised masked objective finetuned with contrastive learning for downstream retrieval tasks. We train ARGformer on genealogies from coalescent simulations and on genealogies inferred from ancient and present-day Homo sapiens genomes. Using only these learned embeddings, without access to genotype matrices, ARGformer captures patterns of global population structure and supports ancestry inference through clustering and nearest-neighbor retrieval. On genealogies that include archaic hominins, ARGformer can highlight Denisovan-derived segments in Oceanian genomes and reveals Oceanian-like ancestry in South American Indigenous populations. AVAILABILITY AND IMPLEMENTATION: ARGformer is available at https://github.com/AI-sandbox/ARGformer.

Humans↗

PubMed1976

Procedural antecedents of behavioral contrast: a re-examination of errorless learning.

Behavioral contrast reliably occurred in pigeons following errorless discrimination training, contrary to Terrace's (1963) observations. In the main experiment, a 60-sec green keylight, associated with a variable-interval 30-sec schedule of reinforcement alternated with a 60-sec period of extinction when the key was dark. Such aspects of the discrimination training procedure as: (1) the amount of prior nondifferential exposure to the positive stimulus before the discrimination was instituted, and (2) the rapidity with which the negative stimulus was introduced (whether progressively or abruptly) directly influenced the amount of behavioral contrast produced. This occurred independently of the number of errors made by a pigeon during acquisition of the discrimination. In a series of control experiments, substitution of a red keylight for the dark key during extinction resulted in greater behavioral contrast, while an increase to 3 min in the duration of the green keylight associated with reinforcement attenuated the behavioral contrast effect.

Journal Article↗

PubMed2006

Learning sequence movements in a homogenous sample of patients with Parkinson's disease.

We investigated the acquisition of sequence movements in Parkinson's disease (PD) by means of the serial reaction time (SRT) task. To this end, we used a sample of PD patients that fell within the same stage of the disease. Sixteen PD patients and 16 age-, sex- and education-matched control subjects performed the SRT task with a first-order conditional (FOC) sequence and with a second-order conditional (SOC) sequence. The results showed that the group of PD patients could be divided into two distinct subgroups: a fast PD patient subgroup (n=11) and a slow PD patient subgroup (n=5). FOC and SOC sequence learning in faster PD patients proved to be highly comparable to the group of controls. In contrast, learning of FOC and SOC sequences was severely impaired in slower PD patients. Since slow PD patients also scored lower on measures of cognitive functioning than faster PD patients, we assume that the deficits in SRT learning of the former reflect some more general cognitive impairment. This indicates that SRT performance can provide additional information about the cognitive abilities of PD patients, and accordingly may contribute to disease screening.

Aged↗

PubMed2003

Learning in Aplysia: looking at synaptic plasticity from both sides.

Until recently, learning and memory in invertebrate organisms was believed to be mediated by relatively simple presynaptic mechanisms. By contrast, learning and memory in vertebrate organisms is generally thought to be mediated, at least in part, by postsynaptic mechanisms. But new experimental evidence from research using a model invertebrate organism, the marine snail Aplysia, indicates that this apparent distinction between invertebrate and vertebrate synaptic mechanisms of learning is invalid: learning in Aplysia cannot be explained in terms of exclusively presynaptic mechanisms. NMDA-receptor-dependent LTP appears to be necessary for classical conditioning in Aplysia. Furthermore, modulation of trafficking of postsynaptic ionotropic glutamate receptors underlies behavioral sensitization in this snail. Exclusively presynaptic processes appear to support only relatively brief memory in Aplysia. More persistent memory is likely to be mediated by postsynaptic processes, or by presynaptic processes whose expression depends upon retrograde signals.

Animals↗

PubMed2005

Distributed neural plasticity for shape learning in the human visual cortex.

Expertise in recognizing objects in cluttered scenes is a critical skill for our interactions in complex environments and is thought to develop with learning. However, the neural implementation of object learning across stages of visual analysis in the human brain remains largely unknown. Using combined psychophysics and functional magnetic resonance imaging (fMRI), we show a link between shape-specific learning in cluttered scenes and distributed neuronal plasticity in the human visual cortex. We report stronger fMRI responses for trained than untrained shapes across early and higher visual areas when observers learned to detect low-salience shapes in noisy backgrounds. However, training with high-salience pop-out targets resulted in lower fMRI responses for trained than untrained shapes in higher occipitotemporal areas. These findings suggest that learning of camouflaged shapes is mediated by increasing neural sensitivity across visual areas to bolster target segmentation and feature integration. In contrast, learning of prominent pop-out shapes is mediated by associations at higher occipitotemporal areas that support sparser coding of the critical features for target recognition. We propose that the human brain learns novel objects in complex scenes by reorganizing shape processing across visual areas, while taking advantage of natural image correlations that determine the distinctiveness of target shapes.

Form Perception↗

PubMed2003

Learned dynamics of reaching movements generalize from dominant to nondominant arm.

Accurate performance of reaching movements depends on adaptable neural circuitry that learns to predict forces and compensate for limb dynamics. In earlier experiments, we quantified generalization from training at one arm position to another position. The generalization patterns suggested that neural elements learning to predict forces coded a limb's state in an intrinsic, muscle-like coordinate system. Here, we test the sensitivity of these elements to the other arm by quantifying inter-arm generalization. We considered two possible coordinate systems: an intrinsic (joint) representation should generalize with mirror symmetry reflecting the joint's symmetry and an extrinsic representation should preserve the task's structure in extrinsic coordinates. Both coordinate systems of generalization were compared with a naïve control group. We tested transfer in right-handed subjects both from dominant to nondominant arm (D-->ND) and vice versa (ND-->D). This led to a 2 x 3 experimental design matrix: transfer direction (D-->ND/ND-->D) by coordinate system (extrinsic, intrinsic, control). Generalization occurred only from dominant to nondominant arm and only in extrinsic coordinates. To assess the dependence of generalization on callosal inter-hemispheric communication, we tested commissurotomy patient JW. JW showed generalization from dominant to nondominant arm in extrinsic coordinates. The results suggest that when the dominant right arm is used in learning dynamics, the information could be represented in the left hemisphere with neural elements tuned to both the right arm and the left arm. In contrast, learning with the nondominant arm seems to rely on the elements in the nondominant hemisphere tuned only to movements of that arm.

Adolescent↗

PubMed2004

EEG correlates of coordinate processing during intermanual transfer.

Goal-directed movements require mapping of target information to patterns of muscular activation. While visually acquired information about targets is initially encoded in extrinsic, object-centered coordinates, muscular activation patterns are encoded in intrinsic, body-related coordinates. Intermanual transfer of movements previously learned with one hand is accomplished by the recall of unmodified extrinsic coordinates if the task is performed in original orientation. Intrinsic coordinates are retrieved in case of mirror-reversed orientation. In contrast, learned extrinsic coordinates are modified during the mirror movement and intrinsic coordinates during the originally oriented task. To investigate the neural processes of recall and modification, electroencephalogram (EEG) recording was employed during the performance of a figure drawing task previously trained with the right hand in humans. The figure was reproduced with the right hand (Learned-task) and with the left hand in original (Normal-task) and mirror orientations (Mirror-task). Prior to movement onset, beta-power and alpha- and beta-coherence decreased during the Normal-task as compared with the Learned-task. Negative amplitudes over fronto-central sites during the Normal-task exceeded amplitudes manifested during the Learned-task. In comparison to the Learned-task, coherences between fronto-parietal sites increased during the Mirror-task. Results indicate that intrinsic coordinates are processed during the pre-movement period. During the Normal-task, modification of intrinsic coordinates was revealed by cerebral activation. Decreased coherences appeared to reflect suppressed inter-regional information flow associated with utilization of intrinsic coordinates. During the Mirror-task, modification of extrinsic coordinates induced activation of cortical networks.

Adult↗

PubMed2025

Orthrus: Towards Evolutionary and Functional RNA Foundation Models.

In the face of rapidly accumulating genomic data, our ability to accurately predict key mature RNA properties that underlie transcript function and regulation remains limited. Pre-trained genomic foundation models offer an avenue to adapt learned RNA representations to biological prediction tasks. However, existing genomic foundation models are trained using strategies borrowed from textual domains that do not leverage biological domain knowledge. Here, we introduce Orthrus, a Mamba-based mature RNA foundation model pre-trained using a novel self-supervised contrastive learning objective with biological augmentations. Orthrus is trained by maximizing embedding similarity between curated pairs of RNA transcripts, where pairs are formed from splice isoforms of 10 model organisms and transcripts from orthologous genes in 400+ mammalian species from the Zoonomia Project. This training objective results in a latent representation that clusters RNA sequences with functional and evolutionary similarities. We find that the generalized mature RNA isoform representations learned by Orthrus significantly outperform genomic foundation models on mRNA property prediction tasks, and requires only a fraction of fine-tuning data to do so. Finally, we show that Orthrus is capable of capturing divergent biological function of individual transcript isoforms.

Journal Article↗

PubMed2002

Early cortical distinction between memories that pertain to ongoing reality and memories that don't.

Patients with anterior limbic lesions, in particular of the posterior orbitofrontal cortex, often act on the basis of memories that do not relate to ongoing reality and justify their behavior with invented stories that can mostly be traced back to real events (spontaneous confabulation). Recent studies demonstrated that the patients fail to suppress activated memory traces that do not pertain to ongoing reality. In the present study, we used a similar paradigm and high-resolution event-related potentials to explore when this suppression happens. Healthy subjects made two runs of a continuous recognition task, composed of the same set of pictures, and were requested to indicate picture recurrences only within the ongoing run. Thus, performance in the first run depends on new learning, whereas the second run requires the ability to realize whether a picture is solely familiar from its occurrence in the previous run ('distracter') or whether it has already appeared in the ongoing second run ('target'). We find that correct rejection (suppression) of currently irrelevant pictures (distracters of run 2) is associated with absent negative deflection of a frontal potential and absence of a specific cortical potential map configuration after 220-300 ms. By contrast, learning and recognition of repeatedly presented information is associated with cortical amplitude modulation after 400-480 ms. These findings indicate that by the time the content of a mental association is recognized and consolidated, its cortical representation has already been adjusted according to whether it relates to ongoing reality or not. This sequence may also explain the ability to distinguish between the memory of a true event and the memory of a thought.

Adult↗

PubMed1988

Preserved memory abilities in thalamic amnesia.

The pattern of preserved learning abilities is described in a severely amnesic patient after bilateral thalamic infarction. Experimental findings cannot be accounted for both by the view that only episodic memory is impaired in amnesia, while semantic memory is spared, and by the theory that what is spared in amnesia is procedural learning contrasted with impaired declarative memory. In agreement with Warrington and Weiskrantz (1982), diencephalic amnesia is considered to be a disconnection syndrome between the frontal and temporal lobes. The conditions for showing spared and impaired memory in amnesics are specified on the basis of the performance of the patient and of the data available in the literature. This allows us to derive practical suggestions for programmes aimed at remediation of memory defects.

Adult↗

PubMed2003

Differential effects of stress on escape and reflex responses to nociceptive thermal stimuli in the rat.

Acute stress has been shown to increase latencies of nociceptive reflexes, and this effect is considered evidence for stress-induced analgesia. However, tests for nociception that rely on motivated operant escape assess cerebral processing of pain and could be modulated independent of reflex responses. We therefore compared the effects of an acute stressor (restraint) on escape responses and lick/guard reflexes to stimulation of the paws by a thermally regulated floor. Testing sessions included a pre-test exposure to 36 degrees C, followed by a test trial in which either escape from 44 or 36 degrees C or reflex responses to 44 degrees C were observed. Behavioral responses to stress were assessed during a three day period, with baseline testing on day 1, post-stress or control testing on day 2, and evaluation of long-term stress effects on day 3. On day 2, half the animals received 15 min of restraint stress, followed by 15-min pre-test and test trials. Licking and guarding responses to thermal stimulation during 44 degrees C test trials were significantly reduced by restraint stress, confirming previously reported stress effects on nociceptive reflexes. In contrast, learned escape responses to the same thermal stimulus were significantly enhanced after stress. The increase in operant sensitivity suggests that acute restraint, a form of psychological stress, produces hyperalgesia for a level of thermal stimulation that preferentially activates C nociceptors. These results are discussed in relation to studies involving physical or psychological forms of stress, different nociceptive stimuli, and assessment strategies used to evaluate thermal pain sensitivity.

Animals↗

PubMed2025

DNABERT-S: pioneering species differentiation with species-aware DNA embeddings.

SUMMARY: We introduce DNABERT-S, a tailored genome model that develops species-aware embeddings to naturally cluster and segregate DNA sequences of different species in the embedding space. Differentiating species from genomic sequences (i.e. DNA and RNA) is vital yet challenging, since many real-world species remain uncharacterized, lacking known genomes for reference. Embedding-based methods are therefore used to differentiate species in an unsupervised manner. DNABERT-S builds upon a pre-trained genome foundation model named DNABERT-2. To encourage effective embeddings to error-prone long-read DNA sequences, we introduce Manifold Instance Mixup (MI-Mix), a contrastive objective that mixes the hidden representations of DNA sequences at randomly selected layers and trains the model to recognize and differentiate these mixed proportions at the output layer. We further enhance it with the proposed Curriculum Contrastive Learning (C2LR) strategy. Empirical results on 28 diverse datasets show DNABERT-S's effectiveness, especially in realistic label-scarce scenarios. For example, it identifies twice more species from a mixture of unlabeled genomic sequences, doubles the Adjusted Rand Index (ARI) in species clustering, and outperforms the top baseline's performance in 10-shot species classification with just a 2-shot training. AVAILABILITY AND IMPLEMENTATION: Model, codes, and data are publically available at https://github.com/MAGICS-LAB/DNABERT_S.

Sequence Analysis, DNA↗

PubMed2026

CeLLTra: aligning cell names with gene expression via a pathway-informed transformer.

MOTIVATION: Single-cell RNA sequencing (scRNA-Seq) technology enables detailed exploration of gene expression at the individual cell level, crucial for annotating cell types and understanding cellular diversity. Traditional methods for cell type annotation often rely on marker genes and manual labeling, posing challenges due to low data quality and incomplete reference datasets. RESULTS: We developed CeLLTra, a novel contrastive learning framework that leverages a Transformer-based model integrating biological pathway information to group genes into super tokens, effectively capturing comprehensive gene expression from scRNA-Seq data. By combining this pathway-informed Transformer with a pretrained domain-specific language model, CeLLTra accurately aligns cell-type annotations with gene expression profiles. Evaluations on a large-scale human scRNA-Seq dataset showed that CeLLTra significantly outperformed state-of-the-art methods in supervised and zero-shot cell-type prediction. Additionally, CeLLTra generalized well to external datasets, improving clustering performance and enabling better characterization of cancerous cell states in tumor-infiltrating myeloid cells from non-small cell lung cancer patients. AVAILABILITY AND IMPLEMENTATION: CeLLTra is freely available on GitHub (https://github.com/WJZheng-group/CeLLTra) and Zenodo (https://doi.org/10.5281/zenodo.17666735). The datasets underlying this article are the following: GSE201333 and GSE127465. All these datasets are publicly available and can be freely accessed on the Gene Expression Omnibus repository.

Humans↗

PubMed2026

GenOT: generative optimal transport enables spatiotemporal interpolation and generation in cross-platform spatial transcriptomics.

Spatial transcriptomics technologies have revolutionized the analysis of spatial gene expression, yet integrating spatial information and generating data across heterogeneous samples remain challenging. We present GenOT, a generative framework combining multi-scale graph self-supervised contrastive learning with optimal transport barycenter theory for efficient cross-slice and cross-platform spatiotemporal interpolation. The core innovation of GenOT lies in introducing an optimal transport barycenter-based interpolation algorithm, which mathematically models spatial distribution differences across heterogeneous samples to reconstruct spatiotemporal gene expression dynamics. Extensive evaluations demonstrate that GenOT consistently outperforms existing approaches in spatial domain identification, cross-platform interpolation, and developmental trajectory reconstruction.

Spatial Transcriptomics↗

PubMed2024

DNABERT-S: Pioneering Species Differentiation with Species-Aware DNA Embeddings.

We introduce DNABERT-S, a tailored genome model that develops species-aware embeddings to naturally cluster and segregate DNA sequences of different species in the embedding space. Differentiating species from genomic sequences (i.e., DNA and RNA) is vital yet challenging, since many real-world species remain uncharacterized, lacking known genomes for reference. Embedding-based methods are therefore used to differentiate species in an unsupervised manner. DNABERT-S builds upon a pre-trained genome foundation model named DNABERT-2. To encourage effective embeddings to error-prone long-read DNA sequences, we introduce Manifold Instance Mixup (MI-Mix), a contrastive objective that mixes the hidden representations of DNA sequences at randomly selected layers and trains the model to recognize and differentiate these mixed proportions at the output layer. We further enhance it with the proposed Curriculum Contrastive Learning (C2LR) strategy. Empirical results on 23 diverse datasets show DNABERT-S's effectiveness, especially in realistic label-scarce scenarios. For example, it identifies twice more species from a mixture of unlabeled genomic sequences, doubles the Adjusted Rand Index (ARI) in species clustering, and outperforms the top baseline's performance in 10-shot species classification with just a 2-shot training. Model, codes, and data is publicly available at https://github.com/MAGlCS-LAB/DNABERT_S.

Journal Article↗