Using computational physiological models of the auditory nerve to understand pitch perception for complex sounds.
From periodic sounds in the natural world, the auditory system extracts pitch, a perceptual correlate of fundamental frequency (F0) which is crucial for perception of speech and music. Many models of pitch perception rely heavily on the rate-place code, the pattern of average firing rates in neural populations tuned to specific frequencies. Using a physiological model of peripheral auditory processing at the level of the auditory nerve, we generated metamers – physically distinct stimuli with identical model outputs – of the rate-place code for stimuli containing one or multiple simultaneous pitches, across a range of frequencies and F0s. We hope that human behavioral and modeled neural responses to these auditory nerve rate metamers will reveal the extent to which different sources of pitch-related information can be used for pitch perception.
How do non-temporal features bias our judgements of timing of auditory sequences?
When listening to sequences of sounds, pitch and timing information interact to form expectations about when and where an auditory event will occur. Disruptions in the pitch or timing of an auditory event (e.g., in virtual settings such as Zoom or phone calls) can create conflicting cues that alter the perception of auditory events making communication more difficult. Furthermore, deficits in utilizing pitch or timing cues are exacerbated in listeners with auditory pathologies such as hearing impairment or amusia that are known to disrupt pitch or timing perception leading to significant problems in listening in noisy environments. This study examines how non-temporal aspects of sounds such as pitch affect relative timing judgments for sequences of sounds. We hypothesize that people use non-temporal cues to group together discrete sound events, leading to a bias in timing where sounds that “belong together” are perceived as occurring closer together in time.
How does the auditory system integrate information over time and frequency to get a coherent pitch percept?
In our daily lives, most of the sounds we encounter are harmonic complex tones. Such tones consist of several pure tone components, all of which can be expressed as some integer n multiplied by some greatest common factor, which we call the fundamental frequency (F0). Notably, a component at the F0 itself is not necessary to evoke a pitch percept at that frequency, and in the case of synchronous component presentation, we refer to this as residual pitch. However, one can elicit a similar percept via short presentations of the components individually, even with gaps of silence between successive harmonics, assuming a sufficient background noise presence; in this case of sequential component presentation, we refer to the percept as virtual pitch. Currently, our research aims to elucidate how spectral and temporal parameters interact to perceive pitch when confronted by “clouds” of complex tone components, particularly in adverse listening conditions and in the presence of multiple complex tones.
Understanding neural correlates of pitch and harmony using EEG
One area of our research is investigating the cortical responses using EEG to varying features of pitch, harmony, and particularly the effect of context on these neural responses. For instance, one of our EEG experiments investigates differences in cortical responses when transitioning from sounds with a salient pitch (harmonic sounds) to those without a salient pitch (inharmonic sounds, including noise). Additionally, another ongoing EEG experiment investigates the differences in cortical evoked potentials using EEG for chord sequences that experiment with participants listening to chord progressions that follow and violate rules of standard western music theory. For all of these studies, we carefully control stimulus parameters to distinguish between sensory priming and contextual priming for all these context effects. This allows for a better understanding of the neural mechanisms underlying pitch perception and the influence of context in music and pitch processing.
Optimizing near-silent fMRI for auditory stimuli.
Functional magnetic resonance imaging (fMRI) is a widely used imaging technique in human neuroscience. It provides crucial insights into cortical representations for perceptual processes, with unmatched spatial resolution compared to other neuroimaging techniques like electroencephalography (EEG) and magnetoencephalography (MEG). However, the magnetic field gradient coils in MRI scanners generate high levels of acoustic noise. Although the effects of this background noise are present in all fMRI studies, they are most problematic in auditory fMRI research. Our research proposes the use of Looping Star, which is a near-silent, multi-echo, 3D radial fMRI technique. Compared to traditional scanning techniques that have sound levels at 105-115 dBA, Looping Star sequences have noise levels at 67-72 dBA. Although this eliminates the confounding effect of loud scanner noise, this is a newly developed method that has poorer resolution and signal to noise ratios (SNR). We are collaborating with Dr. Douglas Noll at the University of Michigan to optimize the silent scanning paradigm using established cortical auditory phenomena as a first step.
