Our Research Lab

Spectro-Temporal Processing

Our work in spectro-temporal processing is grounded in research that goes back at least four decades. Research on spectral shape perception using the auditory profile analysis approach was spear-headed by David M. Green and his colleagues during the 1980s to mid-1990s. This work included relatively simple stimuli including standard stimuli consisting of a multi-tone complex with components spaced on a logarithmic scale and signal stimuli produced by incrementing the level of one of those components. Several investigations used multi-component signals, including increments and decrements in multiple components resulting in sinusoidal spectral shapes.

We have championed a linear-systems analysis approach to spectro-temporal processing, investigating spectral and temporal processing separately and jointly.

STM stimuliOur recent project focused on spectral modulation (SM) perception and how measures of SM depend on the acoustic parameters of duration and presentation level. 

Duration effect: SM detection thresholds were similarly low for the durations of 400 and 200 ms, increased slightly for the 100 ms duration, and increased markedly for the 50 ms duration.

SM duration effects

Spectral modulation detection thresholds as a function of modulation frequency (mean and standard error) with duration as the parameter indicated by symbol type.

Presentation level effect: SM detection thresholds were lowest for 40 dB SL, increased slightly for 20, 30, and 60 dB SL, and markedly higher for the 10 dB SL condition. 

SM level effects

Spectral modulation detection thresholds as a function of modulation frequency (mean and standard error) with sensation level as the parameter indicated by symbol type.

Though fairly robust to changes in duration or level, these results are consistent with previous investigations confirmed that the effects of such parameters should be considered in the context of designing such experiments, models of auditory perception, and/or when interpreting data related to spectral envelope perception.

A third project recently completed involved comparison of multiple stimulus generation methods. Previous studies have used a variety of stimulus generation methods and stimulus parameters to study spectro-temporal modulation (STM) detection.

STM parametersThis makes it difficult to compare outcomes among investigations. Specifically, the choice of stimulus parameters such as carrier type (e.g., tone complex or noise) and modulation waveform shape (i.e., sinusoidal on linear-log or log-log scales) are potentially important considerations. Here we compared the temporal (TM), spectral (SM), and spectro-temporal (STM) modulation detection thresholds at a combination of TM frequency of 4 and 32 Hz and SM frequency of 0.5 and 2 cycles/octaves. The results were rather idiosyncratic across methods and conditions. To better understand these results, we have implemented a model based on The NSL Auditory model (nsltools) developed by the Neural Systems Laboratory was used to predict the detection thresholds for the stimulus conditions of the experiment. This computation model consists of three stages:

Early stage: transforming a given signal to a two-dimensional pattern of neural activity "auditory spectrogram." Our customized MATLAB script is based on Chi, Ru, Shamma (2005) and Ru (2000; Ph.D. Dissertation, U. Maryland).

Cortical stage: cortical stage, representing the estimation of spectral and temporal modulation content from auditory spectrogram to cortical rate-scale representation (original MATLAB script in nsltools)

Threshold prediction: using STRF output to predict a threshold by calculating the zMean of a specific area of each condition and statistic decision.

STM modelThe results showed that: 1) different signal generation methods have a modest effect on detection thresholds; and 2) the model as implemented here can successfully predict detection thresholds in some but not all conditions. We continue to explore predictive models of SM and STM.

Please consider becoming a research participant!