Serial attentional resource allocation during parallel feature value tracking
Curation statements for this article:-
Curated by eLife
eLife assessment
This study presents an important finding on the serial attentional resource allocation during parallel feature value tracking. The evidence supporting the claims of the authors is solid, although further clarification for high-/low-precision assigning, task effectivity of active tracking, and data analysis would have strengthened the study. The work will be of broad interest to psychology and cognitive science.
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
- Neuroscience (eLife)
Abstract
The visual system has evolved the ability to track features like color and orientation in parallel. This property aligns with the specialization of processing these feature dimensions in the visual cortex. But what if we ask to track changing feature-values within the same feature dimension? Parallel tracking would then have to share the same cortical representation, which would set strong limitations on tracking performance. We address this question by measuring the precision of color representations when human observers track the color of two superimposed dot clouds that simultaneously change color along independent trajectories in color-space. We find that tracking precision is highly imbalanced between streams and that tracking precision changes over time by alternating between streams at a rate of ~1 Hz. These observations suggest that, while parallel color tracking is possible, it is highly limited, essentially allowing for only one color-stream to be tracked with precision at a given time.
Article activity feed
-
Author Response
Reviewer #1 (Public Review):
Strengths:
The study addresses an intriguing research question that fills a gap in existing literature, and was carefully designed and well-executed, with a series of experiments and control experiments.
We thank the reviewer for the positive statement about the conception and execution of the study as well as the potential interest to the community within a broader field.
Weaknesses:
- My main concern is the null effect of precision estimation pattern between cued and un-cued trials. It is well established that relative to the un-cued stimuli, the cued stimuli obtain more attentional resource and this study claimed serial attentional resource allocation during parallel feature value tracking. However, all Experiments 3a-c did not find any difference in precision estimates between …
Author Response
Reviewer #1 (Public Review):
Strengths:
The study addresses an intriguing research question that fills a gap in existing literature, and was carefully designed and well-executed, with a series of experiments and control experiments.
We thank the reviewer for the positive statement about the conception and execution of the study as well as the potential interest to the community within a broader field.
Weaknesses:
- My main concern is the null effect of precision estimation pattern between cued and un-cued trials. It is well established that relative to the un-cued stimuli, the cued stimuli obtain more attentional resource and this study claimed serial attentional resource allocation during parallel feature value tracking. However, all Experiments 3a-c did not find any difference in precision estimates between these two types of trials.
We would like to annotate that the terminology „cued versus uncured trials“ in the usual sense of distinguishing between stimuli being attended versus unattended is admittedly somewhat misleading in the current work. In cued and uncured trials of the present experiments 3a-c the allocation of attention is equal. The difference is that the color stream that is attended first is defined (knowable) in the cued but not in the uncued trials. In all cases subjects had to track both color streams and report any of the probed streams as accurately as possible. In other words, the overall allocation of attention in cued and uncured trials is the same. Also, the „cue“ did not provide any information regarding the following probe (no indication of likelihood for a probe in that stream as in an attention experiment). It was entirely irrelevant and was therefore expected not to alter subjects overall performance – as confirmed by the mentioned null-result. The performed test shows, that the reported bias of ~2:1 does not depend on whether in one set of the trials one stream is cued or not. The sole purpose of the “cue” was to subconsciously redirect attention briefly towards that particular stream at the start of each trial in order to ‘phase-reset’ any process, switching/oscillating feature-based resources over time. Performance imbalance across streams is hereby not altered by this phase-reset but remains constant since precision ratio is estimated across a large number of trials and durations. To clarify this issue, we rephrased relevant descriptions in the methods section.
- Results of Exp.1 in the main text were different from those in Figure.
Thank you for spotting that error. We have corrected the figure accordingly.
- It would be helpful to add more details for the assignation of response 1 and response 2 to target 1 and target 2, respectively, in all experiments.
For Experiment 2 and 3 only one response per trial was required by the subjects. This design was chosen to avoid potentially ambiguous response-target assignments.
However in the first experiment, as the reviewer points out, subjects gave two color estimates (one for each of the tracked color streams) within each trial. Given that we intend to split subjects’ target-response differences (precisions) into two distributions (based on the idea that each stream is being maintained by an independent attentional resource), there are two possible ways of assigning responses:
(1) We split responses into a best and worst independent of which response was given first.
(2) Alternatively, we assign target-response pairs based on the order of response. The assumption would be, that the first response would be the one with the highest confidence and would be paired with the target closest. This pairing would occur independent of the second response, which is consequently paired with the remaining target. This leaves open the possibility of the second target-response difference being better than the first one due to resource fluctuations. In general, this strategy would be less ‘rigid’ in dividing the two precision-responses into ‘good’ and ‘bad’ responses and was consequently chosen.
To avoid problems arising from the ambiguity of target-response assignments, in all following experiments (2/3), subjects were required to give one response per trial only. We will go into further detail on this issue with reviewer 3 as well, including a numerical example. The logic behind the target-response assignments in experiment 1 has been described in more detail in the methods.
Reviewer #2 (Publlic Review):
The authors asked the question about whether and how changing feature values within the same feature dimensions are tracked. Using a series of behavioral studies combined with modeling approaches, the authors report interesting results regarding a robust, uneven distribution of attentional resources between two changing feature values (in a 2:1 ratio), alternating at 1 Hz. Although the results are clear, it is important to rule out the possible biases due to computational processes. The results advanced our understanding of how parallel tracking of multiple feature values within the same dimension is achieved.
We thank the reviewer for the summary, including the potential impact on the field and we look forward to clarify methodological imprecisions.
Reviewer #3 (Public Review):
The study is interesting and the results are informative in how well people can report colors of two superimposed dot clouds. It reveals that there are trade-offs between reporting two colors. However, I have a few basic but major concerns with the present study and its conclusions about people's abilities to continuously track color values and the rate at which attention may be allocated across the two streams which I am outlining below.
We thank the reviewer for the positive description of our findings and look forward to address any remaining issues.
- The first concern regards the task that was used to measure continuous tracking of feature values, which in my view is ambiguous in whether it truly assesses active tracking of features or rather short-term memory of the last-seen colors. Specifically, participants were viewing two colored dot clouds that then turned gray, and were asked to report each of the colors they saw using continuous report. The test usually occurred after 6-8s (in Exp. 1 &2), so while not completely predictable, participants could easily perform the task without tracking both feature streams continuously and simply perform the color report based on the very last colors they saw. In other words, it does not seem necessary to know which color belonged to which stream, or what color it was before, to perform the task successfully. Thus, it is unclear to what extent this task is actually measuring active tracking, the same way tracking of spatial locations in multiple-object tracking tasks has been studied, which is the literature that the authors are trying to draw parallels to. In multiple-object tracking tasks, targets and nontarget objects look identical and so to keep track of which of the moving objects are targets, participants need to attend to them actively and selectively. (Similarly, the original feature-tracking study by Blaser et al., at least in their main experiment, people were asked to track an object superimposed on a second object which required continuous and selective tracking of that object).
The reviewer addresses a very fundamental point regarding ‘tracking’ in general: Does tracking rely on attentional processes or mere perception.
The reviewer posits that subjects may simply ‘report based on the very last color they saw’ without the need to track both features streams continuously. Our argument supported by a broad literature on change blindness, inattentional blindness and related phenomena (c.f. Rensink, 2000) is, that one cannot consciously report a changing feature-value without continuously attending to it, in particular when it moves around randomly in feature space. The report of a feature value at a random unpredictable time t by ‘identifying it’ includes its attentive processing immediately before t. Since the time of the probing identification is random, it must continue throughout the trial. We do also rule out any strategy in which subjects only start tracking after some time (the probe appears between 6-8sec after trial onset) since such a strategy would involve processes of temporal attention as well and increase difficulty.
Lastly, the reviewer refers to Blaser et al. as an example in which attentive tracking would be required, since ‘an object [is] superimposed on a second object’. We do absolutely agree. However, the same design principle applies in the current experiment: Two objects with separate values in feature space, that continuously change, are superimposed, that is, spatially inseparable. We do believe that the continuous movement of the feature values through color space separates this work from previous feature-tracking studies like Re et al., in which the presented features remained static. The latter work gives rise to alternate explanations in terms of working memory (mentioned in the next point of the reviewer). Once feature values keep changing and are relevant, a process of updating their internal representations in order to grant access is required (i.e. attention).
- The main claim that tracking two colors relies on a shared and strictly limited resource is primarily based on the relation between the two responses people give, such that the first response about one color tends to be higher accuracy than for the second response of the other color across participants. In my view, this is a relatively weak version of looking at trade-offs in resources, and it would have been more compelling to show such trade-offs at a single-trial level, or assess them with well-established methods that have been developed to look at attentional bottlenecks such as attention-operating characteristics that allow quantifying the cost of adding an additional task in a precise and much more direct manner.
The reviewer suggests showing trade-offs at a single trial level within subject, which is in essence what we have done in experiment 1. Testing both streams simultaneously, however, has the drawback of introducing interference effects during the report (Reporting the first stream may degrade the precision of reporting the second stream) as well as the mentioned ambiguity between targets and responses. The second and third experiment circumvent this by probing only one color stream, as to analyze the data with a minimal set of assumptions. As the dependent measure of ‘precision’ fluctuates highly across trials, we have to estimate an overall tracking resource by creating a ‘precision’ distribution across many trials.
- Finally, the data of the last experiment is taken as evidence that feature-based selection oscillates at 1Hz between the two streams. This is based on response errors changing across time points with respect to an exogenous cue that is thought to "reset" attentional allocation to one stream. Only one of three data sets (which uses relatively sparse temporal sampling) shows a significant interaction between cue and time, and given that there was no a priori prediction of when such interaction should occur, this result begs for a replication to ensure that this is not a false positive result. Furthermore, based on the analyses done in the paper, it may very well be the case that the presumed "switching rate" is entirely non-oscillatory based on a recent very important paper by Geoffrey Brookshire (2022, Nature Human Behavior) that demonstrates that frequency analysis are not just sensitive to periodic but also aperiodic temporal structures. The paper also has a series of suggested analyses that could be used here to further test the current conclusions.
The reviewer is absolutely correct in doubting the oscillatory nature of the results in Exp3. Importantly, in our discussion we do not claim that a regular periodicity of the attentional process maintains both color streams. In contrast, we stress the point of ‘one-feature at a time’, indicating a constraint that entails alternation between two representations. We do not presume any sort of regularity of this process but, instead, consider the switching being determined by the recurrent processing of tuning towards one of the two relevant values. Our interpretation is therefore largely in line with Brookshires criticism of previous attentional oscillation studies. In fact, we entirely share the doubtful interpretation of attentional oscillations that transfer mathematical modelling onto functional processes. In our study we use the tool of Fourier transformation in a mere methodological manner, in order to quantify alternations between our color streams but not to imply an underlying oscillatory process. We cannot draw conclusions about underlying attentional oscillations especially since we quantify the alternation/switch only across one full and one half period, in exp3a and exp3b respectively.
We make the distinction between oscillations as a methodological tool and functional cognitive process more clear in the paper.
-
-
eLife assessment
This study presents an important finding on the serial attentional resource allocation during parallel feature value tracking. The evidence supporting the claims of the authors is solid, although further clarification for high-/low-precision assigning, task effectivity of active tracking, and data analysis would have strengthened the study. The work will be of broad interest to psychology and cognitive science.
-
Reviewer #1 (Public Review):
Summary:
Through a series of psychophysical experiments, Merkel et al examined the process of feature-based resource allocation during parallel feature value tracking, where subjects are asked to simultaneously track changing but spatially inseparable color streams. They find that tracking precision is highly imbalanced between streams, and the tracking precision changes over time by alternating between streams at a rate of ~1Hz.
Strengths:
The study addresses an intriguing research question that fills a gap in existing literature, and was carefully designed and well-executed, with a series of experiments and control experiments.
Weaknesses:
1. My main concern is the null effect of precision estimation pattern between cued and un-cued trials. It is well established that relative to the un-cued stimuli, the …
Reviewer #1 (Public Review):
Summary:
Through a series of psychophysical experiments, Merkel et al examined the process of feature-based resource allocation during parallel feature value tracking, where subjects are asked to simultaneously track changing but spatially inseparable color streams. They find that tracking precision is highly imbalanced between streams, and the tracking precision changes over time by alternating between streams at a rate of ~1Hz.
Strengths:
The study addresses an intriguing research question that fills a gap in existing literature, and was carefully designed and well-executed, with a series of experiments and control experiments.
Weaknesses:
1. My main concern is the null effect of precision estimation pattern between cued and un-cued trials. It is well established that relative to the un-cued stimuli, the cued stimuli obtain more attentional resource and this study claimed serial attentional resource allocation during parallel feature value tracking. However, all Experiments 3a-c did not find any difference in precision estimates between these two types of trials.
2. Results of Exp.1 in the main text were different from those in Figure.
3. It would be helpful to add more details for the assignation of response 1 and response 2 to target 1 and target 2, respectively, in all experiments. -
Reviewer #2 (Public Review):
The authors asked the question about whether and how changing feature values within the same feature dimensions are tracked. Using a series of behavioral studies combined with modeling approaches, the authors report interesting results regarding a robust, uneven distribution of attentional resources between two changing feature values (in a 2:1 ratio), alternating at 1 Hz. Although the results are clear, it is important to rule out the possible biases due to computational processes. The results advanced our understanding of how parallel tracking of multiple feature values within the same dimension is achieved.
-
Reviewer #3 (Public Review):
The study is interesting and the results are informative in how well people can report colors of two superimposed dot clouds. It reveals that there are trade-offs between reporting two colors. However, I have a few basic but major concerns with the present study and its conclusions about people's abilities to continuously track color values and the rate at which attention may be allocated across the two streams which I am outlining below.
The first concern regards the task that was used to measure continuous tracking of feature values, which in my view is ambiguous in whether it truly assesses active tracking of features or rather short-term memory of the last-seen colors. Specifically, participants were viewing two colored dot clouds that then turned gray, and were asked to report each of the colors they …
Reviewer #3 (Public Review):
The study is interesting and the results are informative in how well people can report colors of two superimposed dot clouds. It reveals that there are trade-offs between reporting two colors. However, I have a few basic but major concerns with the present study and its conclusions about people's abilities to continuously track color values and the rate at which attention may be allocated across the two streams which I am outlining below.
The first concern regards the task that was used to measure continuous tracking of feature values, which in my view is ambiguous in whether it truly assesses active tracking of features or rather short-term memory of the last-seen colors. Specifically, participants were viewing two colored dot clouds that then turned gray, and were asked to report each of the colors they saw using continuous report. The test usually occurred after 6-8s (in Exp. 1 &2), so while not completely predictable, participants could easily perform the task without tracking both feature streams continuously and simply perform the color report based on the very last colors they saw. In other words, it does not seem necessary to know which color belonged to which stream, or what color it was before, to perform the task successfully. Thus, it is unclear to what extent this task is actually measuring active tracking, the same way tracking of spatial locations in multiple-object tracking tasks has been studied, which is the literature that the authors are trying to draw parallels to. In multiple-object tracking tasks, targets and nontarget objects look identical and so to keep track of which of the moving objects are targets, participants need to attend to them actively and selectively. (Similarly, the original feature-tracking study by Blaser et al., at least in their main experiment, people were asked to track an object superimposed on a second object which required continuous and selective tracking of that object).
The main claim that tracking two colors relies on a shared and strictly limited resource is primarily based on the relation between the two responses people give, such that the first response about one color tends to be higher accuracy than for the second response of the other color across participants. In my view, this is a relatively weak version of looking at trade-offs in resources, and it would have been more compelling to show such trade-offs at a single-trial level, or assess them with well-established methods that have been developed to look at attentional bottlenecks such as attention-operating characteristics that allow quantifying the cost of adding an additional task in a precise and much more direct manner.
Finally, the data of the last experiment is taken as evidence that feature-based selection oscillates at 1Hz between the two streams. This is based on response errors changing across time points with respect to an exogenous cue that is thought to "reset" attentional allocation to one stream. Only one of three data sets (which uses relatively sparse temporal sampling) shows a significant interaction between cue and time, and given that there was no a priori prediction of when such interaction should occur, this result begs for a replication to ensure that this is not a false positive result. Furthermore, based on the analyses done in the paper, it may very well be the case that the presumed "switching rate" is entirely non-oscillatory based on a recent very important paper by Geoffrey Brookshire (2022, Nature Human Behavior) that demonstrates that frequency analysis are not just sensitive to periodic but also aperiodic temporal structures. The paper also has a series of suggested analyses that could be used here to further test the current conclusions.
-
