Characterising semantic prioritisation in visual working memory
Curation statements for this article:-
Curated by eLife
eLife Assessment
This useful study addresses a timely question about semantic prioritisation in visual working memory, using behavioural manipulations and drift-diffusion modelling. However, the strength of evidence is incomplete for the broader claims about working-memory representations because the main interpretation relies on indirect inferences from non-decision time, which cannot uniquely identify memory access or retrieval.
This article has been Reviewed by the following groups
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
- Evaluated articles (eLife)
Abstract
Cognitive operations require recently encountered information to remain available beyond the moment of sensory input. However, how such transient representations are accessed, and how they differ from sensory processing and long-term memory, remains unclear. Here, we combine hierarchical drift-diffusion modelling with behavioural manipulations to dissociate the decision processes underlying feature prioritisation in visual working memory. We first reanalysed a previously collected working memory dataset to characterise semantic and perceptual judgements at the level of latent decision processes. Semantic judgements were associated with reduced non-decision time across conditions, indicating faster access to task-relevant information, while advantages in evidence accumulation emerged selectively under higher cognitive demands. Two further experiments manipulated attentional prioritisation using retro-cues and dissociated the effects of interference from mere maintenance. Across manipulations, semantic prioritisation was selectively expressed in pre-accumulation processes and was amplified when representations fell outside the focus of attention or had to be maintained under interference. Together, these results suggest that semantic representations remain more readily accessible than perceptual details when working memory representations fall outside the focus of attention, consistent with a shift towards more abstract, long-term memory-like formats under conditions of limited attentional support.
Article activity feed
-
eLife Assessment
This useful study addresses a timely question about semantic prioritisation in visual working memory, using behavioural manipulations and drift-diffusion modelling. However, the strength of evidence is incomplete for the broader claims about working-memory representations because the main interpretation relies on indirect inferences from non-decision time, which cannot uniquely identify memory access or retrieval.
-
Reviewer #1 (Public review):
Summary:
This paper investigates whether semantic prioritization in visual working memory reflects pre-decisional access, evidence accumulation, or both, using drift diffusion modeling across a reanalysis of prior data and two new experiments. The core finding - that semantic information receives a robust pre-decisional access advantage that is amplified by attentional disruption rather than temporal delay alone - is novel and contributes meaningfully to ongoing debates about the format and accessibility of working memory representations.
Strengths:
The experimental approach is well-motivated, and the use of drift-diffusion modeling to decompose decision components adds analytical value beyond standard RT and accuracy measures. The two new experiments are pre-registered and address important questions. The …
Reviewer #1 (Public review):
Summary:
This paper investigates whether semantic prioritization in visual working memory reflects pre-decisional access, evidence accumulation, or both, using drift diffusion modeling across a reanalysis of prior data and two new experiments. The core finding - that semantic information receives a robust pre-decisional access advantage that is amplified by attentional disruption rather than temporal delay alone - is novel and contributes meaningfully to ongoing debates about the format and accessibility of working memory representations.
Strengths:
The experimental approach is well-motivated, and the use of drift-diffusion modeling to decompose decision components adds analytical value beyond standard RT and accuracy measures. The two new experiments are pre-registered and address important questions. The broader theoretical conclusion - that working memory limits are shaped not only by storage capacity but by which representational formats remain accessible under attentional uncertainty - is an important and timely contribution to the field.
Weaknesses:
The central interpretive claims rely heavily on differences in non-decision time, a parameter that aggregates many processes unrelated to memory retrieval, making it rather difficult to uniquely attribute the observed effects to access or retrieval mechanisms specifically. Additionally, the characterization of the two memory conditions as genuinely perceptual versus semantic warrants further justification, as both may primarily require categorical rather than format-specific knowledge.
-
Reviewer #2 (Public review):
This manuscript aims to characterize how semantic information is prioritized relative to perceptual details in visual working memory. The central claim is that semantic judgements benefit from faster pre‑decisional access (shorter non‑decision time), and that advantages in evidence accumulation emerge under higher cognitive demands (e.g., when items are outside the focus of attention or must be maintained under interference). Based on this, the paper argues that unattended working‑memory contents are reformatted into more abstract, long‑term‑memory‑like semantic representations that remain more readily accessible than fine‑grained perceptual features.
Strengths:
(1) The question is timely and relevant to current research about the format of visual working memory.
(2) Behaviorally, the semantic advantage is …
Reviewer #2 (Public review):
This manuscript aims to characterize how semantic information is prioritized relative to perceptual details in visual working memory. The central claim is that semantic judgements benefit from faster pre‑decisional access (shorter non‑decision time), and that advantages in evidence accumulation emerge under higher cognitive demands (e.g., when items are outside the focus of attention or must be maintained under interference). Based on this, the paper argues that unattended working‑memory contents are reformatted into more abstract, long‑term‑memory‑like semantic representations that remain more readily accessible than fine‑grained perceptual features.
Strengths:
(1) The question is timely and relevant to current research about the format of visual working memory.
(2) Behaviorally, the semantic advantage is carefully documented in many conditions across datasets.
(3) The use of hierarchical drift-diffusion modelling is helpful to decompose the semantic advantage into cognitive processes such as non‑decision time and drift‑rate components.
Weaknesses:
(1) The strong claims about visual working‑memory representation and "long‑term‑memory‑like" formats rest on an indirect inference from decision‑model parameters to representational content, and this link is not convincingly established. Non‑decision time, as implemented here, bundles many things, such as probe processing, cue processing, retrieval/access, and motor preparation, so reduced non‑decision time for semantic probes could reflect easier question reading, simpler response mapping, or more efficient decision preparation rather than a genuine advantage in accessing semantic memory representations. Although the manuscript acknowledges that non‑decision time includes multiple processes, it nonetheless treats this parameter as primary evidence for a retrieval‑stage semantic advantage, which overstates what the data can uniquely support.
(2) The modelling approach is relatively constrained and does not fully address the underdetermination inherent in mapping latent drift-diffusion parameters onto specific psychological mechanisms. The preferred model that allows multiple parameters (non‑decision time, drift rate, threshold) to vary provides only modest improvements in predictive accuracy over simpler models, and several key drift‑rate effects are present only in particular load or lag conditions. As a result, the theoretical interpretation that semantic prioritization primarily reflects faster access and secondarily more efficient accumulation under high demand appears rather post hoc, and alternative accounts focused on generic task efficiency or strategy differences remain plausible.
(3) The operationalization of "semantic" is narrow and largely categorical, focusing on animacy (animal/object) and a perceptual format dimension (photo/drawing), rather than richer semantic or associative relations among items. This makes it difficult to generalize the conclusions to broader claims about semantic structure and its integration into working‑memory representations. Important recent work on how semantic and associative relationships facilitate the formation, maintenance, and retrieval of visual working memory is not adequately integrated into the theoretical framing. Consequently, the discussion tends to generalize from a specific probe structure to a broader semantic prioritization theory without engaging fully with the existing literature on semantic facilitation and neural decoding of working‑memory content.
(4) The paper contrasts its behavioral/model‑based results with prior neural decoding findings, but the comparison is not fair. Neural decoding provides complementary evidence about the content and format of working memory representations, whereas drift-diffusion parameters reflect downstream decision dynamics given a probe. Because the current work does not include any direct representational or neural measure, its conclusions about representational "reformatting" and long‑term‑memory‑like access remain speculative and, in places, feel like a stretch.
(5) Overall, while the data show a semantic advantage in decision‑stage measures and the modelling provides an informative decomposition of this advantage, the manuscript does not fully achieve its stated aim of characterizing the representational format of visual working memory or demonstrating a mechanistic shift toward long‑term‑memory‑like semantic representations. The work primarily informs decision‑process analyses of the conditions under which semantic judgements are faster and more robust, rather than the nature of visual working‑memory representations themselves.
-
Author response:
We thank the reviewers for their constructive and careful assessment of our manuscript. We are encouraged that both reviewers recognised the value of the empirical contribution: the semantic advantage is robust across experiments, the two new experiments are pre-registered, and the drift-diffusion modelling provides an informative decomposition of behavioural performance. At the same time, both reviewers raise an important and convergent point: the manuscript currently places too much interpretive weight on non-decision time and sometimes moves too quickly from decision-model parameters to claims about the representational format of working memory.
We agree that this aspect of the manuscript should be revised. In the next version, we will substantially soften claims about adaptive reformatting and long-term-memory-like …
Author response:
We thank the reviewers for their constructive and careful assessment of our manuscript. We are encouraged that both reviewers recognised the value of the empirical contribution: the semantic advantage is robust across experiments, the two new experiments are pre-registered, and the drift-diffusion modelling provides an informative decomposition of behavioural performance. At the same time, both reviewers raise an important and convergent point: the manuscript currently places too much interpretive weight on non-decision time and sometimes moves too quickly from decision-model parameters to claims about the representational format of working memory.
We agree that this aspect of the manuscript should be revised. In the next version, we will substantially soften claims about adaptive reformatting and long-term-memory-like formats. We will instead frame the central contribution more precisely: semantic-category judgements show a reliable advantage at stages preceding evidence accumulation and this advantage is modulated by attentional prioritisation and interference during the maintenance interval. Our data constrain the dynamics with which different kinds of information become available for WM-guided decisions, but they do not, on their own, provide a direct measure of representational format. This hypothesis should be tested in future experiments.
At the same time, we think the data provide stronger constraints on alternative explanations than the current manuscript makes clear. The reviewers correctly note that non-decision time is not a pure retrieval parameter, as we also note in the discussion. It can include probe encoding, response preparation, motor execution, and other processes. We will therefore avoid more explicitly equating NDT directly with retrieval latency. However, many of the alternatives raised by the reviewers, such as easier question reading or simpler response mapping for semantic probes, predict a relatively fixed semantic–perceptual offset. In our experiments, the probes and response mappings are held constant across attentional conditions, while the semantic NDT advantage changes as a function of whether the relevant item can be prioritised in advance or must be selected/reactivated at test. We will restructure the Results and Discussion to make these condition × feature interactions central to the argument.
We will also clarify the logic of Experiment 1. We agree with Reviewer 1 that a valid retro-cue likely triggers retrieval or reactivation of the cued item. Our original phrasing, which described the valid-cue condition as reducing retrieval demands, was imprecise. The critical manipulation is better described as shifting item prioritisation/retrieval earlier in the trial. Under valid cueing, the relevant item can be prioritised before the probe appears, whereas under neutral cueing, item selection and access must occur after probe onset. We will rewrite this section accordingly.
We will also clarify our operationalisation of semantic and perceptual categories. The present contrast is specifically between semantic category information (animate versus inanimate) and perceptual-format information (photograph versus drawing). We agree that the perceptual judgement is still categorical and does not measure fine-grained perceptual fidelity. We will therefore avoid broad claims about semantic structure or perceptual detail in general. However, as pointed in the manuscript, we believe the contrast remains meaningful: the two dimensions are orthogonal within the same stimuli, and previous work using the same feature space showed the opposite ordering during perception (Linde-Domingo et al., 2019), where perceptual-format information was available before semantic-category information. We will move this argument earlier in the manuscript and present it as converging evidence for dissociable access dynamics, while acknowledging that it does not by itself prove representational format.
In response to the modelling concerns, we will expand the model-validation section. Specifically, we plan to add posterior predictive checks for the reported models, report model comparisons more transparently, clarify when more complex models do or do not provide practically meaningful improvements, and include sensitivity analyses using alternative parameterisations where identifiable.
We will also make several methodological clarifications. First, because the reanalysis of Kerrén et al. (2022) forms a substantial part of the manuscript, we will add a fuller description of the original task in the main text, including how the probed item was indicated at test. Second, we will rewrite the unclear sentence describing pseudo-random stimulus selection in Experiment 1 and add a control analysis testing whether performance differs when the probed item belongs to the majority versus minority category within the trial. Third, we will clarify the stimulus repetition scheme and discuss possible long-term-memory contributions. Importantly, because semantic and perceptual probes are applied to the same items from the same trials, any repetition history or proactive-interference contribution is shared across the two probe types, although we agree that this should be discussed explicitly.
Finally, we will revise the broader theoretical framing. We will remove or substantially qualify claims linking the present data directly to episodic memory and imagery. We will also integrate the recent literature suggested by Reviewer 2 on semantic structure, associative relations, long-term-memory contributions to working memory, and boundary conditions for semantic labelling effects. This will allow us to position the study as one piece of a broader literature on how semantic information influences WM performance, rather than as direct evidence for a general representational reformatting mechanism.
In summary, the revised manuscript will make a narrower but stronger claim: semantic-category information shows a robust pre-accumulation advantage during WM-guided decisions, and this advantage is shaped by attentional prioritisation and interference during maintenance. We will present this as evidence about WM access dynamics and decision components, not as direct evidence that WM representations are transformed into long-term-memory-like formats.
-
