the vision
and memory lab

Effective visualizations not only communicate information about individual datapoints, but also allow people to quickly extract summaries of the data. The visual perception literature has revealed 'ensemble' perception mechanisms that provide a way for people to extract statistical summaries, like the mean position of the datapoints, from visual displays like scatterplots. Prior work has focused on how people extract the mean of only a single, unified cluster of datapoints, and has not tested how accurately people extract summary statistics from more complex displays, and what strategies they use to do so. Scatterplots often contain clustered data, and identifying the global mean by aggregating across clusters can provide valuable insight for identifying trends, comparing groups, and guiding decision-making. The current studies systematically investigate how people make intuitive judgments about the global mean of two spatial clusters in scatterplots. We focus specifically on the case where people must aggregate over a larger main cluster and an 'outlier' cluster with relatively fewer points, and we vary both the number of items in each cluster and the dispersion of clusters. Our results suggest that people qualitatively differ in their perception of the mean, but all groups of participants reliably overweight the outlier group in their intuitive judgments of the mean of the datapoints.

Previous work has shown that semantically meaningful properties of visually presented real-world objects (their color, their state/configuration of their parts/pose, the features that differentiate them from other exemplars of the same category) are stored with a high degree of independence in long-term memory (e.g., are frequently swapped or misbound across objects). But is this feature independence due to the visual representation of the objects, or because of verbal encoding? Semantically meaningful features can also be labeled by distinct words, which can be recombined to produce independent descriptions of real-world object features. Here, we directly test how much of the pattern of feature independence arises from visual vs. verbal encoding. In two experiments, during the study phase we orthogonally varied the match or mismatch of state (e.g., open/closed) and color information between images of objects and their verbal descriptions (Experiment 1) or between images of two exemplars from the same category (Experiment 2). At test, observers had to choose a previously presented image or description in a 4-AFC task. Whereas in Experiment 1 we found quite a small effect of visual-verbal mismatch on memory for images, the effect of mismatch between exemplars in Experiment 2 was dramatic: memory for a feature was reasonably good when it matched between exemplars, but dropped to chance otherwise. Importantly, this effect was observed both for color and object state independently. We conclude that independent, feature-based storage of objects in long-term memory is provided primarily by visual representations with possible minor influences of verbal encoding.

Previous work has shown that visual working memory performance is higher for recognizable and meaningful stimuli relative to meaningless, but physically matched, stimuli (e.g., Asp et al., 2021). Here, we test whether the benefit for meaningful stimuli arises due to active storage in working memory or can, at least in part, be explained by reliance on other more durable memory traces, such as long-term memory. We manipulated meaningfulness using ambiguous Mooney faces (Exp. 1) and objects (Exp. 2) vs. scrambled versions of the same stimuli. Although items were repeated 10x more often in the high interference condition than the low interference condition, we found equivalent objective and subjective effects of meaningfulness at high levels of proactive interference, suggesting that persistent passive long-term memory traces do not play a critical role for the meaningfulness advantage in working memory tasks.

Understanding how people judge their own knowledge and abilities is crucial for everything from education to medical diagnosis. Yet we lack precise models that can predict how confident someone will be in their decisions. Here, we introduce a novel quantitative model that allows us, uniquely, to make predictions about confidence judgments from accuracy alone and to transfer these predictions fully across tasks. We focus on a suite of visual working memory tasks, though our modeling approach applies to a wide range of perception and memory phenomena. Following recent proposals, our approach postulates that generalizable models of metacognition must consider not just a discrepancy between metacognitive processes and behavior, but also people’s latent representations of the environment and its structure. We demonstrate that this model successfully predicts metacognitive judgments in the typical situation where confidence tracks memory performance, and that it goes beyond previous work by making high precision predictions of entire distributions of memory errors as a function of confidence. Moreover, we show that the model predicts, rather than fits, a range of dissociations between memory performance and confidence caused by careful choice of the probes used to test memory and by variations in how people represent the structure of the stimulus space. Our framework offers a straightforward explanation of metacognition, allows prediction in a novel way, integrates recent advancements in computational modeling of metacognition with artificial agents and humans, and sheds new light on how judgments of confidence are constrained by memory organization.

Building unified theories of human perceptual memory requires distinguishing domain-general principles from mechanisms specific to each sensory system. To advance these theories, it is essential to compare memory performance across senses. Here, we created fair comparisons between visual and auditory memory by a) ensuring that stimulus similarity structure, the confusability of remembered items with incorrect alternatives (foils), was matched, and b) considering how stimulus presentation affects memory performance for each system. We used developments in artificial neural networks to select maximally similar and dissimilar foils for both visual and auditory stimuli. Additionally, we modified experimental parameters, predicting that presentation mode (simultaneous, sequential, etc.), and its alignment with the ecological constraints of hearing and vision, would differentially impact auditory versus visual memory. Prior work has shown a visual memory advantage, but this could be due to stimulus and task parameters. We found that maximally dissimilar auditory and visual foils are comparably confusable; however, auditory memory performance degrades more rapidly than visual memory as stimulus similarity increases, revealing a fundamental contrast in how these systems retrieve information. Adjusting stimulus presentation led to findings that auditory memory could be better, worse, or comparable to visual memory. Apparent disparities in memory capacity across vision and hearing may largely reflect differences in sensitivity to similarity structure and task. These findings establish new principles for cross-sensory memory research and demonstrate how experimental design can distort cross-sensory comparisons and undermine efforts to develop theories of memory.

Data visualizations often display averages without underlying data points, aiming to enhance comprehension through visual simplification. Yet the theory that this simplicity improves comprehension remains untested. Using a new drag-and-drop measure, we tested this theory’s most basic prediction: that viewers will locate explicitly plotted, isolated averages more accurately than averages they must estimate from raw data. Remarkably, we found the opposite—accuracy was lower for standard bar or line plots depicting averages than for raw-data plots. This discovery stemmed from two observed phenomena: (1) average-blindness—the mislocation of explicitly marked averages, often placed within bars or along lines rather than at the intended markers; and (2) ensemble-acuity—the accurate estimation of averages from raw data (or ensembles), with variability comparable to confidence intervals and few outright errors. Our findings reveal a paradox of certainty: certainty about the average’s location can obscure it, whereas the uncertainty inherent in raw data can clarify it.

What constrains working memory capacity? Classic theories place visual working memory close to perceptual systems, with fixed limits. Yet, emerging evidence shows that visual working memory capacity is increased for real-world objects compared to simple or abstract stimuli. The present study demonstrates that this memory advantage arises from semantic understanding of real-world objects – contrary to classic perceptual accounts of this cognitive system. Using counterfeit objects generated by generative adversarial networks that match real objects in terms of object form and visual similarity, we show that improvements in behavioral performance and increases in neural delay activity emerge solely for semantically meaningful, real objects. Correlation analyses indicate that subjective familiarity ratings predict memory for real objects, whereas stimulus colourfulness predicts memory for artificial objects, suggesting distinct mechanisms support memory for different stimulus types. Thus, conceptual knowledge exerts strong effects on visual working memory, significantly extending current theories that emphasize low-level perceptual features.

Visual representations that are actively held in mind are often systematically distorted by new visual stimuli, prompting claims that memories are vulnerable to unwanted integration with new perceptual inputs when they recruit shared sensory representations. Here we challenge this vulnerability perspective by showing that individuals rationally adapt the degree to which memory and perception are integrated based on their recent history of being mutually informative. Across three experiments, we show that the exact same pairs of remembered and perceived stimuli are more likely to be perceived as similar and elicit larger memory biases when preceded by other pairs that were similar as well. These adaptive modulations in integration with stimulus history do not depend on corresponding changes in sensory strength, implying a separate, higher-order mechanism beyond sensory recruitment. These findings are consistent with the predictions of normative cognitive models that have successfully characterized biases across visual perception and memory.

Conceptual knowledge structures human memory, with better memory performance for more conceptual diverse information even when it is perceptually similar. To what extent do modern deep neural networks capture the conceptual structure relevant for memory performance? We investigated how human judgments of within-category diversity relate to memorability and to the representational structure of deep neural networks. Participants sorted exemplar images from 498 object categories (from the THINGS database) based on either conceptual similarity (function, purpose, typical context) or perceptual similarity (shape, color, texture). We measured category-level memorability using a recognition memory task with within-category foils. We then tested how memory performance related to human diversity scores and to diversity computed from CLIP, DINOv3, and VGG16 embeddings across multiple layers. Human conceptual and perceptual diversity were correlated but dissociable, with some categories showing high conceptual but low perceptual diversity (e.g., electrical plugs that look similar but function in different countries). Critically, conceptual diversity was the strongest predictor of memorability and adding perceptual diversity or model-based measures did not improve prediction. Across models, correlations with human conceptual and perceptual judgments both increased with layer depth, and all three architectures converged to similarly good predictions at their final layers. However, none matched the predictive power of human conceptual diversity ratings in predicting human memorability. These findings extend prior work linking conceptual distinctiveness to memory by demonstrating that purely local, within-category structure, independent of a category's position in global semantic space, predicts how well individual exemplars are remembered. The gap between human and model-derived diversity estimates suggests that the typical implementations of vision models lack the flexible, context-sensitive feature weighting that humans deploy when organizing objects conceptually, highlighting an important constraint for computational accounts of human visual memory.

Some pictures, videos, faces, voices, and words are reliably remembered better than others, a phenomenon known as memorability. This has been widely interpreted as reflecting a stable, intrinsic property of the stimuli themselves. Here we argue that this consistency is better explained by the relational structure of the stimulus sets used to measure memorability, rather than as an intrinsic property of the items. Using simulations with a global matching model of recognition memory, we show that apparent intrinsic memorability emerges naturally from distinctiveness — how dissimilar an item is from others in a given set of encoded images — without any stimulus-intrinsic mechanism. In behavioral experiments, we demonstrate that manipulating the similarity structure of image sets can invert memorability rankings, making previously memorable images forgettable and vice versa. We also show that features previously identified as intrinsically memorable (such as “body-part related”) are the most representationally distinctive features in standard databases, explaining their memorability. To address these large variations in distinctiveness we created a new image set that reduced variability in representational distinctiveness, and found it significantly diminishes all feature-memorability relationships. These findings suggest that memorability is not a fixed attribute of items themselves, but an emergent property of which items are distinctive within the context, with implications for how we attribute stable psychological properties to stimuli more broadly.

A longstanding question in cognitive science is whether the structure of mental representations — deriving from perceptual and semantic knowledge — can be used to generate a priori quantitative predictions of the specific errors people make in memory. Using both human similarity judgments and an ensemble of modern pretrained deep neural networks trained for general visual tasks, with no memory-task or face-identification training, we measure the representational geometry of face stimuli and simple visual features (e.g., color, shape, orientation, location) independently of any memory experiment. Combined with a simple single-parameter decision rule, these independently measured geometries predict the full distribution of visual working memory errors for those stimuli, including which specific items people are most likely to confuse, without any fitting to the memory data. Human and deep network-derived geometries agree closely, and together they outperform standard mixture model and tuning-function models of memory while using fewer free parameters. The same pipeline, with a fixed modeling procedure, transferred unchanged to thirty new face sets as well as novel tasks and simple stimuli like orientation, shape, color and location, in each case predicting the specific pattern of errors specifically from that set’s unique network-derived geometry. Because the same geometric measurement underlies predictions across different aspects of behavior, the results suggest that representational geometry, measured with modern tools, provides a direct and falsifiable a priori link between internal representations and the errors people make in memory.

Khvostov et al. (Psychonomic Bulletin & Review 32: 2903–2912, 2025) present compelling evidence that observers have explicit access to detailed ensemble feature distributions, challenging the traditional view that only summary statistics are available. Here, we demonstrate that the Target Confusability Competition (TCC) ensemble model (Robinson & Brady, Nature Human Behaviour, 7: 1638–1651, 2023) provides a straightforward process-level account of these results. Without any parameter tuning, the model accurately predicts the observed response patterns across Gaussian, uniform, and bimodal color distributions. This alignment underscores the utility of TCC-ensemble in explaining ensemble perception and highlights the value of similarity-based encoding and integration mechanisms in supporting access to distributional structure.

Visual perception is imperfect. Fortunately, the environment contains an incredible amount of structure that observers can learn to exploit through experience. As a result, representations of the environment that are perceived and stored in memory are often biased towards or away from other stimuli, reference points, and expectations that are acquired across varying timescales. We propose that this myriad of bias phenomena reflects a shared adaptive principle where the brain optimizes its noisy sensory representations to support behavior within its resource constraints. Under this principle, experience is stabilized through the integration of mutually informative stimuli, whilst the discriminability between stimuli is maintained through adaptation. We suggest that this principle emerges naturally from predictive and adaptive coding frameworks that can operate across multiple levels of processing and predict attraction and repulsion even within the same task or stimulus. Viewing these biases as emergent properties of a hierarchical statistical learning system offers new insight into how the brain balances stability and flexibility when shaping our perception and memory of the world.

Understanding how people integrate gist and item-specific information is central to explaining how memory changes over time. We examined how these representations interact in visual memory by integrating a Bayesian framework with representational models of individual item and gist memory to predict gist-based distortions. Participants completed separate tasks measuring gist and item-specific memory, and we used these data with independent measures of stimulus representations to quantify memory fidelity. We substituted these parameters into our model to predict memory errors in a third task where memory for individual items was biased by category-level structure. Our model predicted entire distributions of people’s memory errors, including the magnitude and direction of memory biases, and fine-grained individual differences, such as their skew and variance, at different offsets. Our findings highlight the power of combining normative Bayesian with algorithmic representational modelling approaches to understand how people integrate noisy memory representations at different levels of abstraction.

Understanding 3D representations of spatial information, particularly in naturalistic scenes, remains a significant challenge in vision science. This is largely because of conceptual difficulties in disentangling higher-level 3D information from co-occurring features and cues (e.g., the 3D shape of a scene image is necessarily defined by “low-level” spatial frequency and orientation information). Recent work has employed newer models and analysis techniques that attempt to mitigate these difficulties within a model-comparison framework. For example, one such study reported 3D-surface features were uniquely present in areas OPA, PPA, and MPA/RSC (areas typically referred to as ‘scene-selective’), above and beyond a Gabor-wavelet baseline model. Here, we tested whether these findings generalized to a new stimulus set that, on average, dissociated static Gabor-wavelet baseline features from 3D scene-surface features. Surprisingly, we found evidence that a Gabor-wavelet baseline model—commonly thought of as a “low-level” or “2D” model—better fit voxel responses in areas OPA, PPA and MPA/RSC compared to a model with 3D-surface information. We highlight that this difference in results could be due to differences in the baseline conditions used across studies. These findings emphasize that much of the information in “scene-selective” regions—potentially even information about 3D surfaces—may be in the form of spatial frequency and orientation information often considered 2D or low-level. Disentangling lower-level and higher-level visual information is a continuing fundamental challenge for model-comparison approaches in visual cognition, and it motivates future work investigating which visual features could cue higher-level properties in our real-world visual experience—both within and beyond current model comparison frameworks.

Published claims should be reproducible, yielding the same result when the same analysis is applied to the same data1,2. Here we assess reproducibility in a stratified random sample of 600 papers published from 2009 to 2018 in 62 journals spanning the social and behavioural sciences. The authors of 144 (24.0%, 95% confidence interval (CI) = 20.8–27.6%) papers made data available to assess reproducibility and, for 38 others, we obtained source data to reconstruct the dataset. We assessed 143 out of the 182 available datasets and found that 76.6 (53.6%, 95% CI = 45.8–60.7%) papers were rated as precisely reproducible and 105.0 (73.5%, 95% CI = 66.4–80.0%) were rated as at least approximately reproducible (within 15% of the original effects or within 0.05 of original P values) after inverse weighting each of the 551 claims by the number of claims per paper. We observed higher reproducibility for papers from political science and economics compared with other fields, for more recent papers compared with older papers and for papers from journals that require data sharing. Implementation of measures to verify that research is reproducible is needed to support trustworthiness in the complex enterprise of knowledge production3,4.

Pursuing replicability— independent evidence for previous claims — is important for creating generalizable knowledge1,2. Here we attempted replications of 274 claims of positive results from 164 quantitative papers published from 2009 to 2018 in 54 journals in the social and behavioural sciences. Replications were high powered on average to detect the original effect size (median of 99.6%), used original materials when relevant and available, and were peer reviewed in advance through a standardized internal protocol. Replications showed statistically significant results in the original pattern for 151 of 274 claims (55.1% (95% confidence interval (CI) 49.2–60.9%)) and for 80.8 of 164 papers (49.3% (95% CI 43.8–54.7%)), weighed for replicating multiple claims per paper. We observed modest variation in replication rates across disciplines (42.5–63.1%), although some estimates had high uncertainty. The median Pearson’s r effect size was 0.25 (95% CI 0.21–0.27) for original studies and 0.10 (95% CI 0.09–0.13) for replication studies, an 82.4% (95% CI 67.8–88.2%) reduction in shared variance. Thirteen methods for evaluating replication success provided estimates ranging from 28.6% to 74.8% (median of 49.3%). Some decline in effect size and significance is expected based on power to detect original effects and regression to the mean because we replicated only positive results. We observe that challenges for replicability extend across social–behavioural sciences, illustrating the importance of identifying conditions that promote or inhibit replicability3,4.

Visual working memory is a core cognitive function that allows active storage of task-relevant visual information. Contrary to the common assumption that the capacity of this system is fixed with respect to a single feature dimension, recent research has shown that working memory performance for a simple visual feature—color—is improved when this feature is encoded as part of a real-world object relative to an unrecognizable scrambled object. Using EEG (n = 24), we here demonstrate that this performance benefit is supported by increased neural engagement during the retention period, as indexed by enlarged contralateral delay activity during maintenance. Furthermore, the pattern of neural activity across parietal-occipital electrodes was more stable across time, suggesting that real-world objects may support more robust memory representations. Finally, we report a novel fronto-central ERP that distinguishes between real-world objects and scrambled objects during encoding and maintenance processes. Overall, our results demonstrate that active visual working memory capacity for simple features is not fixed but can expand depending on what context these features are encoded in.

What happens when there are many different objects that must be maintained simultaneously in visual working memory? Prominent models focus largely on individual objects in describing memory limits: e.g., claims that we can remember only a fixed number of items, or that a single resource limit describes the cost of holding additional items. While some acknowledge interactions between items, these interactions are not given a prominent role in most models. Here we show that instead of items being represented independently, the visual display is always compressed by utilizing clusters of items and the gist of the display. We reanalyze data from 11 previously available datasets (comprising 137,986 trials in total). We find strong evidence for non-independent representations, including chunking and use of the ‘gist’ of the display, which is present in nearly every study at every set size. We then present a model for understanding this chunking and the resulting systematic biases (e.g., items attracting and repelling one another) based on psychophysical similarity. Overall, this work provides strong evidence that in order to understand visual working memory, we need to consider how our memory system takes advantage of the relationship between items. This is in contrast to the way the majority of the field studies visual working memory and suggests a major paradigm shift is required to think of memory in terms of clusters, chunks, and gist rather than in terms of independent items.

Across many studies, verbal labels have been shown to significantly affect perceptual and cognitive processes. Here, we ask whether verbal labels can also improve visual working memory performance by making objects more recognizable. In a series of experiments, participants were asked to remember visual details of unrecognizable shapes derived from real-world objects. Participants were either provided with category labels prior to stimulus presentation or not. We reasoned that the labels could aid object recognition, which would then help maintain them in working memory based on recent work showing that meaningful, recognizable objects are better remembered than arbitrary visual shapes. Contrary to our hypothesis, we found no reliable increase in visual working memory performance when participants were given verbal labels. This pattern of results persisted across various experimental manipulations, including different methods of distorting the objects, testing label memory, and using within- or between-category stimulus sets. Overall, our results indicate that category labels did not readily enhance working memory performance for obfuscated shapes that are difficult to recognize, suggesting that improving visual working memory for meaningful stimuli may depend on how the visual system reorganizes incoming visual information perceptually rather than associating visual inputs with verbal knowledge. Thus, verbal labels do not seem to have as broad and general effects on cognitive functioning as previously assumed. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

The visual system takes advantage of redundancy in the world by extracting summary statistics, a phenomenon known as ensemble perception. Ensemble representations are formed for low-level features like orientation and size and high-level features such as facial identity and expression. Whereas recent research has shown that the visual system forms intact ensemble representations even when faces are partially occluded via solid bars, how ensemble perception is impacted with the addition of naturalistic objects such as face masks or sunglasses is largely unknown. To investigate this, we conducted a series of experiments using continuous report tasks in which faces (either varying in identity or expression) were partially occluded with a surgical mask or sunglasses and participants had to report the average face using a face wheel. We found evidence that participants could still accurately extract the average even when a significant portion of it was occluded with either face masks or sunglasses. In a second experiment, however, we found performance was worse when the face wheel was variable trial to trial. Thus part of the preservation of performance in occlusion arises from the visual system learning the features of the particular face wheel being used. Overall, our results suggest that the visual system is able to establish robust ensemble representations for faces with naturalistic occlusions, but that robustness appears to be supported at least partially by learning information about the particular features that are informative for a given set of faces.

Data visualizations often display averages without raw data to simplify communication and enhance understanding, especially for lay audiences. However, the theory that such simplification improves understanding remains untested. Here, we test this theory’s most basic prediction—that at minimum, the average itself is conveyed better by plotted averages than by plotted raw data. Remarkably, we find the opposite: under a wide range of conditions, overall accuracy of average estimation is higher with raw data. This is due to frequent, severe misinterpretations of both bar and line graphs depicting averages. In contrast, raw data yields some variability but few outright errors; notably, the observed variability is comparable to the uncertainty captured by confidence intervals. We conclude that plotted raw data provides valuable context that helps prevent misunderstandings of the average. Our findings challenge the notion that plotted averages alone yield enhanced understanding and emphasize the value of raw data in communicating evidence.

Working memory is crucial for short-term information processing, but its limited capacity means items are not represented with perfect fidelity to the external world. Many systematic patterns of error exist that are thought to be telling of the underlying mechanisms that process and maintain information in memory. Here, we suggest that the processes governing some of these patterns of errors are interrelated and highly individual. Specifically, we look at how perceptual structure relates to stimulus-specific biases in color and further explore the possible implication of this connection for contextual biases like serial dependence and repulsion between concurrently presented items. In Experiment 1, using a novel within-participant serial reproduction method, we reveal reliable attractors in color space across individuals, as well as individual differences that significantly influence these stimulus-specific biases. Simulations based on an independently measured perceptual structure of the stimulus space reproduce the group-level differences but do not capture the observed individual variation. In Experiment 3, we investigate how contextual biases-serial dependence when remembering one item and repulsion when remembering two items-interact with stimulus-specific properties. We identify color-specific properties of these contextual biases, as well as individual differences in the magnitude, direction, and stimulus-specific nature of these biases. We argue that because stimulus-specific biases are connected to perceptual structure, this same latent structure may impact contextual biases. Overall, we show a strong connection between stimulus-specific biases, contextual biases, and perceptual structure, as well as rich individual differences in these biases. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Research on best practices in theory assessment highlights that testing theories is challenging because they inherit a new set of assumptions as soon as they are linked to a specific methodology. In this article, we integrate and build on this work by demonstrating the breadth of these challenges. We show that tracking auxiliary assumptions is difficult because they are made at different stages of theory testing and at multiple levels of a theory. We focus on these issues in a reanalysis of a seminal study and its replications, both of which use a simple working-memory paradigm and a mainstream computational modeling approach. These studies provide the main evidence for “all-or-none” recognition models of visual working memory and are still used as the basis for how to measure performance in popular visual working-memory tasks. In our reanalysis, we find that core practical auxiliary assumptions were unchecked and violated; the original model comparison metrics and data were not diagnostic in several experiments. Furthermore, we find that models were not matched on “theory general” auxiliary assumptions, meaning that the set of tested models was restricted, and not matched in theoretical scope. After testing these auxiliary assumptions and identifying diagnostic testing conditions, we find evidence for the opposite conclusion. That is, continuous resource models outperform all-or-none models. Together, our work demonstrates why tracking and testing auxiliary assumptions remains a fundamental challenge, even in prominent studies led by careful, computationally minded researchers. Our work also serves as a conceptual guide on how to identify and test the gamut of auxiliary assumptions in theory assessment, and we discuss these ideas in the context of contemporary approaches to scientific discovery.

Visual working memory is traditionally studied using abstract, meaningless stimuli. Although studies using such simplified stimuli have been insightful in understanding the mechanisms of visual working memory, they also potentially limit our ability to understand how people encode and store conceptually rich and meaningful stimuli in the real world. Recent studies have demonstrated that meaningful and familiar visual stimuli that connect to existing knowledge are better remembered than abstract colors or shapes, indicating that meaning can unlock additional working memory capacity. These findings challenge current models of visual working memory and suggest that its capacity is not fixed but depends on the type of information that is being remembered and, in particular, how that information connects to preexisting knowledge.

The ability to sustain internal representations of the sensory environment beyond immediate perception is a fundamental requirement of cognitive processing. In recent years, debates regarding the capacity and fidelity of the working memory (WM) system have advanced our understanding of the nature of these representations. In particular, there is growing recognition that WM representations are not merely imperfect copies of a perceived object or event. New experimental tools have revealed that observers possess richer information about the uncertainty in their memories and take advantage of environmental regularities to use limited memory resources optimally. Meanwhile, computational models of visuospatial WM formulated at different levels of implementation have converged on common principles relating capacity to variability and uncertainty. Here we review recent research on human WM from a computational perspective, including the neural mechanisms that support it.

The capacity of visual working and visual long-term memory plays a critical role in theories of cognitive architecture and the relationship between memory and other cognitive systems. Here, we argue that before asking the question of how capacity varies across different stimuli or what the upper bound of capacity is for a given memory system, it is necessary to establish a methodology that allows a fair comparison between distinct stimulus sets and conditions. One of the most important factors determining performance in a memory task is target/foil dissimilarity. We argue that only by maximizing the dissimilarity of the target and foil in each stimulus set can we provide a fair basis for memory comparisons between stimuli. In the current work we focus on a way to pick such foils objectively for complex, meaningful real-world objects by using deep convolutional neural networks, and we validate this using both memory tests and similarity metrics. Using this method, we then provide evidence that there is a greater capacity for real-world objects relative to simple colors in visual working memory; critically, we also show that this difference can be reduced or eliminated when non-comparable foils are used, potentially explaining why previous work has not always found such a difference. Our study thus demonstrates that working memory capacity depends on the type of information that is remembered and that assessing capacity depends critically on foil dissimilarity, especially when comparing memory performance and other cognitive systems across different stimulus sets.

Previous studies have found that real-world objects’ identities are better remembered than simple features like colored circles, and this effect is particularly pronounced when these stimuli are encoded one by one in a serial, item-based way. Recent work has also demonstrated that memory for simple features like color is improved if these colors are part of real-world objects, suggesting that meaningful objects can serve as a robust memory scaffold for their associated low-level features. However, it is unclear whether the improved color memory that arises from the colors appearing on real-world objects is affected by encoding format, in particular whether items are encoded sequentially or simultaneously. We test this using randomly colored silhouettes of recognizable versus unrecognizable scrambled objects that offer a uniquely controlled set of stimuli to test color working memory of meaningful versus non-meaningful objects. Participants were presented with four stimuli (silhouettes of objects or scrambled shapes) simultaneously or sequentially. After a short delay, they reported either which colors or which shapes they saw in a two-alternative forced-choice task. We replicated previous findings that meaningful stimuli boost working memory performance for colors (Exp. 1). We found that when participants remembered the colors (Exp. 2) there was no difference in performance across the two encoding formats. However, when participants remembered the shapes and thus identity of the objects (Exp. 3), sequential presentation resulted in better performance than simultaneous presentation. Overall, these results show that different encoding formats can flexibly impact visual working memory depending on what the memory-relevant feature is.

Both in everyday life and in memory research, people tend to think that items are ‘held’ in mind, in the same way that a real-world object can be held in one’s hand. Inspired by this metaphor, traditional work on visual working memory and visual long-term memory focuses on understanding how many objects are remembered or forgotten, or held or lost, in particular circumstances. By contrast, newer computational and empirical work on visual memory focuses on the role of noise in memory representations — in which memories are thought to vary continually in ‘strength’ or ‘precision’ — as well as the role of the visual hierarchy and priors in structuring memory. In this Review, we merge these contemporary theories and evidence. We describe how fundamentally noisy memory representations are instantiated at different levels of the visual hierarchy and support both visual working memory and long-term memory. We also discuss how thinking of memory in this way can direct further research and illuminate the nature of cognitive function more broadly. Sections

"Dogs" are connected to "cats" in our minds, and "backyard" to "outdoors." Does the structure of this semantic knowledge differ across people? Network-based approaches are a popular representational scheme for thinking about how relations between different concepts are organized. Recent research uses graph theoretic analyses to examine individual differences in semantic networks for simple concepts and how they relate to other higher-level cognitive processes, such as creativity. However, it remains ambiguous whether individual differences captured via network analyses reflect true differences in measures of the structure of semantic knowledge, or differences in how people strategically approach semantic relatedness tasks. To test this, we examine the reliability of local and global metrics of semantic networks for simple concepts across different semantic relatedness tasks. In four experiments, we find that both weighted and unweighted graph theoretic representations reliably capture individual differences in local measures of semantic networks (e.g., how related pot is to pan versus lion). In contrast, we find that metrics of global structural properties of semantic networks, such as the average clustering coefficient and shortest path length, are less robust across tasks and may not provide reliable individual difference measures of how people represent simple concepts. We discuss the implications of these results and offer recommendations for researchers who seek to apply graph theoretic analyses in the study of individual differences in semantic memory.

When asked to remember a color, do people remember a point estimate (e.g., a particular shade of red), a point estimate plus an uncertainty estimate, or are memory representations rich probabilistic distributions over feature space? We asked participants to report the color of a circle held in working memory. Rather than collecting a single report per trial, we had participants place multiple bets to create trialwise uncertainty distributions. Bet dispersion correlated with performance, indicating that internal uncertainty guided bet placement. While the first bet was on average the most precisely placed, the later bets systematically shifted the distribution closer to the target, resulting in asymmetrical distributions about the first bet. This resulted in memory performance improvements when averaging across bets, and overall suggests that memory representations contain more information than can be conveyed by a single response. The later bets contained target information even when the first response would generally be classified as a guess or report of an incorrect item, suggesting that such failures are not all-or-none. This paradigm provides multiple pieces of evidence that memory representations are rich and probabilistic. Crucially, standard discrete response paradigms underestimate the amount of information in memory representations.

In many decision tasks, we have a set of alternative choices and are faced with the problem of how to use our latent beliefs and preferences about each alternative to make a single choice. Cognitive and decision models typically presume that beliefs and preferences are distilled to a scalar latent strength for each alternative, but it is also critical to model how people use these latent strengths to choose a single alternative. Most models follow one of two traditions to establish this link. Modern psychophysics and memory researchers make use of signal detection theory, assuming that latent strengths are perturbed by noise, and the highest resulting signal is selected. By contrast, many modern decision theoretic modeling and machine learning approaches use the softmax function (which is based on Luce’s choice axiom; Luce, 1959) to give some weight to non-maximal-strength alternatives. Despite the prominence of these two theories of choice, current approaches rarely address the connection between them, and the choice of one or the other appears more motivated by the tradition in the relevant literature than by theoretical or empirical reasons to prefer one theory to the other. The goal of the current work is to revisit this topic by elucidating which of these two models provides a better characterization of latent processes in m-alternative decision tasks, with a particular focus on memory tasks. In a set of visual memory experiments, we show that, within the same experimental design, the softmax parameter β varies across m-alternatives, whereas the parameter d′ of the signal-detection model is stable. Together, our findings indicate that replacing softmax with signal-detection link models would yield more generalizable predictions across changes in task structure. More ambitiously, the invariance of signal detection model parameters across different tasks suggests that the parametric assumptions of these models may be more than just a mathematical convenience, but reflect something real about human decision-making.

Does sensory information reach conscious awareness in a discrete, all-or-nothing manner or a gradual, continuous manner? To answer this question, we examined behavioral performance across four different paradigms that manipulate visual awareness: the attentional blink, backward masking, the Sperling iconic memory paradigm, and retro-cuing. We then asked how well we could account for participants’ ( N = 112 adults) behavior using a signal detection framework that factors in psychophysical scaling to model participants’ responses along a single continuum. We found that this model easily accounted for the data from each of these diverse paradigms. Moreover, we reanalyzed the data from prior studies that had posited a discrete view of perceptual awareness and found that our continuous signal detection model outperformed the models that had been used to support an all-or-nothing view of consciousness. This set of data is consistent with the idea that conscious awareness occurs along a graded continuum.

Ensemble perception is a process by which we summarize complex scenes. Despite the importance of ensemble perception to everyday cognition, there are few computational models that provide a formal account of this process. Here we develop and test a model in which ensemble representations reflect the global sum of activation signals across all individual items. We leverage this set of minimal assumptions to formally connect a model of memory for individual items to ensembles. We compare our ensemble model against a set of alternative models in five experiments. Our approach uses performance on a visual memory task for individual items to generate zero-free-parameter predictions of interindividual and intraindividual differences in performance on an ensemble continuous-report task. Our top-down modelling approach formally unifies models of memory for individual items and ensembles and opens a venue for building and comparing models of distinct memory processes and representations. Robinson and Brady present a computational model of ensemble perception as the global sum of feature activations of individual items.

Abstract. Purpose Hindsight bias—where people falsely believe they can accurately predict something once they know about it—is a pervasive decision-making phenomenon, including in the interpretation of radiological images. Evidence suggests it is not only a decision-making phenomenon but also a visual perception one, where prior information about an image enhances our visual perception of the contents of that image. The current experiment investigates to what extent expert radiologists perceive mammograms with visual abnormalities differently when they know what the abnormality is (a visual hindsight bias), above and beyond being biased at a decision level. Approach N = 40 experienced mammography readers were presented with a series of unilateral abnormal mammograms. After each case, they were asked to rate their confidence on a 6-point scale that ranged from confident mass to confident calcification. We used the random image structure evolution method, where the images repeated in an unpredictable order and with varied noise, to ensure any biases were visual, not cognitive. Results Radiologists who first saw an original image with no noise were more accurate in the max noise level condition [area under the curve ( AUC ) = 0.60] than those who first saw the degraded images (AUC = 0.55; difference: p = 0.005), suggesting that radiologists’ visual perception of medical images is enhanced by prior visual experience with the abnormality. Conclusions Overall, these results provide evidence that expert radiologists experience not only decision level but also visual hindsight bias, and have potential implications for negligence lawsuits.

Prominent theories of visual working memory postulate that the capacity to maintain a particular visual feature is fixed. In contrast to these theories, recent studies have demonstrated that meaningful objects are better remembered than simple, nonmeaningful stimuli. Here, we tested whether this is solely because meaningful stimuli can recruit additional features—and thus more storage capacity—or whether simple visual features that are not themselves meaningful can also benefit from being part of a meaningful object. Across five experiments (30 young adults each), we demonstrated that visual working memory capacity for color is greater when colors are part of recognizable real-world objects compared with unrecognizable objects. Our results indicate that meaningful stimuli provide a potent scaffold to help maintain simple visual feature information, possibly because they effectively increase the objects’ distinctiveness from each other and reduce interference.

While most visual working memory studies use static stimuli with unchanging features, objects in the real world are often dynamic, introducing significant differences in the surface feature information hitting the retina from the same object over time (e.g., changes in orientation, lighting, shadows). Previous research on dynamic stimuli has shown that change detection is improved if objects obey rules of physical motion, but it is unclear how memory for visual features interacts with object motion. In the current study, we investigated whether object motion facilitates greater temporal integration of continuously changing surface feature information. In a series of experiments, participants were asked to report the final color of continuously changing colored dots that were either moving or stationary on the screen. We found that the reported colors “lagged behind” the physical states of the dots when they were in motion. We also observed that the precision of memory responses was significantly higher for stimuli in the moving condition compared to the stationary condition. Together, these findings suggest that memory representation is improved – but lagged – for moving objects, consistent with the idea that object motion may facilitate integration of object information over longer intervals.

Working memory is a reconstructive process that requires integrating multiple hierarchical representations of objects. This hierarchical reconstruction allows us to overcome perceptual uncertainty and limited cognitive capacity but yields systematic biases in working memory as individual items are influenced by the ensemble statistics of the scene, or of their particular group. Given the importance of the hierarchical encoding of a display, we aim to characterize what structures people use to encode visual scenes using a nonparametric data-driven approach. In Experiment 1, we examine visuospatial memory for locations by asking participants to recall the locations of objects in a serial reproduction task. We show that people report items in a more compact structure than they initially were and organize them into clustered spatial groups. In Experiment 2, we explicitly introduce discrete color groups, allowing us to test whether the color feature governs the spatial grouping. We find that the spatial structures were color-contingent. By analyzing color groups, we circumvent the grouping uncertainty in Experiment 1 and further reveal that people compress color groups into collinear structures with similar orientations and equidistant spacing. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

We argue that critical areas of memory research rely on problematic measurement practices and provide concrete suggestions to improve the situation. In particular, we highlight the prevalence of memory studies that use tasks (like the “old/new” task: “have you seen this item before? yes/no”) where quantifying performance is deeply dependent on counterfactual reasoning that depends on the (unknowable) distribution of underlying memory signals. As a result of this difficulty, different literatures in memory research (e.g., visual working memory, eyewitness identification, picture memory, etc.) have settled on a variety of fundamentally different metrics to get performance measures from such tasks (e.g., A′ , corrected hit rate, percent correct, d′ , diagnosticity ratios, K values, etc.), even though these metrics make different, contradictory assumptions about the distribution of latent memory signals, and even though all of their assumptions are frequently incorrect. We suggest that in order for the psychology and neuroscience of memory to become a more cumulative, theory-driven science, more attention must be given to measurement issues. We make a concrete suggestion: The default memory task for those simply interested in performance should change from old/new (“did you see this item’?”) to two-alternative forced-choice (“which of these two items did you see?”). In situations where old/new variants are preferred (e.g., eyewitness identification; theoretical investigations of the nature of memory signals), receiver operating characteristic (ROC) analysis should be performed rather than a binary old/new task.

Visual working memory is highly limited, and its capacity is tied to many indices of cognitive function. For this reason, there is much interest in understanding its architecture and the sources of its limited capacity. As part of this research effort, researchers often attempt to decompose visual working memory errors into different kinds of errors, with different origins. One of the most common kinds of memory error is referred to as a “swap,” where people report a value that closely resembles an item that was not probed (e.g., an incorrect, non-target item). This is typically assumed to reflect confusions, like location binding errors, which result in the wrong item being reported. Capturing swap rates reliably and validly is of great importance because it permits researchers to accurately decompose different sources of memory errors and elucidate the processes that give rise to them. Here, we ask whether different visual working memory models yield robust and consistent estimates of swap rates. This is a major gap in the literature because in both empirical and modeling work, researchers measure swaps without motivating their choice of swap model. Therefore, we use extensive parameter recovery simulations with three mainstream swap models to demonstrate how the choice of measurement model can result in very large differences in estimated swap rates. We find that these choices can have major implications for how swap rates are estimated to change across conditions. In particular, each of the three models we consider can lead to differential quantitative and qualitative interpretations of the data. Our work serves as a cautionary note to researchers as well as a guide for model-based measurement of visual working memory processes.

Working memory is thought to have a highly limited storage capacity, which constrains many cognitive abilities including fluid intelligence. Here we show that working memory is limited not only by storage capacity but also by failures to retrieve or make decisions about items that were successfully maintained. Participants memorized an array of circles and then reported a probed item’s color. Many responses were far from the true color, which is often interpreted as a failure to store the item. However, when participants were then given two choices and had to choose the correct color, performance was well above chance regardless of participants’ error on the first response. These results demonstrate an important role for retrieval and decision stages in working memory: sometimes items that are successfully stored are not successfully reported. Moreover, this data highlight that memory capacity is underestimated by traditional measures.

Change detection tasks are commonly used to measure and understand the nature of visual working memory capacity. Across three experiments, we examine whether the nature of the memory signals used to perform change detection are continuous or all-or-none and consider the implications for proper measurement of performance. In Experiment 1, we find evidence from confidence reports that visual working memory is continuous in strength, with strong support for an equal variance signal detection model with no guesses or lapses. Experiments 2 and 3 test an implication of this, which is that K should confound response criteria and memory. We found K values increased by roughly 30% when criteria are shifted despite no change in the underlying memory signals. Overall, our data call into question a large body of work using threshold measures, like K, to analyze change detection data. This metric confounds response bias with memory performance and is inconsistent with the vast majority of visual working memory models, which propose variations in precision or strength are present in working memory. Instead, our data indicate an equal variance signal detection model (and thus, d’)—without need for lapses or guesses—is sufficient to explain change detection performance.

Social interactions are dynamic and unfold over time. To make sense of social interactions, people must aggregate sequential information into summary, global evaluations. But how do people do this? Here, to address this question, we conducted nine studies (N = 1,583) using a diverse set of stimuli. Our focus was a central aspect of social interaction—namely, the evaluation of others’ emotional responses. The results suggest that when aggregating sequences of images and videos expressing varying degrees of emotion, perceivers overestimate the sequence’s average emotional intensity. This tendency for overestimation is driven by stronger memory of more emotional expressions. A computational model supports this account and shows that amplification cannot be explained only by nonlinear perception of individual exemplars. Our results demonstrate an amplification effect in the perception of sequential emotional information, which may have implications for the many types of social interactions that involve repeated emotion estimation. Goldenberg et al. show that we tend to overestimate the average intensity of a sequence of emotional expressions and that this is caused by increased memory for stronger expressions.

When holding multiple items in visual working memory, representations of individual items are often attracted to, or repelled from, each other. While empirically well-established, existing frameworks do not account for both types of distortions, which appear to be in opposition. Here, we demonstrate that both types of memory distortion may confer functional benefits under different circumstances. When there are many items to remember and subjects are near their capacity to accurately remember each item individually, memories for each item become more similar (attraction). However, when remembering smaller sets of highly similar but discernible items, memory for each item becomes more distinct (repulsion), possibly to support better discrimination. Importantly, this repulsion grows stronger with longer delays, suggesting that it dynamically evolves in memory and is not just a differentiation process that occurs during encoding. Furthermore, both attraction and repulsion occur even in tasks designed to mitigate response bias concerns, suggesting they are genuine changes in memory representations. Together, these results are in line with the theory that attraction biases act to stabilize memory signals by capitalizing on information about an entire group of items, whereas repulsion biases reflect a tradeoff between maintaining accurate but distinct representations. Both biases suggest that human memory systems may sacrifice veridical representations in favor of representations that better support specific behavioral goals.

Items that are held in visual working memory can guide attention toward matching features in the environment. Predominant theories propose that to guide attention, a memory item must be internally prioritized and given a special template status, which builds on the assumption that there are qualitatively distinct states in working memory. Here, we propose that no distinct states in working memory are necessary to explain why some items guide attention and others do not. Instead, we propose variations in attentional guidance arise because individual memories naturally vary in their representational fidelity, and only highly accurate memories automatically guide attention. Across a series of experiments and a simulation we show that (a) items in working memory vary naturally in representational fidelity; (b) attention is guided by all well-represented items, though frequently only one item is represented well enough to guide; and (c) no special working memory state for prioritized items is necessary to explain guidance. These findings challenge current models of attentional guidance and working memory and instead support a simpler account for how working memory and attention interact: Only the representational fidelity of memories, which varies naturally between items, determines whether and how strongly a memory representation guides attention. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Visual working memory is a capacity-limited cognitive system used to actively store and manipulate visual information. Visual working memory capacity is not fixed, but varies by stimulus type: Stimuli that are more meaningful are better remembered. In the current work, we investigate what conditions lead to the strongest benefits for meaningful stimuli. We propose that in some situations participants may try to encode the entire display holistically (i.e., in a quick "snapshot"). This may lead them to treat objects as simply meaningless, colored "blobs", rather than individually and in a high-level way, which could reduce benefits of meaningful stimuli. In a series of experiments, we directly test whether real-world objects, colors, perceptually matched less-meaningful objects, and fully scrambled objects benefit from deeper processing. We systematically vary the presentation format of stimuli at encoding to be either simultaneous-encouraging a parallel, "take-a-quick-snapshot" strategy-or present the stimuli sequentially, promoting a serial, each-item-at-once strategy. We find large advantages for meaningful objects in all conditions, but find that real-world objects-and to a lesser degree lightly scrambled, still meaningful versions of the objects-benefit from the sequential encoding and thus deeper, focused-on-individual-items processing, while colors do not. Our results suggest single-feature objects may be an outlier in their affordance of parallel, quick processing, and that in more realistic memory situations, visual working memory likely relies upon representations resulting from in-depth processing of objects (e.g., in higher-level visual areas) rather than solely being represented in terms of their low-level features. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Data can be visually represented using visual channels like position, length or luminance. An existing ranking of these visual channels is based on how accurately participants could report the ratio between two depicted values. There is an assumption that this ranking should hold for different tasks and for different numbers of marks. However, there is surprisingly little existing work that tests this assumption, especially given that visually computing ratios is relatively unimportant in real-world visualizations, compared to seeing, remembering, and comparing trends and motifs, across displays that almost universally depict more than two values. To simulate the information extracted from a glance at a visualization, we instead asked participants to immediately reproduce a set of values from memory after they were shown the visualization. These values could be shown in a bar graph (position (bar)), line graph (position (line)), heat map (luminance), bubble chart (area), misaligned bar graph (length), or 'wind map' (angle). With a Bayesian multilevel modeling approach, we show how the rank positions of visual channels shift across different numbers of marks (2, 4 or 8) and for bias, precision, and error measures. The ranking did not hold, even for reproductions of only 2 marks, and the new probabilistic ranking was highly inconsistent for reproductions of different numbers of marks. Other factors besides channel choice had an order of magnitude more influence on performance, such as the number of values in the series (e.g., more marks led to larger errors), or the value of each mark (e.g., small values were systematically overestimated). Every visual channel was worse for displays with 8 marks than 4, consistent with established limits on visual memory. These results point to the need for a body of empirical studies that move beyond two-value ratio judgments as a baseline for reliably ranking the quality of a visual channel, including testing new tasks (detection of trends or motifs), timescales (immediate computation, or later comparison), and the number of values (from a handful, to thousands).

Extant research has shown that previously acquired categorical knowledge affects recognition memory, and that differences in category learning strategies impact classification accuracy. However, it is unknown whether different learning strategies also have downstream effects on subsequent recognition memory. The present study investigates the effect of two unidimensional rule-based category learning strategies – learning (a) with or without explicit instruction, and (b) with or without supervision – on subsequent recognition memory. Our findings suggest that acquiring categorical knowledge increased both hits (Experiments 1 and 2) and false-alarms (Experiment 1) for category-congruent items regardless of the particular strategy employed in initially learning these categories. There were, however, small processing speed advantages during recognition memory for both explicit instruction and supervised practice relative to neither (Experiment 2). We discuss these findings in the context of how prior knowledge influences recognition memory, and in relation to similar findings showing schematic effects on episodic memory.

“Similarity” is often thought to dictate memory errors. For example, in visual memory, memory judgements of lures are related to their psychophysical similarity to targets: an approximately exponential function in stimulus space (Schurgin et al. 2020). However, similarity is ill-defined for more complex stimuli, and memory errors seem to depend on all the remembered items, not just pairwise similarity. Such effects can be captured by a model that views similarity as a byproduct of Bayesian generalization (Tenenbaum & Griffiths, 2001). Here we ask whether the propensity of people to generalize from a set to an item predicts memory errors to that item. We use the “number game” generalization task to collect human judgements about set membership for symbolic numbers and show that memory errors for numbers are consistent with these generalization judgements rather than pairwise similarity. These results suggest that generalization propensity, rather than “similarity”, drives memory errors.

Chunks allow us to use long-term knowledge to efficiently represent the world in working memory. Most views of chunking assume that when we use chunks, this results in the loss of specific perceptual details, since it is presumed the contents of chunks are decoded from long-term memory rather than reflecting the exact details of the item that was presented. However, in two experiments, we find that in situations where participants make use of chunks to improve visual working memory, access to instance-specific perceptual detail (that cannot be retrieved from long-term memory) increased, rather than decreased. This supports an alternative view: that chunks facilitate the encoding and retention into memory of perceptual details as part of structured, hierarchical memories, rather than serving as mere “content-free” pointers. It also provides a strong contrast to accounts in which working memory capacity is assumed to be exhaustively described by the number of chunks remembered.

Memories are encoded in a manner that depends on our knowledge and expectations (“schemas”). Consistent with this, expertise tends to improve memory: Experts have elaborated schemas in their domains of expertise, allowing them to efficiently represent information in this domain (e.g., chess experts have enhanced memory for realistic chess layouts). On the other hand, in most situations, people tend to remember abnormal or surprising items best—those that are also rare or out-of-the-ordinary occurrences (e.g., surprising—but not random—chess board configurations). This occurs, in part, because such images are distinctive relative to other images. In the current work, we ask how these factors interact in a particularly interesting case—the domain of radiology, where experts actively search for abnormalities. Abnormality in mammograms is typically focal but can be perceived in the global “gist” of the image. We ask whether, relative to novices, expert radiologists show improved memory for mammograms. We also test for any additional advantage for abnormal mammograms that can be thought of as unexpected or rare stimuli in screening. We find that experts have enhanced memory for focally abnormal images relative to normal images. However, radiologists showed no memory benefit for images of the breast that were not focally abnormal, but were only abnormal in their gist. Our results speak to the role of schemas and abnormality in expertise; the necessity for spatially localized abnormalities versus abnormalities in the gist in enhancing memory; and the nature of memory and decision-making in radiologists.

When storing multiple objects in visual working memory, observers sometimes misattribute perceived features to incorrect locations or objects. These misattributions are called binding errors (or swaps) and have been previously demonstrated mostly in simple objects whose features are easy to encode independently and arbitrarily chosen, like colors and orientations. Here, we tested whether similar swaps can occur with real-world objects, where the connection between features is meaningful rather than arbitrary. In Experiments 1 and 2, observers were simultaneously shown four items from two object categories. Within a category, the two exemplars could be presented in either the same or different states (e.g., open/closed; full/empty). After a delay, both exemplars from one of the categories were probed, and participants had to recognize which exemplar went with which state. We found good memory for state information and exemplar information on their own, but a significant memory decrement for exemplar–state combinations, suggesting that binding was difficult for observers and swap errors occurred even for meaningful real-world objects. In Experiment 3, we used the same task, but in one-half of the trials, the locations of the exemplars were swapped at test. We found that there are more errors in general when the locations of exemplars were swapped. We concluded that the internal features of real-world objects are not perfectly bound in working memory, and location updates impair object and feature representations. Overall, we provide evidence that even real-world objects are not stored in an entirely unitized format in working memory.

Almost all models of visual working memory—the cognitive system that holds visual information in an active state—assume it has a fixed capacity: Some models propose a limit of three to four objects, where others propose there is a fixed pool of resources for each basic visual feature. Recent findings, however, suggest that memory performance is improved for real-world objects. What supports these increases in capacity? Here, we test whether the meaningfulness of a stimulus alone influences working memory capacity while controlling for visual complexity and directly assessing the active component of working memory using EEG. Participants remembered ambiguous stimuli that could either be perceived as a face or as meaningless shapes. Participants had higher performance and increased neural delay activity when the memory display consisted of more meaningful stimuli. Critically, by asking participants whether they perceived the stimuli as a face or not, we also show that these increases in visual working memory capacity and recruitment of additional neural resources are because of the subjective perception of the stimulus and thus cannot be driven by physical properties of the stimulus. Broadly, this suggests that the capacity for active storage in visual working memory is not fixed but that more meaningful stimuli recruit additional working memory resources, allowing them to be better remembered.

When you search repeatedly for a set of items among very similar distractors, does that make you more efficient in locating the targets? To address this, we had observers search for two categories of targets among the same set of distractors across trials. Visual and conceptual similarity of the stimuli were validated with a multidimensional scaling analysis, and separately using a deep neural network model. After a few blocks of visual search trials, the distractor set was replaced. In three experiments, we manipulated the level of discriminability between the targets and distractors before and after the distractors were replaced. Our results suggest that in the presence of repeated distractors, observers generally become more efficient. However, the difficulty of the search task does impact how efficient people are when the distractor set is replaced. Specifically, when the training is easy, people are more impaired in a difficult transfer test. We attribute this effect to the precision of the target template generated during training. In particular, a coarse target template is created when the target and distractors are easy to discriminate. These coarse target templates do not transfer well in a context with new distractors. This suggests that learning with more distinct targets and distractors can result in lower performance when context changes, but observers recover from this effect quickly (within a block of search trials).

Almost all models of visual memory implicitly assume that errors in mnemonic representations are linearly related to distance in stimulus space. Here we show that neither memory nor perception are appropriately scaled in stimulus space; instead, they are based on a transformed similarity representation that is nonlinearly related to stimulus space. This result calls into question a foundational assumption of extant models of visual working memory. Once psychophysical similarity is taken into account, aspects of memory that have been thought to demonstrate a fixed working memory capacity of around three or four items and to require fundamentally different representations—across different stimuli, tasks and types of memory—can be parsimoniously explained with a unitary signal detection framework. These results have substantial implications for the study of visual memory and lead to a substantial reinterpretation of the relationship between perception, working memory and long-term memory. Schurgin et al. propose a model of visual memory, arguing against a distinction between how many items are represented and how precisely they are represented, and in favour of a view based on continuous representations in noisy channels.

When objects move, their motion is governed by the laws of physics. We investigated whether multiple objects that move while correctly obeying aspects of Newtonian physics are easier to track than those that do not accurately obey the laws of physics. Participants were asked to track multiple objects that either did or did not take on the correct angles and/or speeds after collisions with each other. We found an advantage for tracking when objects obeyed realistic physics, such that people were more accurate when objects reflected from each other at proper angles and when objects varied in speed after collisions (as opposed to always maintaining the same speed). This advantage was independent of a variety of low-level factors that would be expected to affect object tracking, such as object spacing. However, we also found that performance was not affected when objects' speed changed randomly after each collision (so long as it varied), nor when the reflection angles were jittered moderately after collisions. We conclude that perceptual noise seriously limits many aspects of object trajectory estimation, but nevertheless people are sensitive to at least a subset of the Newtonian laws of physics under demanding attentional tracking conditions. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

People can rapidly and efficiently categorize the animacy of individual objects and scenes, even with few visual features available. Does this necessarily mean that the visual system has an unlimited capacity to process animacy across the entire visual field? We tested this in an ensemble task requiring observers to judge the relative numerosity of animate vs. inanimate items in briefly presented sets of objects. We generated a set of morphed “animacy continua” between pairs of animal and inanimate object silhouettes and tested them in both individual object categorization and ensemble enumeration. For the ensemble task, we manipulated the ratio between animate and inanimate items present in the display and used both “segmentable” (including only definitely animate and inanimate items) and “non-segmentable” (middle-value, ambiguous morphs pictures were shown along with the extremes) distributions of items. We found that observers failed to integrate animacy information from multiple items, as they showed very poor performance in the ensemble task and were not sensitive to the distribution type despite their categorization rate for individual objects being near 100%. A control condition using the same design with color as a category-defining dimension elicited both good individual object and ensemble categorization performance and a strong effect of the segmentability type. We conclude that good individual categorization does not necessarily allow people to build ensemble animacy representations, thus showing the limited capacity of animacy perception.

Existing knowledge shapes and distorts our memories, serving as a prior for newly encoded information. Here, we investigate the role of stable long-term priors (e.g. categorical knowledge) in conjunction with priors arising from recently encountered information (e.g. 'serial dependence') in visual working memory for color. We use an iterated reproduction paradigm to allow a model-free assessment of the role of such priors. In Experiment 1, we find that participants' reports reliably converge to certain areas of color space, but that this convergence is largely distinct for different individuals, suggesting responses are biased by more than just shared category knowledge. In Experiment 2, we explicitly manipulate trial n-1 and find recent history plays a major role in participants' reports. Thus, we find that both global prior knowledge and recent trial information have biasing influences on visual working memory, demonstrating an important role for both short- and long-term priors in actively maintained information.

Long-term memory is often considered easily corruptible, imprecise, and inaccurate, especially in comparison to working memory. However, most research used to support these findings relies on weak long-term memories: those where people have had only one brief exposure to an item. Here we investigated the fidelity of visual long-term memory in more naturalistic setting, with repeated exposures, and ask how it compares to visual working memory fidelity. Using psychophysical methods designed to precisely measure the fidelity of visual memory, we demonstrate that long-term memory for the color of frequently seen objects is as accurate as working memory for the color of a single item seen 1 s ago. In particular, we show that repetition greatly improves long-term memory, including the ability to discriminate an item from a very similar item (fidelity), in both a lab setting (Experiments 1-3) and a naturalistic setting (brand logos, Experiment 4). Overall, our results demonstrate the impressive nature of visual long-term memory fidelity, which we find is even higher fidelity than previously indicated in situations involving repetitions. Furthermore, our results suggest that there is no distinction between the fidelity of visual working memory and visual long-term memory, but instead both memory systems are capable of storing similar incredibly high-fidelity memories under the right circumstances. Our results also provide further evidence that there is no fundamental distinction between the "precision" of memory and the "likelihood of retrieving a memory," instead suggesting a single continuous measure of memory strength best accounts for working and long-term memory. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Prevailing theories of visual working memory assume that each encoded item is stored or forgotten as a separate unit independent from other items. Here, we show that items are not independent and that the recalled orientation of an individual item is strongly influenced by the summary statistical representation of all items (ensemble representation). We find that not only is memory for an individual orientation substantially biased toward the mean orientation, but the precision of memory for an individual item also closely tracks the precision with which people store the mean orientation (which is, in turn, correlated with the physical range of orientations). Thus, individual items are reported more precisely when items on a trial are more similar. Moreover, the narrower the range of orientations present on a trial, the more participants appear to rely on the mean orientation as representative of all individuals. This can be observed not only when the range is carefully controlled, but also shown even in randomly generated, unstructured displays, and after accounting for the possibility of location-based 'swap' errors. Our results suggest that the information about a set of items is represented hierarchically, and that ensemble information can be an important source of information to constrain uncertain information about individuals. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Understanding how people rate their confidence is critical for the characterization of a wide range of perceptual, memory, motor and cognitive processes. To enable the continued exploration of these processes, we created a large database of confidence studies spanning a broad set of paradigms, participant populations and fields of study. The data from each study are structured in a common, easy-to-use format that can be easily imported and analysed using multiple software packages. Each dataset is accompanied by an explanation regarding the nature of the collected data. At the time of publication, the Confidence Database (which is available at https://osf.io/s46pr/) contained 145 datasets with data from more than 8,700 participants and almost 4 million trials. The database will remain open for new submissions indefinitely and is expected to continue to grow. Here we show the usefulness of this large collection of datasets in four different analyses that provide precise estimations of several foundational confidence-related effects. This Resource introduces a new public database that enables researchers to re-analyse a large corpus of studies into meta-cognitive confidence judgements.

Building a unified representation of an event requires binding object and scene information in visual long-term memory (VLTM). While previous studies have examined how humans remember individual objects and scenes, little is known about the mechanisms that support object-scene binding. In this study, we examined whether language plays a role in binding objects and scenes in VLTM. Participants studied a large number of object-scene pairs, either while performing no concurrent task, a concurrent verbal shadowing task, or a concurrent rhythmic shadowing task. Participants were then tested on their memory for the individual objects and scenes (entity memory) or their memory for which objects were displayed in which scenes (object-scene binding). We found that (1) the rhythmic load and verbal load impaired memory for objects and scenes to a similar extent, but (2) the verbal load impaired object-scene binding significantly more than the rhythmic load. Thus, suppressing verbal resources during encoding selectively disrupts object-scene binding in long-term memory. We conclude that language networks play an important role in object-scene binding in VLTM.

People can store thousands of real-world objects in visual long-term memory with high precision. But are these objects stored as unitary, bound entities, as often assumed, or as bundles of separable features? We tested this in several experiments. In the first series of studies, participants were instructed to remember specific exemplars of real-world objects presented in a particular state (e.g., open/closed, full/empty, etc.), and then were asked to recognize either which exemplars they had seen (e.g., I saw this coffee mug), or which exemplar-state conjunctions they had seen (e.g., I saw this coffee mug and it was full). Participants had a large number of within-category confusions, for example misremembering which states went with which exemplars, while simultaneously showing strong memory for the features themselves (e.g., which states they had seen, which exemplars they had seen). In a second series of studies, we found further evidence of independence: participants were very good at remembering which exemplars they had seen independently of whether these items were presented in a new or old state, but the same did not occur for features known to be truly holistically represented. Thus, we find through 2 lines of evidence that the features of real-world objects that support exemplar discrimination and state discrimination are not bound, suggesting visual objects are not inherently unitary entities in memory. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Working memory is a core cognitive system that actively maintains information in an accessible state to support a variety of everyday tasks. Crucially, working memory performance has frequently been shown to strongly correlate with fluid intelligence. Traditionally when these correlations have been observed, the working memory tasks involved required a high degree of manipulation and executive function, as opposed to solely utilizing short-term storage capacity. However, recent work has claimed that simple storage capacity is also correlated with fluid intelligence, and that this is driven by a particularly special and dissociable component of capacity, the ‘number of items represented’ (rather than the precision of those representations). These results have been used to argue that investigating the underlying mechanisms of capacity limitations may be critical to understanding aspects of fluid intelligence. Here we demonstrate that such correlations do not arise solely or primarily from simple storage capacity (nor a single dissociable component of capacity), but are driven by the availability of strategic encoding of different kinds of visual representations. Specifically, a working memory task that decreased the utility of storing and making use of spatial ensemble information, while holding constant the number of items to be remembered and the exact changes participants needed to detect, significantly reduced the correlation between working memory performance and fluid intelligence. Thus, despite being probed on the same items, with the same foils, at the same set size, only working memory displays that allowed for the strategic use of both item and ensemble representations correlated with fluid intelligence. These results provide evidence against the hypothesis that simple storage alone is related to fluid intelligence. They also demonstrate that participants make use of more complex and structured representations rather than solely individual item representation, and that strategic utilization of these representations is what correlates strongly with fluid intelligence.

Although Bastin et al. propose a useful model for thinking about the structure of memory and memory deficits, their distinction between entities and relational encoding is incompatible with data showing that even individual objects – prototypical “entities” – are made up of distinct features which require binding. Thus, “entity” and “relational” brain regions may need to solve fundamentally the same problems.

Previous studies have shown that category learning affects subsequent recognition memory. However, questions remain as to how category learning affects discriminability during recognition. In this three-stage study, we employed sets of simulated flowers with category- and non-category-inclusion features appearing with equal probabilities. In the learning stage, participants were asked to categorize flowers by identifying the category-inclusion feature. Next, in the studying stage, participants memorized a new set of flowers, a third of which belonged to the learned category. Finally, in the testing stage, participants received a recognition test with old and new flowers, some from the learned category, some from a not-learned category, some from both categories, and some from neither category. We applied hierarchical Bayesian signal detection theory models to recognition performance and found that prior category learning affected both discriminability as well as criterion bias. That is, people that learned the category well, exhibited improved discriminability and a shifted bias toward flowers from the learned relative to the not learned category.

How do people use human-made objects (artifacts) to learn about the people and actions that created them? We test the richness of people’s reasoning in this domain, focusing on the task of judging whether social transmission has occurred (i.e. whether one person copied another). We develop a formal model of this reasoning process as a form of rational inverse planning, which predicts that rather than solely focusing on artifacts’ similarity to judge whether copying occurred, people should also take into account availability constraints (the materials available), and functional constraints (which materials work). Using an artifact-building task where two characters build tools to solve a puzzle box, we find that this inverse planning model predicts trial-by-trial judgments, whereas simpler models that do not consider availability or functional constraints do not. This suggests people use a process like inverse planning to make flexible inferences from artifacts’ features about the source of design ideas.

Is there a fixed limit on how many objects we can hold actively in mind? Generally, researchers have found participants are worse at remembering a small number of objects if those objects are more complex, suggesting a limited resource rather than a fixed number of objects best explains working memory performance. However, some evidence has suggested that stimulus similarity better accounts for these effects and that, after accounting for such similarity, the data support a slot-based fixed item limit for working memory. Much of the evidence used to support the latter claim relies on working memory displays containing different categories of items. It has been found that, for large, across-category changes, performance does not differ for different kinds of complex stimuli. However, many of these studies fail to adequately control for the potential use of ensemble information in discriminating such large changes. Here, we sought to identify how much ensemble representations may explain performance across these tasks. In Experiment 1, we observed that, as set size increased from four to 12 items, capacity estimates for across-category changes increased linearly as well, providing evidence against the claim of a fixed capacity. In Experiment 2, we controlled for stimulus complexity and similarity but varied the utility of ensemble representations for the change-detection task. We observed significantly greater capacity when ensemble information could be used. Altogether, these results are contrary to a slot-like, fixed-object constraint on working memory capacity and consistent with object complexity and ensemble representations strongly affecting working memory performance.

Both visual attention and visual working memory tend to be studied with very simple stimuli and low-level paradigms, designed to allow us to understand the representations and processes in detail, or with fully realistic stimuli that make such precise understanding difficult but are more representative of the real world. In this chapter we argue for an intermediate approach in which visual attention and visual working memory are studied by scaling up from the simplest settings to more complex settings that capture some aspects of the complexity of the real-world, while still remaining in the realm of well-controlled stimuli and well-understood tasks. We believe this approach, which we have been taking in our labs, will allow a more generalizable set of knowledge about visual attention and visual working memory while maintaining the rigor and control that is typical of vision science and psychophysics studies.

How people process images is known to affect memory for those images, but these effects have typically been studied using explicit task instructions to vary encoding. Here, we investigate the effects of intrinsic variation in processing on subsequent memory, testing whether recognizing an ambiguous stimulus as meaningful (as a face vs as shape blobs) predicts subsequent visual memory even when matching the perceptual features and the encoding strategy between subsequently remembered and subsequently forgotten items. We show in adult humans of either sex that single trial EEG activity can predict whether participants will subsequently remember an ambiguous Mooney face image (e.g., an image that will sometimes be seen as a face and sometimes not be seen as a face). In addition, we show that a classifier trained only to discriminate between whether participants perceive a face versus non-face can generalize to predict whether an ambiguous image is subsequently remembered. Furthermore, when we examine the N170, an event-related potential index of face processing, we find that images that elicit larger N170s are more likely to be remembered than those that elicit smaller N170s, even when the exact same image elicited larger or smaller N170s across participants. Thus, images processed as meaningful, in this case as a face, during encoding are better remembered than identical images that are not processed as a face. This provides strong evidence that understanding the meaning of a stimulus during encoding plays a critical role in visual memory. SIGNIFICANCE STATEMENT Is visual memory inherently visual or does meaning and other conceptual information necessarily play a role even in memory for detailed visual information? Here we show that it is easier to remember an image when it is processed in a meaningful way, as indexed by the amount of category-specific brain activity it elicits. In particular, we use single-trial EEG activity to predict whether an image will be subsequently remembered, and show that the main driver of this prediction ability is whether or not an image is seen as meaningful or non-meaningful. This shows that the extent to which an image is processed as meaningful can be used to predict subsequent memory even when controlling for perceptual factors and encoding strategies that typically differ across images.

The ability to perceive and remember the spatial layout of a scene is critical to understanding the visual world, both for navigation and for other complex tasks that depend upon the structure of the current environment. However, surprisingly little work has investigated how and when scene layout information is maintained in memory. One prominent line of work investigating this issue is a scene-priming paradigm (e.g., Sanocki & Epstein, 1997), in which different types of previews are presented to participants shortly before they judge which of two regions of a scene is closer in depth to the viewer. Experiments using this paradigm have been widely cited as evidence that scene layout information is stored across brief delays and have been used to investigate the structure of the representations underlying memory for scene layout. In the present experiments, we better characterize these scene-priming effects. We find that a large amount of visual detail rather than the presence of depth information is necessary for the priming effect; that participants show a preview benefit for a judgment completely unrelated to the scene itself; and that preview benefits are susceptible to masking and quickly decay. Together, these results suggest that "scene priming" effects do not isolate scene layout information in memory, and that they may arise from low-level visual information held in sensory memory. This broadens the range of interpretations of scene priming effects and suggests that other paradigms may need to be developed to selectively investigate how we represent scene layout information in memory.

Visual working memory is the mechanism supporting the continued maintenance of information after sensory inputs are removed. Although the capacity of visual working memory is limited, memoranda that are spaced farther apart on a 2-D display are easier to remember, potentially because neural representations are more distinct within retinotopically organized areas of visual cortex during memory encoding, maintenance, or retrieval. The impact on memory of spatial separability in depth is less clear, even though depth information is essential to guiding interactions with objects in the environment. On one account, separating memoranda in depth may facilitate performance if interference between items is reduced. However, depth information must be inferred indirectly from the 2-D retinal image, and less is known about how visual cortex represents depth. Thus, an alternative possibility is that separation in depth does not attenuate between-items interference; it may even impair performance, as attention must be distributed across a larger volume of 3-D space. We tested these alternatives using a stereo display while participants remembered the colors of stimuli presented either near or far in the 2-D plane or in depth. Increasing separation in-plane and in depth both enhanced performance. Furthermore, participants who were better able to utilize stereo depth cues showed larger benefits when memoranda were separated in depth, particularly for large memory arrays. The observation that spatial separation in the inferred 3-D structure of the environment improves memory performance, as is the case in 2-D environments, suggests that separating memoranda in depth might reduce neural competition by utilizing cortically separable resources.

Humans have remarkable visual long-term memory abilities, capable of storing thousands of objects with significant detail. However, it remains unknown how such memory is utilized during the short-term maintenance of information. Specifically, if people have a previously encoded memory for an item, how does this affect subsequent working memory for that same item? Here, we demonstrate people can quickly and accurately make use of visual long-term memories and therefore maintain less perceptual information actively in working memory. We assessed how much perceptual information is actively maintained in working memory by measuring neural activity during the delay period of a working memory task using electroencephalography. We find that despite maintaining less perceptual information in working memory when long-term memory representations are available, there is no decrement in memory performance. This suggests under certain circumstances people can dynamically disengage working memory maintenance and instead use long-term memories when available. However, this does not mean participants always utilize long-term memory. In a follow-up experiment, we introduced additional perceptual interference into working memory and found participants actively maintained items in working memory even when they had existing long-term memories available. These results clarify the kinds of conditions under which long-term and working memory operate. Specifically, working memory is engaged when new information is encountered or perceptual interference is high. Visual long-term memory may otherwise be rapidly accessed and utilized in lieu of active perceptual maintenance. These data demonstrate the interactions between working memory and long-term memory are more dynamic and fluid than previously thought.

People can quickly and accurately compute not only the mean size of a set of items but also the size variability of the items. However, it remains unknown how these statistics are estimated. Here we show that neither parallel access to all items nor random subsampling of just a few items is sufficient to explain participants' estimations of size variability. In three experiments, we had participants compare two arrays of circles with different variability in their sizes. In the first two experiments, we manipulated the congruency of the range and variance of the arrays. The arrays with congruent range and variability information were judged more accurately, indicating the use of range as a proxy for variability. Experiments 2B and 3 showed that people also are not invariant to low- or mid-level visual information in the arrays, as comparing arrays with different low-level characteristics (filled vs. outlined circles) led to systematic biases. Together, these experiments indicate that range and low- or mid-level properties are both utilized as proxies for variability discrimination, and people are flexible in adopting these strategies. These strategies are at odds with the claim of parallel extraction of ensemble statistics per se and random subsampling strategies previously proposed in the literature.

Human-made objects (artifacts) often provide rich social information about the people who created them. We explore how people reason about others from the objects they create, characterizing inferences about when social transmission of ideas (copying) has occurred. We test whether judgments are driven by perceptual heuristics, or structured explanation- based reasoning. We develop a Bayesian model of explanation-based inference from artifacts and a simpler model of perceptual heuristics, and ask which better predicts people’s judgments. Our artifact-building task involved two characters who built toy train tracks. Participants viewed pairs of tracks, and judged whether copying had occurred. Our explanation-based model accurately predicted on a trial-by- trial basis when participants inferred copying; the perceptual heuristics model was significantly less accurate. Efficient design ‘explained away’ similarity, making similarity weaker evidence of copying for efficient tracks. Overall, data show that like intuitive archeologists, people make rich explanation-based inferences about others from the objects they create.

Human memory systems are subject to many imperfections, including memory distortions and the creation of false memories. Here, we demonstrate a case where memory distortion is adaptive, increasing the overall accuracy of memories. Participants viewed multiple real-world objects from a given category (10 airplanes, 10 backpacks…), and later recalled the color of each object. Participants were generally accurate, but even when they remembered having seen an item and remembered its color, they nevertheless reported the color as closer to the average color of its category than it really was. Although participants’ memories were systematically distorted, they were distorted in a way that is consistent with minimizing their average error according to a simple Bayesian analysis. In addition, and consistent with the Bayesian analysis, the bias toward the category center was larger when participant’s had greater uncertainty about the color of an item, but was present in all circumstances -- even when participants remembered an item, remembered its color, and reported high confidence in their color memory. Thus, memory distortion may not always be maladaptive: in some cases, distortion can result from a memory system that optimally combines information in the service of the broader goals of the person. Furthermore, this framework for thinking about memory distortion suggests that false memory can be thought of on a continuum with true memory: the greater uncertainty participants have about an individual item memory, the more they weight their gist memory; with no item information, they weight only their gist memory.

When people need to remember a whole set of words or images, they tend to remember any particular item as more closely resembling the gist of the entire set than it really was. Here we show that when participants are asked to remember only a few items -- and so maintaining the distinctiveness of this item in memory is of particular importance -- memory for that item is distorted in the opposite direction, amplifying its salient features. In a sequence of 4 experiments, we asked participants to remember the aspect ratio and size of a rectangle and draw it after various delays. Participants reliably exaggerated its distinctive feature in every experiment. This distortion occurred not just at initial encoding but also during memory consolidation and persisted for several hours. Thus, when remembering only a few items, memory amplifies the distinctive features of these items, a form of adaptive memory distortion.

It is clear that unreinforced repetition (familiarization) influences affective responses to social stimuli, but its effects on the perception of facial emotion are unknown. Reporting the results of two experiments, we show for the first time that repeated exposure enhances the perceived happiness of facial expressions. In Experiment 1, using a paradigm in which subjects’ responses were orthogonal to happiness in order to avoid response biases, we found that faces of individuals who had previously been shown were deemed happier than novel faces. In Experiment 2, we replicated this effect with a rapid “happy or angry” categorization task. Using psychometric function fitting, we found that for subjects to classify a face as happy, they needed less actual happiness to be present in the face if the target was familiar than if it was novel. Critically, our results suggest that familiar faces appear happier than novel faces because familiarity selectively enhances the impact of positive stimulus features.

Traditionally, recognizing the objects within a scene has been treated as a prerequisite to recognizing the scene itself. However, research now suggests that the ability to rapidly recognize visual scenes could be supported by global properties of the scene itself rather than the objects within the scene. Here, we argue for a particular instantiation of this view: That scenes are recognized by treating them as a global texture and processing the pattern of orientations and spatial frequencies across different areas of the scene without recognizing any objects. To test this model, we asked whether there is a link between how proficient individuals are at rapid scene perception and how proficiently they represent simple spatial patterns of orientation information (global ensemble texture). We find a significant and selective correlation between these tasks, suggesting a link between scene perception and spatial ensemble tasks but not nonspatial summary statistics In a second and third experiment, we additionally show that global ensemble texture information is not only associated with scene recognition, but that preserving only global ensemble texture information from scenes is sufficient to support rapid scene perception; however, preserving the same information is not sufficient for object recognition. Thus, global ensemble texture alone is sufficient to allow activation of scene representations but not object representations. Together, these results provide evidence for a view of scene recognition based on global ensemble texture rather than a view based purely on objects or on nonspatially localized global properties.

Previous research has shown that prior knowledge structures or schemas affect recognition memory. However, since the acquisition of schemas occurs over prolonged periods of time, few paradigms allow the direct manipulation of schema acquisition to study their effect on memory performance. Recently, a number of parallelisms in recognition memory between studies involving schemas and studies involving category learning have been identified. The current paper capitalizes on these findings and offers a novel experimental paradigm that allows manipulation of category learning between individuals to study the effects of schema acquisition on recognition. First, participants learn to categorize computer-generated items whose category-inclusion criteria differ between participants. Next, participants study items that belong to either the learned category, the non-learned category, both, or neither. Finally, participants receive a recognition test that includes old and new items, either from the learned, the non-learned, or neither category. Using variations on this paradigm, four experiments were conducted. The results from the first three studies suggest that learning a category increases hit rates for old category-consistent items and false alarm rates for new category-consistent lures. Absent the category learning, no such effects are evident, even when participants are exposed to the same learning trials as those who learned the categories. The results from the fourth experiment suggest that, at least for false alarm rates, the effects of category learning are not solely attributable to frequency of occurrence of category-consistent items during learning. Implications for recognition memory as well as advantages of the proposed paradigm are discussed.

Visual working memory is the cognitive system that holds visual information in an active state, making it available for cognitive processing and protecting it against interference. Here, we demonstrate that visual working memory has a greater capacity than previously measured. In particular, we use EEG to show that, contrary to existing theories, enhanced performance with real-world objects relative to simple stimuli in short-term memory tasks is reflected in active storage in working memory and is not entirely due to the independent usage of episodic long-term memory systems. These data demonstrate that working memory and its capacity limitations are dependent upon our knowledge. Thus, working memory is not fixed-capacity; instead, its capacity is dependent on exactly what is being remembered.

Influential slot and resource models of visual working memory make the assumption that items are stored in memory as independent units, and that there are no interactions between them. Consequently, these models predict that the number of items to be remembered (the set size) is the primary determinant of working memory performance, and therefore these models quantify memory capacity in terms of the number and quality of individual items that can be stored. Here we demonstrate that there is substantial variance in display difficulty within a single set size, suggesting that limits based on the number of individual items alone cannot explain working memory storage. We asked hundreds of participants to remember the same sets of displays, and discovered that participants were highly consistent in terms of which items and displays were hardest or easiest to remember. Although a simple grouping or chunking strategy could not explain this individual-display variability, a model with multiple, interacting levels of representation could explain some of the display-by-display differences. Specifically, a model that includes a hierarchical representation of items plus the mean and variance of sets of the colors on the display successfully accounts for some of the variability across displays. We conclude that working memory representations are composed only in part of individual, independent object representations, and that a major factor in how many items are remembered on a particular display is interitem representations such as perceptual grouping, ensemble, and texture representations.

A central question for models of visual working memory is whether the number of objects people can remember depends on object complexity. Some influential "slot" models of working memory capacity suggest that people always represent 3-4 objects and that only the fidelity with which these objects are represented is affected by object complexity. The primary evidence supporting this claim is the finding that people can detect large changes to complex objects (consistent with remembering at least 4 individual objects), but that small changes cannot be detected (consistent with low-resolution representations). Here we show that change detection with large changes greatly overestimates individual item capacity when people can use global representations of the display to detect such changes. When the ability to use such global ensemble or texture representations is reduced, people remember individual information about only 1-2 complex objects. This finding challenges models that propose people always remember a fixed number of objects, regardless of complexity, and supports a more flexible model with an important role for spatial ensemble representations.

Ensemble perception, including the ability to "see the average" from a group of items, operates in numerous feature domains (size, orientation, speed, facial expression, etc.). Although the ubiquity of ensemble representations is well established, the large-scale cognitive architecture of this process remains poorly defined. We address this using an individual differences approach. In a series of experiments, observers saw groups of objects and reported either a single item from the group or the average of the entire group. High-level ensemble representations (e.g., average facial expression) showed complete independence from low-level ensemble representations (e.g., average orientation). In contrast, low-level ensemble representations (e.g., orientation and color) were correlated with each other, but not with high-level ensemble representations (e.g., facial expression and person identity). These results suggest that there is not a single domain-general ensemble mechanism, and that the relationship among various ensemble representations depends on how proximal they are in representational space.

Our ability to actively maintain information in visual memory is strikingly limited. There is considerable debate about why this is so. As with many questions in psychology, the debate is framed dichotomously: Is visual working memory limited because it is supported by only a small handful of discrete “slots” into which visual representations are placed, or is it because there is an insufficient supply of a “resource” that is flexibly shared among visual representations? Here, we argue that this dichotomous framing obscures a set of at least eight underlying questions. Separately considering each question reveals a rich hypothesis space that will be useful for building a comprehensive model of visual working memory. The questions regard (1) an upper limit on the number of represented items, (2) the quantization of the memory commodity, (3) the relationship between how many items are stored and how well they are stored, (4) whether the number of stored items completely determines the fidelity of a representation (vs. fidelity being stochastic or variable), (5) the flexibility with which the memory commodity can be assigned or reassigned to items, (6) the format of the memory representation, (7) how working memories are formed, and (8) how memory representations are used to make responses in behavioral tasks. We reframe the debate in terms of these eight underlying questions, placing slot and resource models as poles in a more expansive theoretical space.

Are real-world objects represented as bound units? Although a great deal of research has examined binding between the feature dimensions of simple shapes, little work has examined whether the featural properties of real-world objects are stored in a single unitary object representation. In a first experiment, we found that information about an object's color is forgotten more rapidly than the information about an object's state (e.g., open, closed), suggesting that observers do not forget objects as entirely bound units. In a second and third experiment, we examined whether state and exemplar information are forgotten separately or together. If these properties are forgotten separately, the probability of getting one feature correct should be independent of whether the other feature was correct. We found that after a short delay, observers frequently remember both state and exemplar information about the same objects, but after a longer delay, memory for the two properties becomes independent. This indicates that information about object state and exemplar are forgotten separately over time. We thus conclude that real-world objects are not represented in a single unitary representation in visual memory.

The MemToolbox is a collection of MATLAB functions for modeling visual working memory. In support of its goal to provide a full suite of data analysis tools, the toolbox includes implementations of popular models of visual working memory, real and simulated data sets, Bayesian and maximum likelihood estimation procedures for fitting models to data, visualizations of data and fit, validation routines, model comparison metrics, and experiment scripts. The MemToolbox is released under the permissive BSD license and is available at http://memtoolbox.org.

Visual long-term memory can store thousands of objects with surprising visual detail, but just how detailed are these representations, and how can one quantify this fidelity? Using the property of color as a case study, we estimated the precision of visual information in long-term memory, and compared this with the precision of the same information in working memory. Observers were shown real-world objects in random colors and were asked to recall the colors after a delay. We quantified two parameters of performance: the variability of internal representations of color (fidelity) and the probability of forgetting an object’s color altogether. Surprisingly, the fidelity of color information in long-term memory was comparable to the asymptotic precision of working memory. These results suggest that long-term memory and working memory may be constrained by a common limit, such as a bound on the fidelity required to retrieve a memory representation.

When remembering a real-world scene, people encode both detailed information about specific objects and higher order information like the overall gist of the scene. However, formal models of change detection, like those used to estimate visual working memory capacity, assume observers encode only a simple memory representation that includes no higher order structure and treats items independently from one another. We present a probabilistic model of change detection that attempts to bridge this gap by formalizing the role of perceptual organization and allowing for richer, more structured memory representations. Using either standard visual working memory displays or displays in which the items are purposefully arranged in patterns, we find that models that take into account perceptual grouping between items and the encoding of higher order summary information are necessary to account for human change detection performance. Considering the higher order structure of items in visual working memory will be critical for models to make useful predictions about observers' memory capacity and change detection abilities in simple displays as well as in more natural scenes.

As we move through the world, information about objects moves to different spatial frequencies. How the visual system successfully integrates information across these changes to form a coherent percept is thus an important open question. Here we investigate such integration using hybrid faces, which contain different images in low and high spatial frequencies. Observers judged how similar a hybrid was to each of its component images while walking toward or away from it or having the stimulus moved toward or away from them. We find that when the stimulus is approaching, observers act as if they are integrating across spatial frequency separately at each moment. However, when the stimulus is receding, observers show a perceptual hysteresis effect, holding on to details that are imperceptible in a static stimulus condition. Thus, observers appear to make optimal inferences by sticking with their previous interpretation when losing information but constantly reinterpreting their input when gaining new information.

Traditional memory research has focused on identifying separate memory systems and exploring different stages of memory processing. This approach has been valuable for establishing a taxonomy of memory systems and characterizing their function, but has been less informative about the nature of stored memory representations. Recent research on visual memory has shifted towards a representation-based emphasis, focusing on the contents of memory, and attempting to determine the format and structure of remembered information. The main thesis of this review will be that one cannot fully understand memory systems or memory processes without also determining the nature of memory representations. Nowhere is this connection more obvious than in research that attempts to measure the capacity of visual memory. We will review research on the capacity of visual working memory and visual long-term memory, highlighting recent work that emphasizes the contents of memory. This focus impacts not only how we estimate the capacity of the system - going beyond quantifying how many items can be remembered, and moving towards structured representations - but how we model memory systems and memory processes.

Influential models of visual working memory treat each item to be stored as an independent unit and assume that there are no interactions between items. However, real-world displays have structure that provides higher-order constraints on the items to be remembered. Even in the case of a display of simple colored circles, observers can compute statistics, such as mean circle size, to obtain an overall summary of the display. We examined the influence of such an ensemble statistic on visual working memory. We report evidence that the remembered size of each individual item in a display is biased toward the mean size of the set of items in the same color and the mean size of all items in the display. This suggests that visual working memory is constructive, encoding displays at multiple levels of abstraction and integrating across these levels, rather than maintaining a veridical representation of each item independently.

Behavioral and computational studies suggest that visual scene analysis rapidly produces a rich description of both the objects and the spatial layout of surfaces in a scene. However, there is still a large gap in our understanding of how the human brain accomplishes these diverse functions of scene understanding. Here we probe the nature of real-world scene representations using multivoxel functional magnetic resonance imaging pattern analysis. We show that natural scenes are analyzed in a distributed and complementary manner by the parahippocampal place area (PPA) and the lateral occipital complex (LOC) in particular, as well as other regions in the ventral stream. Specifically, we study the classification performance of different scene-selective regions using images that vary in spatial boundary and naturalness content. We discover that, whereas both the PPA and LOC can accurately classify scenes, they make different errors: the PPA more often confuses scenes that have the same spatial boundaries, whereas the LOC more often confuses scenes that have the same content. By demonstrating that visual scene analysis recruits distinct and complementary high-level representations, our results testify to distinct neural pathways for representing the spatial boundaries and content of a visual scene.

Observers can store thousands of object images in visual long-term memory with high fidelity, but the fidelity of scene representations in long-term memory is not known. Here, we probed scene-representation fidelity by varying the number of studied exemplars in different scene categories and testing memory using exemplar-level foils. Observers viewed thousands of scenes over 5.5 hr and then completed a series of forced-choice tests. Memory performance was high, even with up to 64 scenes from the same category in memory. Moreover, there was only a 2% decrease in accuracy for each doubling of the number of studied scene exemplars. Surprisingly, this degree of categorical interference was similar to the degree previously demonstrated for object memory. Thus, although scenes have often been defined as a superset of objects, our results suggest that scenes and objects may be entities at a similar level of abstraction in visual long-term memory.

When encoding a scene into memory, people store both the overall gist of the scene and detailed information about a few specific objects. Moreover, they use the gist to guide their choice of which specific objects to remember. However, formal models of change detection, like those used to estimate visual working memory capacity, generally assume people represent no higher-order structure about the display and choose which items to encode at random. We present a probabilistic model of change detection that attempts to bridge this gap by formalizing the encoding of both specific items and higherorder information about simple working memory displays. We show that this model successfully predicts change detection performance for individual displays of patterned dots. More generally, we show that it is necessary for the model to encode higher-order structure in order to accurately predict human performance in the change detection task. This work thus confirms and formalizes the role of higher-order structure in visual working memory.

Humans have a massive capacity to store detailed information in visual long-term memory. The present studies explored the fidelity of these visual long-term memory representations and examined how conceptual and perceptual features of object categories support this capacity. Observers viewed 2,800 object images with a different number of exemplars presented from each category. At test, observers indicated which of 2 exemplars they had previously studied. Memory performance was high and remained quite high (82% accuracy) with 16 exemplars from a category in memory, demonstrating a large memory capacity for object exemplars. However, memory performance decreased as more exemplars were held in memory, implying systematic categorical interference. Object categories with conceptually distinctive exemplars showed less interference in memory as the number of exemplars increased. Interference in memory was not predicted by the perceptual distinctiveness of exemplars from an object category, though these perceptual measures predicted visual search rates for an object target among exemplars. These data provide evidence that observers’ capacity to remember visual information in long-term memory depends more on conceptual structure than perceptual distinctiveness.

validity effects, varies systematically as a function of variations in similarity. Importantly, this type of systematic variation also provides the necessary conditions for assessing the presence of a mixture distribution, allowing the opportunity to test whether or not the two-process model of attention allocation applies to instances of attentional capture. A mixture analysis appliedtotheresultsofthesecondexperimentfoundthattheobtainedvariance for the intermediate distributionswas significantly less than that predicted by a mixture of capture and no capture trials, disconfirming a two-process model.

The information that individuals can hold in working memory is quite limited, but researchers have typically studied this capacity using simple objects or letter strings with no associations between them. However, in the real world there are strong associations and regularities in the input. In an information theoretic sense, regularities introduce redundancies that make the input more compressible. The current study shows that observers can take advantage of these redundancies, enabling them to remember more items in working memory. In 2 experiments, covariance was introduced between colors in a display so that over trials some color pairs were more likely to appear than other color pairs. Observers remembered more items from these displays than from displays where the colors were paired randomly. The improved memory performance cannot be explained by simply guessing the high-probability color pair, suggesting that observers formed more efficient representations to remember more items. Further, as observers learned the regularities, their working memory performance improved in a way that is quantitatively predicted by a Bayesian learning model and optimal encoding scheme. These results suggest that the underlying capacity of the individuals' working memory is unchanged, but the information they have to remember can be encoded in a more compressed fashion.

Observers can resume a previously interrupted visual search trial significantly more quickly than they can start a new search trial (Lleras, Rensink, & Enns, 2005). This rapid resumption of search is possible because evidence accumulated during the previous exposure, a perceptual hypothesis, can carry over to a subsequent presentation. We present four interrupted visual search experiments in which the content of the perceptual hypotheses used during visual search trials was characterized. These experiments suggest that prior to explicit target identification, observers have accumulated evidence about the locations, but not the identities, of local, task-relevant distractors, as well as preliminary evidence for the identity of the target. Our results characterize the content of perceptual search hypotheses and highlight the utility of interrupted search for studying online search processing prior to target identification.

The human capacity for music consists of certain core phenomena, including the tendency to entrain, or align movement, to an external auditory pulse [1–3]. This ability, fundamental both for music production and for coordinated dance, has been repeatedly highlighted as uniquely human [4–11]. However, it has recently been hypothesized that entrainment evolved as a by-product of vocal mimicry, generating the strong prediction that only vocal mimicking animals may be able to entrain [12, 13]. Here we provide comparative data demonstrating the existence of two proficient vocal mimicking nonhuman animals (parrots) that entrain to music, spontaneously producing synchronized movements resembling human dance. We also provide an extensive comparative data set from a global video database systematically analyzed for evidence of entrainment in hundreds of species both capable and incapable of vocal mimicry. Despite the higher representation of vocal nonmimics in the database and comparable exposure of mimics and nonmimics to humans and music, only vocal mimics showed evidence of entrainment. We conclude that entrainment is not unique to humans and that the distribution of entrainment across species supports the hypothesis that entrainment evolved as a by-product of selection for vocal mimicry.

A large body of literature has shown that observers often fail to notice significant changes in visual scenes, even when these changes happen right in front of their eyes. For instance, people often fail to notice if their conversation partner is switched to another person, or if large background objects suddenly disappear.1,2 These ‘change blindness’ studies have led to the inference that the amount of information we remember about each item in a visual scene may be quite low.1 However, in recent work we have demonstrated that long-term memory is capable of storing a massive number of visual objects with significant detail about each item.3 In the present paper we attempt to reconcile these findings by demonstrating that observers do not experience ‘change blindness’ with the real world objects used in our previous experiment if they are given sufficient time to encode each item. Our results (see also refs. 4 and 5) suggest that one of the major causes of change blindness for real-world objects is a lack of encoding time or attention to each object.

One of the major lessons of memory research has been that human memory is fallible, imprecise, and subject to interference. Thus, although observers can remember thousands of images, it is widely assumed that these memories lack detail. Contrary to this assumption, here we show that long-term memory is capable of storing a massive number of objects with details from the image. Participants viewed pictures of 2,500 objects over the course of 5.5 h. Afterward, they were shown pairs of images and indicated which of the two they had seen. The previously viewed item could be paired with either an object from a novel category, an object of the same basic-level category, or the same object in a different state or pose. Performance in each of these conditions was remarkably high (92%, 88%, and 87%, respectively), suggesting that participants successfully maintained detailed representations of thousands of images. These results have implications for cognitive models, in which capacity limitations impose a primary computational constraint (e.g., models of object recognition), and pose a challenge to neural models of memory storage and retrieval, which must be able to account for such a large and detailed storage capacity.

Previous work on visual short-term memory (VSTM) capacity has typically used patches of color or simple features which are drawn from a uniform distribution, and estimated the capacity of VSTM to be 3-4 items (Luck & Vogel, 1997). Here, we introduce covariance information between colors, and ask if VSTM can take advantage of this redundancy to form a more efficient representation of the displays. We find that observers can successfully remember 5 colors on these displays, significantly higher than the 3 colors remembered when the displays were changed to be uniformly distributed in the final block of the experiment. We suggest that quantifying capacity in terms of number of objects remembered fails to capture factors such as object complexity or statistical redundancy, and that information theoretic measures are better suited to characterizing the capacity of VSTM. We use Huffman coding to model our data, and demonstrate that the data are consistent with a fixed VSTM capacity in bits rather than in terms of number of objects.

Recent work has shown that observers can parse streams of syllables, tones, or visual shapes and learn statistical regularities in them without conscious intent (e.g., learn that A is always followed by B). Here, we demonstrate that these statistical-learning mechanisms can operate at an abstract, conceptual level. In Experiments 1 and 2, observers incidentally learned which semantic categories of natural scenes covaried (e.g., kitchen scenes were always followed by forest scenes). In Experiments 3 and 4, category learning with images of scenes transferred to words that represented the categories. In each experiment, the category of the scenes was irrelevant to the task. Together, these results suggest that statistical-learning mechanisms can operate at a categorical level, enabling generalization of learned regularities using existing conceptual knowledge. Such mechanisms may guide learning in domains as disparate as the acquisition of causal knowledge and the development of cognitive maps from environmental exploration.

Predictive visual context facilitates visual search, a benefit termed contextual cuing (M. M. Chun & Y. Jiang, 1998). In the original task, search arrays were repeated across blocks such that the spatial configuration (context) of all of the distractors in a display predicted an embedded target location. The authors modeled existing results using a connectionist architecture and then designed new behavioral experiments to test the model's assumptions. The modeling and behavioral results indicate that learning may be restricted to the local context even when the entire configuration is predictive of target location. Local learning constrains how much guidance is produced by contextual cuing. The modeling and new data also demonstrate that local learning requires that the local context maintain its location in the overall global context.