Memory replay during human sleep happens with slow brain wave and spindle patterns

The experiment consisted of two experimental sessions (object and scene condition), separated by at least 1 week (mean = 8.5 ± 0.85 days). The order of the two sessions was counterbalanced across participants. On experimental days participants arrived at the sleep laboratory at 11 a.m. The experimental session started with the set-up for polysomnographic recordings during which electrodes for electroencephalographic (EEG), electromyographic (EMG), and electrocardiographic (ECG) recordings were applied. Before the experimental sessions, participants were habituated to the environment by spending an adaptation nap in the sleep laboratory.

At around 12 a.m. the experiment started with a modified version of the psychomotor vigilance task ("PVT"⁷¹), followed by the memory task (for details see Memory Task below). The sleep period began at ~1 p.m. and participants were given 120 min to nap (mean total sleep time: 101.63 ± 2.23 min; for sleep characteristics see Supplementary Table 2). Afterwards, the vigilance of all participants was assessed using the PVT and memory performance was tested again. At the end of each session a localizer task was conducted (see Localizer Task for details).

A set of in total 360 verbs and 240 images (half objects and half scenes) served as experimental stimuli during both sessions. Objects were images of animals, food, clothing, tools, or household items presented on a plain white background (e.g., a hammer). Scenes were images of nameable landscapes or places (e.g., a coffee shop). All images were taken from⁷².

For the recording of behavioral responses and the presentation of all experimental tasks, Psychophysics Toolbox Version 3⁷³ and MATLAB 2018b (MathWorks, Natick, USA) were used. Participants completed a practice run (five trials) of each experimental task in advance to ensure they fully understood the instructions. Responses were made via keyboard presses on a dedicated PC. Across all experimental phases, presentation order of stimuli was randomized across participants.

The vigilance of the participants was assessed using a modified version of the "PVT"⁷¹ before the encoding phase and right after the sleep period. Participants were presented with a centered fixation cross on the computer screen. Every 2–10 s the fixation cross was replaced by a counter counting up from 0 to 2 s in steps of 20 ms. Participants were instructed to stop the counter as fast as possible by pressing the space bar. After each trial participants were provided with feedback about their reaction time. The task was administered for 5 min. For PVT related results see Supplementary Fig. 8.

The experiment began with an image familiarization phase. The purpose of this part was (i) to facilitate learning of the verb-image pairs in the main encoding session and (ii) to provide the proper image names for subsequent cued recall. Each trial started with a fixation cross, presented for 1.5 ± 0.1 s. Subsequently, participants saw one of 130 images showing objects or scenes (depending on the experimental session). About 120 of these images were part of the subsequent learning material and were accompanied by a caption naming the exemplar. Ten additional images, which were not further used during the experiment, were accompanied by an erroneous description. Each stimulus combination was presented for 2.5 s on the computer screen. The participants' task was to press a button whenever they encountered a wrong image-word combination.

Participants learned pairwise associations between 120 verbs and images. The images comprised either objects or scenes (depending on experimental session).

Each trial started with a fixation cross, presented for 1.5 ± 0.1 s. Afterwards, a verb (e.g., "jump") was presented for 1 s on the computer screen and immediately followed by the to-be-associated image for 4 s. Participants were instructed to form a vivid mental image or story linking the verb and the object/scene. After the presentation of the image (4 s), they had to indicate whether the image they had formed was realistic or bizarre. In addition, participants were informed that their memory performance for verb- image pairs would be tested later. The learning block was run twice with varying trial order to reach satisfactory levels of presleep memory performance (as determined in a pilot study).

In order to prevent any testing effect on our behavioral measures of memory consolidation^41,42, only half of the learned verb-image combinations was tested during the presleep memory test. Thus, the presleep memory test included 60 randomly chosen verbs intermixed with 30 new verbs, which were not seen by the participants before ("foils"). Each trial started with a fixation cross, presented for 1.5 ± 0.1 s. After the fixation cross, a verb was presented on the computer screen. After 3 s, participants had to indicate whether the verb was "old" (i.e., part of the learning material) or "new"' (i.e., it was not seen during learning) within the next 10 s. In case of "new" responses, participants immediately moved on to the next trial. In case of "old" responses, participants were required to type a description of the image they had in mind or to type "do not know" in case they could not recall the target image. Trials were coded as correct if (i) the participant typed the same caption as shown during the familiarization phase or (ii) the description unambiguously matched the content of the image

The nap period began at ~1 p.m. Participants had the opportunity to sleep in a laboratory bedroom for 120 min, while their brain activity was monitored using polysomnography).

Twenty minutes after waking up, participants performed another memory test on the remaining 60 study items. This followed the same procedures as the presleep memory test with the exception that new foil verbs were used.

During the localizer task participants were presented with a new set of images comprising objects and scenes (90 objects and 90 scenes, irrespective of session). Each trial started with a fixation cross, presented for 1.5 ± 0.1 s. Subsequently, a randomly chosen image (object or scene) was presented on the computer screen for a minimum of 2.5 and a maximum of 10 s. Each image was presented twice during the task and participants were instructed to indicate whether it was shown for the first ("new") or second ("old") time (mean distance between successive presentations = 8.06, range = 2–33).

By administering the localizer task at the very end of each session, we assured that participants engaged exclusively with a given stimulus category before sleep (objects or scenes, respectively). The rationale of this approach was to keep the category-specific representations during learning as pure as possible, in an effort to bias their reactivation during the subsequent sleep period. However, presenting both stimulus categories during the localizer task ensured that category-specific classifier evidence during sleep would not merely reflect general differences between sessions (e.g., electrode impedances, electrode positions, etc.).

To assess recognition memory performance, we calculated the sensitivity index d′ [i.e., z(Hits)—z(False Alarms)] according to signal detection theory. Proportions of 0 and 1 were replaced by 1/2 N and 1–1/2 N, respectively, with N representing the number of trials in each proportion (i.e., N = 60, see ref. ⁴³).

For associative memory performance we calculated the proportion of correctly recalled images relative to the number of recognized words (i.e., (recalled images/hits)100). To correlate levels of memory retention and reactivation strength we derived a "retention index". We computed the proportion of post-sleep recalled images (out of hits) in relation to presleep memory performance (i.e., (recalled out of hits post-sleep/recalled out of hits presleep)100) and collapsed these measures across sessions. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$*$$\end{document} * \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$*$$\end{document} *

EEG data were preprocessed using the FieldTrip toolbox for EEG/MEG analysis⁷⁵. All data were downsampled to 200 Hz. Subsequently, the localizer and sleep data were segmented into epochs. The temporal range of the epochs was [–1 to 3] s around stimulus onset for localizer trials. As in other studies concentrating on the coordination of SOs and spindles (e.g.,^8,30–32,76) we specifically focused on electrode Cz due to the spatial distribution of both oscillations. Both oscillations show strong presence over central areas, rendering Cz an optimal target zone for investigating concomitant activity of SOs and (fast) spindles. Hence, for the sleep data, slow oscillation—spindle epochs [−2.5 to +2.5 s] time-locked to SO downstates were extracted from channel Cz (for details see Event detection).

Noisy EEG channels were identified by visual inspection, discarded, and interpolated, using a weighted average of the neighboring channels. The localizer data were additionally subjected to an independent component analysis⁷⁷ and ICA components associated with eye blinks and eye movements were identified and rejected.

SOs and sleep spindles were identified for each participant, based on established detection algorithms^8,44. Following standard procedures, all sleep data were re-referenced against linked mastoids for sleep scoring and event detection^74,78,79; please note that the classification results reported in Fig. 2b remained unchanged when using a CAR scheme). SOs were detected as follows: Data were filtered between 0.3–1.25 Hz (two-pass FIR bandpass filter, order = three cycles of the low frequency cut-off). Only movement-free data (as determined during sleep scoring) from NREM sleep stages 2 and 3 were taken into account. All zero-crossings were determined in the filtered signal at channel Cz, and event duration was determined for SO candidates (that is, downstates followed by upstates) as time between two successive positive- to-negative zero-crossings. Events that met the SO duration criteria (minimum of 0.8 and maximum of 2 s, 0.5–1.25 Hz) entered the analysis. 5-s-long segments (±2.5 s centered on the downstate) were extracted from the unfiltered raw signal.

For spindle detection, data were filtered between 12–18 Hz^25,80 (two-pass FIR bandpass filter, order = three cycles of the low frequency cut-off), and again only artifact-free data from NREM sleep stages 2 and 3 were used for event detection. The root mean square (RMS) signal was calculated for the filtered signal at channel Cz using a moving average of 200 ms, and a spindle amplitude criterion was defined as the 75% percentile of RMS values. Whenever the signal exceeded this threshold for more than 0.5 s but less than 3 s (duration criteria), a spindle event was detected. Epochs time-locked to the minimum spindle trough (−2.5 to +2.5 s) were extracted from the unfiltered raw signal for all events. To isolate SO-spindle complexes, we determined for all SOs whether a spindle was detected following the SO (SO downstate + 1.5 s). Finally, SO-spindle events were extracted (−2.5 to +2.5 s with regards to the SO downstate) from the raw signal at channel Cz.

For the analysis of SO-spindle coupling^8,24, we filtered the SO-spindle data in the SO range (0.3–1.25 Hz, two-pass Butterworth bandpass filter), applied a Hilbert transform and extracted the instantaneous phase angle. Next we filtered the same data segments in the spindle range (12–18 Hz two-pass Butterworth bandpass filter), Hilbert transformed the signal and extracted the instantaneous amplitude. Only data points within ±1.5 s were considered to avoid filter-related edge artifacts. Then we detected the maximal sleep spindle amplitude in channel Cz and isolated the corresponding SO phase angle. The preferred phase of SO-spindle coupling was then obtained from averaging all individual events' preferred phases of each participant, and the resulting distribution across participants was tested against uniformity (Rayleigh test, CircStat toolbox⁸¹).

Multivariate classification of single-trial EEG data was performed using MVPA-Light, a MATLAB-based toolbox for multivariate pattern analysis⁸². For all multivariate analyses, a LDA was used as a classifier⁸². Prior to classification, all data were re-referenced using a common average reference (CAR).

For classification within the localizer task, the localizer data were z-scored across all trials for each time point separately. Next, data from both sessions were collapsed and subjected to a principal component analysis (PCA), which transforms the data into linearly uncorrelated components, ordered by the amount of variance explained by each component⁸³. PCA was applied to reduce dimensionality and limit over-fitting⁸⁴ and the first 30 principal components were retained for further analysis^85–87. To quantify whether object and scene representations can be differentiated in the localizer, the classifier was trained and tested to discriminate between object and scene trials. Data were smoothed using a running average window of 150 ms. The EEG channels served as features and a different classifier was trained and tested on every time point. As metric, we used Area Under the ROC Curve (AUC), which indexes the mean accuracy with which a randomly chosen pair of Class A and Class B trials could be assigned to their correct classes (0.5 = random performance; 1.0 = perfect performance). To avoid overfitting, data were split into training and test sets using fivefold cross-validation⁸⁸. Since cross-validation results are stochastic due to the random assignment of trials into folds, the analysis was repeated five times and results were averaged. For statistical evaluation, surrogate decoding performance was calculated by shuffling the training labels 250 times. Resulting surrogate performance values were then averaged, providing baseline values for each participant under the null hypothesis of label exchangeability.

To investigate differential evidence for object vs. scene representations as a function of prior learning during SO-spindle complexes (Fig. 2b), we used the temporal generalization method⁸⁹. Prior to decoding, a baseline correction was applied based on the whole trial ([−0.5 to 3 s] for localizer segments; [–1.5 to 1.5 s] for SO-spindle segments). Next, localizer and sleep data were z-scored across trials and collapsed across sessions. PCA was applied to the pooled wake-sleep data and the first 30 principal components were retained. Localizer and sleep data were smoothed using a running average window of 150 ms. A classifier was then trained for every time point in the localizer data (Fig. 2b, vertical axis) and applied on every time point during SO-spindle complexes (horizontal axis). No cross-validation was required since localizer and sleep datasets were independent. As metric, we again used AUC (see above). For statistical evaluation, surrogate decoding performance was calculated by shuffling the training labels (stemming from the localizer task) 250 times. Again, the resulting performance values were averaged, providing baseline values for each participant under the null hypothesis of label exchangeability

To resolve the topography of diagnostic features, we conducted a "searchlight decoding procedure". In brief, PCA components were projected back to sensor space and the classification procedure was repeated across moving kernels of small electrode clusters, with neighboring electrodes being selected as features [feature number range: 5 to 9]. Classifiers were trained for every time point in the localizer data and applied on every time point during SO-spindle complexes. Finally, classification values were collapsed across our time windows of interest [localizer time: 1000 to 2000 ms; SO-spindle time: 800 to 1200 ms] and tested against chance level (corrected for multiple comparisons across space). A broad cluster of above-chance classification comprising bilateral parietal and occipital areas emerged (p_cluster = 0.004).

Time–frequency analysis of the SO-spindle segments was performed using FieldTrip. Frequency decomposition of the data, using Fourier analysis based on sliding time windows (moving forward in 50 ms increments). The window length was set to five cycles of a given frequency (frequency range: 1–30 Hz in 1 Hz steps). The windowed data segments were multiplied with a Hanning taper before Fourier analysis. Afterwards, power values were z-scored across time [−4 to 4 s]. The longer time segments were chosen to allow for resolving low frequency activity within the time windows of interest [−1.5 to 1.5 s] and avoid edge artifacts.