(C) PLOS One [1]. This unaltered content originally appeared in journals.plosone.org.
Licensed under Creative Commons Attribution (CC BY) license.
url:https://journals.plos.org/plosone/s/licenses-and-copyright

------------



The human insula processes both modality-independent and pain-selective learning signals

['Björn Horing', 'Department Of Systems Neuroscience', 'University Medical Center Hamburg-Eppendorf', 'Hamburg', 'Christian Büchel']

Date: 2022-05

Prediction errors (PEs) are generated when there are differences between an expected and an actual event or sensory input. The insula is a key brain region involved in pain processing, and studies have shown that the insula encodes the magnitude of an unexpected outcome (unsigned PEs). In addition to signaling this general magnitude information, PEs can give specific information on the direction of this deviation—i.e., whether an event is better or worse than expected. It is unclear whether the unsigned PE responses in the insula are selective for pain or reflective of a more general processing of aversive events irrespective of modality. It is also unknown whether the insula can process signed PEs at all. Understanding these specific mechanisms has implications for understanding how pain is processed in the brain in both health and in chronic pain conditions. In this study, 47 participants learned associations between 2 conditioned stimuli (CS) with 4 unconditioned stimuli (US; painful heat or loud sound, of one low and one high intensity each) while undergoing functional magnetic resonance imaging (fMRI) and skin conductance response (SCR) measurements. We demonstrate that activation in the anterior insula correlated with unsigned intensity PEs, irrespective of modality, indicating an unspecific aversive surprise signal. Conversely, signed intensity PE signals were modality specific, with signed PEs following pain but not sound located in the dorsal posterior insula, an area implicated in pain intensity processing. Previous studies have identified abnormal insula function and abnormal learning as potential causes of pain chronification. Our findings link these results and suggest that a misrepresentation of learning relevant PEs in the insular cortex may serve as an underlying factor in chronic pain.

Because signed intensity PEs have a direct conceptual overlap to systems like reward (or relief) and punishment, as well as a large implications for adaptive behavior, we have placed another focus on their cerebral correlates. Employing our novel paradigm, we were also in the position to investigate signed intensity PEs. Focusing on pain, we expected them to be either represented as a distinct part of the anterior insula or within the mid to posterior insula. The former is suggested by inherent differences in salience between the 2 intensities and the latter by the notion that a signed PE necessitates some form of intensity encoding, which has been observed in the dorsal posterior insula [ 22 , 35 , 40 , 41 ].

We expected that SCR resembles unsigned PEs, as SCR is generally considered to reflect arousal-related activation [ 27 – 29 ] and thus the sign of the PE—representing its valence—should not affect it. Concerning fMRI and following results from previous studies using painful stimulation to investigate PEs, particularly in a multimodal context [ 12 , 19 , 22 , 24 , 30 ], we focus on a region of interest including insular and opercular cortices contralateral to stimulation, while also reporting whole brain results. For the processing of pain, the insula is of particular importance given its rich structural and functional (somato)sensory connections, including strong internal connectivity [ 30 – 33 ]: The dorsal posterior insula has been demonstrated to have a preferential involvement in painful stimulation [ 22 , 34 – 37 ]. However, insular processing especially in anterior segments also occurs across sensory modalities, i.e., has been implicated in multimodal integration and the processing of supramodal dimensions like unpleasantness, salience, and PE processing [ 23 , 38 , 39 ]. Concerning PEs, we expected to replicate previous results [ 12 , 19 ] showing the representation of unsigned PEs in the anterior insula. More importantly, we expected that this signal occurs independent of the modality of the US (i.e., both for sound and pain). In agreement with this nonspecific response, we also expected modality PEs to be represented in the anterior insula. However, in this case, the magnitude of the difference could feasibly be weaker or stronger than the intensity PEs: A weaker signal might arise as the intensity—and thus salience and other general aspects—are intendedly not different between the expected and the received US; a stronger signal could arise if the qualitative change between the 2 modalities dominated the cerebral responses.

To further investigate the existence of signed PEs and the modality specificity of unsigned PEs, as well as the underlying neuronal mechanisms, we used a Pavlovian transreinforcer reversal learning paradigm [ 25 , 26 ]. This involves 2 visual stimuli as CS and 2 intensities of painful heat or loud sounds as US (for brevity, these are referred to as “pain” and “sound” forthwith). Across sensory modalities, stimuli were chosen to be roughly comparable in salience as indicated by similar skin conductance responses (SCRs) [ 27 ]. Reversals occurred between US intensity but within US modality (e.g., CS predicting low pain will next predict high pain) or within US intensity but between US modality (e.g., CS predicting loud sound will next predict high pain). Analyses focused on PEs within and across modalities, using advanced surface-based analyses of high-resolution fMRI together with SCRs.

Granted that unsigned PEs resemble a surprise signal, they could plausibly involve similar regions for all surprising events, independent of the stimulus sensory modality. Crucially, the representation of unsigned pain PEs in the anterior insula [ 12 , 19 ] raises the question of whether these are specific to pain or simply related to aversive events. Control conditions using comparator modalities are essential to tease out the unique contributions of painful stimulation to the observed cerebral activity [ 19 , 22 – 24 ]. In such designs, modality PEs can be an important source of variance. Another critical question is therefore to understand which brain regions are active during the processing of such modality PEs, which is an understudied aspect given the relative scarcity of cross-modal experiments.

Previous studies investigating PEs in the context of aversive learning have observed signal changes in the anterior insula related to unsigned PEs [ 6 , 12 , 19 – 21 ]. Unfortunately, in many studies, a signed PE signal is nonorthogonal to stimulus expectation, which poses a problem with a short interval between CS and US, and the low temporal resolution of functional magnetic resonance imaging (fMRI). Consequently, these studies were suboptimal to investigate signed PEs.

However, PEs can also be computed as unsigned [ 10 – 12 ]. An unsigned PE simply indicates the presence of an unexpected event regardless of its valence. Unsigned PEs are therefore conceptually related to constructs like surprise or salience and may contain information concerning the urgency of behavioral change [ 13 ]. Computational models of learning can include either type of PE or both [ 4 , 10 , 14 – 16 ]—for example, the Pearce–Hall model incorporates the unsigned PE as a factor to increase the learning rate after highly incongruent (surprising) events [ 14 , 17 ], whereas a hybrid model contains both terms [ 10 , 17 , 18 ].

Apart from its role in signaling tissue damage, pain is increasingly considered to be a preeminent teaching signal [ 1 , 2 ] in the context of reinforcement learning models [ 3 ]. For example, delta rule learning models in classical fear conditioning, such as the Rescorla–Wagner model [ 4 ], almost exclusively employ pain as unconditioned stimulus (US). In this and similar models, the value of predictive cues (conditioned stimuli, CS) is updated by the difference between the expected and the experienced outcome, i.e., a prediction error (PE). In this case, the PE needs to be signed and signals the direction of the difference between expectation and event, i.e., whether the outcome is better or worse than expected. In the case of an aversive event like painful stimulation, this is relevant for shaping future behavior. Reinforcement learning particularly relies on these valences, and different neuronal correlates have been reported for aversive compared to appetitive PEs [ 5 – 8 ]. This has important clinical implications, as pathological learning mechanisms [ 1 , 9 ] have been reported in chronic pain.

Because the signed intensity PE effect is calculated by fitting a line through 3 predictive values (negative intensity PE meaning intensity lower than expected or positive intensity PE meaning intensity higher than expected, as compared to intensity as expected), the question arose whether the effect was constituted differently by negative or positive PEs. To ascertain this in an adjunct analysis, we set up a general linear model using both signed PEs as separate regressors; we then obtained the z-values of the pain and sound peak, respectively. For pain (XYZ MNI 36.4/−17.3/15.8), values were z = 2.582 (for negative intensity PE) and z = 3.053 (for positive intensity PE); for sound (XYZ MNI 49.4/−16.6/−13.4), values were z = 1.300 and z = 3.922, respectively. This indicates a comparable contribution of the negative and positive intensity PE component for pain, while sound activation more strongly driven by the positive intensity PE (louder than expected) component. Conjunction analyses combining negative and positive intensity PEs, performed separately for either modality, yielded p[uncorr.] = 0.00484 for pain, p[uncorr.] = 0.04468 for sound, which were not significant after correction for multiple comparisons.

Contrary to uIntPEs, circumscribed activation was detected for pain sIntPEs without any overlap with sound sIntPEs. Peak activation is located in the dorsal posterior insula (dpIns 1 ). For sound, several clusters in the anterior insula (e.g., aIns 3 ) were found, as well as middle temporal gyrus (MTG 1 ). Activations are overlaid on an average brain surface; for display purposes, activations in the whole brain lateral view are thresholded at p[uncorr.] < 0.001. The black line in the zoomed-in view delineates the region of interest and includes activations within the small volume FWE corrected at p[corr.] < 0.05. In the fMRI signal bar graphs, black rectangles highlight the individual regressors used for analysis. fMRI signal labels refer to the regressors used for each modality: “main” for main effects of modality, “ratings” for behavioral ratings, “modPE” for modality PEs, “uIntPE” for unsigned intensity PEs, and “sIntPE” for signed intensity PEs. Data used to produce the figure can be found at https://www.doi.org/10.17605/OSF.IO/7JBV3 . fMRI, functional magnetic resonance imaging; FWE, Family-Wise Error; PE, prediction error.

After ascertaining the effects for unsigned PEs for both intensity and modality, the final question for our fMRI data referred to differences and commonalities following signed intensity PEs, i.e., correlations of brain activation with higher than expected intensity ( Fig 9 ). For pain, we observed an activation in the dorsal posterior insula (XYZ MNI 36.4/−17.3/15.8, T = 4.0, p[corr.] = 0.023). The dorsal posterior insula is an area considered of fundamental importance for the processing of pain intensity [ 22 , 35 , 42 ]. For sound itself, the peak activation was observed outside the region of interest, in the middle temporal gyrus (XYZ MNI 49.4/−16.6/−13.4, T = 4.1, p[uncorr.] = 2 × 10 −05 ) (see Fig 6 ). Within the region of interest, sound-related activation was found in the anterior insula (XYZ MNI 36.7/11.0/−10.2, T = 4.2, p[corr.] = 0.015). Notably, these are adjacent to the unsigned PE activations (Figs 6 – 8 ). All signed intensity PE peaks, both for pain and sound, show no significant representation of a signed PE in the other modality (see opposite sIntPE fMRI signals in Fig 9 ). Consequently, a conjunction analyses revealed no overlap.

The fMRI signal plot shows that the peak in the anterior insula (aIns 1 ) encodes PEs for every contrast included in the conjunction. Activations are overlaid on an average brain surface; for display purposes, activations in the whole brain lateral view are thresholded at p[uncorr.] < 0.001. The black line in the zoomed-in view delineates the region of interest and includes activations within the small volume FWE corrected at p[corr.] < 0.05. In the fMRI signal bar graph, black rectangles highlight the regressors used for analysis; solid line indicates analysis with the respective individual regressor, and dashed line indicates conjunction analysis. fMRI signal labels refer to the regressors used for each modality: “main” for main effects of modality, “ratings” for behavioral ratings, “modPE” for modality PEs, “uIntPE” for unsigned intensity PEs, and “sIntPE” for signed intensity PEs. Data used to produce the figure can be found at https://www.doi.org/10.17605/OSF.IO/7JBV3 . fMRI, functional magnetic resonance imaging; FWE, Family-Wise Error; PE, prediction error.

As a next step, we wanted to more formally assess the apparent overlap between both types of unsigned PEs. To do so, we simply computed the conjunction between unsigned intensity and modality PE ( Fig 8 ). This analysis corroborated the anterior insula peak determined by separate analyses above. Furthermore, activation extended dorsally through the middle frontal gyrus and also included medial prefrontal areas adjacent to the dorsal anterior cingulate cortex.

As with unsigned intensity PEs, peak activation following modality PEs in either modality is located in the anterior insula (aIns 1 ) and is largely subsumed in the common activation. Activations are overlaid on an average brain surface; for display purposes, activations in the whole brain lateral view are thresholded at p[uncorr.] < 0.001. The black line in the zoomed-in view delineates the region of interest and includes activations within the small volume FWE corrected at p[corr.] < 0.05. Peaks are shown for the small volume only. See Supporting information for peak positions in whole brain ( S10 Fig ) and brain volume slices ( S11 Fig ). In the fMRI signal bar graph, black rectangles highlight the regressors used for analysis; solid line indicates analysis with the respective individual regressor, and dashed line indicates conjunction analysis. fMRI signal labels refer to the regressors used for each modality: “main” for main effects of modality, “ratings” for behavioral ratings, “modPE” for modality PEs, “uIntPE” for unsigned intensity PEs, and “sIntPE” for signed intensity PEs. Data used to produce the figure can be found at https://www.doi.org/10.17605/OSF.IO/7JBV3 . fMRI, functional magnetic resonance imaging; FWE, Family-Wise Error; PE, prediction error.

Following these 2 observations, we proceeded to investigate the nature of the overlap between the 2 types of PE. Like with unsigned intensity PEs, we observed widespread activation following each modality PE separately ( Fig 7 ). Likewise, all unimodal activation is subsumed in the conjunction analysis, which indicates a large dorsal anterior insula cluster in our region of interest (XYZ MNI 32.3/22.4/−3.4, T = 5.4, p[corr.] = 5 × 10 −05 ). Beyond this region, widespread common activation is observed, for example, in the superior parietal lobule, precuneus, temporoparietal junction, middle frontal gyrus and frontal operculum, and medial orbital gyrus ( S10 Fig ).

Two aspects were of particular interest to us considering unsigned intensity PE results: First, that brain activation related to unsigned intensity PEs ( Fig 6 ) was distinct from the rating-related activation ( Fig 5 ). Second, the fMRI signal of the common activation in the anterior insula clearly indicated that modality PEs are likewise encoded in this area.

In both modalities, widespread activation was observed. However, conjunction analyses revealed that the majority of the observed activation actually overlapped between the modalities (green in Fig 6 ). The anterior insula constituted the dominant cluster of this overlap, with symmetric bilateral peaks (XYZ MNI = 34.6/23.5/−1.5, T = 5.8, p[corr. wb.] = 1 × 10 −04 ); whole brain significant frontal (medial and lateral), temporal, and parietal activation was also observed ( S8 Fig ).

Peak activation following either modality is located in the anterior insula (aIns 1 ) and is subsumed in the common activation. Activations are overlaid on an average brain surface; for display purposes, activations in the whole brain lateral view are thresholded at p[uncorr.] < 0.001. The black line in the zoomed-in view delineates the region of interest and includes activations within the small volume FWE corrected at p[corr.] < 0.05. See Supporting information for peak positions in whole brain ( S8 Fig ) and brain volume slices ( S9 Fig ). In the fMRI signal bar graph, black rectangles highlight the regressors used for analysis; solid line indicates analysis with the respective individual regressor, and dashed line indicates conjunction analysis. fMRI signal labels refer to the regressors used for each modality: “main” for main effects of modality, “ratings” for behavioral ratings, “modPE” for modality PEs, “uIntPE” for unsigned intensity PEs, and “sIntPE” for signed intensity PEs. Data used to produce the figure can be found at https://www.doi.org/10.17605/OSF.IO/7JBV3 . fMRI, functional magnetic resonance imaging; FWE, Family-Wise Error; PE, prediction error.

Having ascertained strictly stimulus-related effects, our next analysis included an investigation of unsigned intensity PEs within and between either modality ( Fig 6 ). The guiding question here was whether any differences and commonalities between the modalities would emerge. Since we used the actual expectation queried from participants, “prediction error” here means that participants explicitly expected one intensity but received the other. Consequently, the unsigned PE implies some extent of surprise.

Next, we tested for fMRI responses correlated with stimulus perception, i.e., pain and sound VAS ratings ( Fig 5B and 5C ). For pain ratings, associations arose in the dorsal posterior insula (XYZ MNI = 35.2, y = −17.4, z = 18.6, T = 7.2, p[corr.] = 1 × 10 −09 ). For sound ratings, we observed a peak directly adjacent to the small surface (XYZ MNI 59.8, y = −33.9, z = 5.4, T = 4.8, p[corr.] = 0.016). Common activation between pain and sound ratings peaked in the central operculum (XYZ MNI 53.2, y = −2.7, z = 8.9, T = 4.8, p[corr.] = 0.001). Of note, the central operculum peak (CO 2 in Fig 5C ) is located slightly anterior to that found for the modality (main effect) conjunction (CO 1 in Fig 5A ) but shows barely any sound modality activation; conversely, peak aIns1 indicates that no rating effects are encoded here. See Supporting information for additional activations ( S4 and S6 Figs).

Activations are overlaid on an average brain surface; for display purposes, activations in the whole brain lateral view are thresholded at p[uncorr.] < 0.001. The black line in the zoomed-in view delineates the region of interest and includes activations within the small volume FWE corrected at p[corr.] < 0.05. Peaks are shown for small volume only; bar plots show beta weights of BOLD activation obtained from a general linear model (see Materials and methods ) from the respective peaks. See Supporting information for peak positions in whole brain ( S4 and S6 Figs) and brain volume slices ( S5 and S7 Figs). (A) Differential and shared activation following painful heat stimulation and loud sound stimulation. Peak activation following heat is located in (peri)insular areas contralateral to stimulation, namely the dorsal posterior insula (dpIns 1 ), and extending through the central and parietal opercula. Peak activation following sound is located in the superior temporal gyrus. Common activation (green) is located in the central operculum (CO 1 ) and dorsal anterior insula (aIns 1 ), among other regions. (B) fMRI signal (arbitrary units) for peaks detected in panel A (US onset effects) or C (parametric modulation by ratings). Black rectangles highlight the regressors used for analysis; solid line indicates analysis with the respective individual regressor, and dashed line indicates conjunction analysis. fMRI signal labels refer to the regressors used for each modality: “main” for main effects of modality, “ratings” for behavioral ratings, “modPE” for modality PEs, “uIntPE” for unsigned intensity PEs, and “sIntPE” for signed intensity PEs. (C) Differential and shared correlations with pain ratings (for heat) and unpleasantness ratings (for sound). Activation correlated with pain ratings is focused on the dorsal posterior insula (dpIns 1 ). Activation correlated with sound ratings is focused on the superior temporal gyrus. Conjunction activation peaks in central operculum (CO 2 ) and precentral gyrus. Data used to produce the figure can be found at https://www.doi.org/10.17605/OSF.IO/7JBV3 . BOLD, blood oxygenation level dependent; fMRI, functional magnetic resonance imaging; FWE, Family-Wise Error; PE, prediction error; US, unconditioned stimulus.

We first obtained an overview of modality-related effects ( Fig 5A and 5B ) and rating-related effects ( Fig 5B and 5C ) of the US. All locations are reported using Montreal Neurological Institute (MNI) coordinates (XYZ MNI ). As expected, heat stimulation was followed by larger activation in widespread insular and opercular areas, with the highest peak in the dorsal posterior insula (XYZ MNI 35.5/−17.9/21.4, T = 12.2, p[corr.] ≈ 0). Activation following sound stimulation peaks in the superior temporal gyrus (XYZ MNI 65.9/−23.8/10.2, T = 25.7, p[corr. wb.] ≈ 0), just outside the extended insula mask. Notably, a conjunction of both heat and sound main effects shows activation in the central operculum (XYZ MNI 53.0/−10.3/15.1, T = 8.3, p[corr.] = 8 × 10 −13 ), dorsal anterior insula (XYZ MNI 37.6/18.4/−7.0, T = 5.6, p[corr.] = 2 × 10 −05 ), and several regions in between peaks for both modalities.

Fig 4D shows the average perireversal trial effect on SCR, over all US. It shows a large increase in SCR during both modality and intensity reversals; note that this analysis does not consider actual participant expectation, just the position related to the reversal trial. SCR is highest during the reversal trial and rapidly reaches a lower plateau even one trial later. Comparing the prereversal trial to immediate postreversal (trials −1 to +1), SCR is not significantly different if a modality reversal occurred (p = 0.54704); this is also the case if an intensity reversal occurred (p = 0.071164).

In 4 consequent analyses, we investigated differences in SCR following PEs in all US separately, meaning that all intensity PEs are now signed. Results indicate that the intPE > noPE effect of the global analysis is driven by this contrast in the high sound US (light blue bars, t[1119] = 4.732, p = 3 × 10 −6 ; random intercept linear mixed model); it does not reach significance following any other US. Conversely, modality PEs are followed by larger SCR in all US (all modPE > noPE p < 0.001; smallest effect modPE > intPE t[1090] = 2.045, p = 0.041079).

Notably, we performed an adjunct analysis on whether the direction of intensity PEs (i.e., signed intensity PEs) had an impact. We obtained mean SCR differences per participant between no PE and intensity PE trials for each modality and intensity separately, thereby accounting for higher intensity-related base SCRs; next, we contrasted these (now signed) PE-related differences between the low and high intensity. For pain, results indicate no effect (PE-related SCR difference for low pain mean ± SE 0.036 ± 0.052, for high pain 0.0922 ± 0.0622, paired t test t[36] = −0.725, p = 0.4731), while for sound, a more ambiguous yet nonsignificant result arose (PE-related SCR difference for low sound mean ± SE 0.060 ± 0.054, for high sound 0.199 ± 0.054, paired t test t[35] = −1.931, p = 0.0616).

Further investigating SCR differences following PEs, we first distinguished SCR when participants correctly predicted the US from trials when either an intensity PE or modality PE was made ( Fig 4C ). The following statistics include all trials—not just reversals—where an incorrect prediction was made. As shown in the first block (gray bars), over all US and controlling for modality and intensity, SCR following unsigned intensity PEs are larger than those following no PE (intPE > noPE, t[4397] = 4.336, p = 2 × 10 −05 ), while SCR following modality PEs are even larger (modPE > noPE, t[4397] = 12.345, p = 2 × 10 −34 ; modPE > intPE, t[4397] = 6.398, p = 2 × 10 −10 ).

All plots are based on log- and z-transformed data. (A) SCR in relation to US onsets, by US modality/intensity. Note the differences in latencies between the 2 modalities (pain in red/yellow has a later onset, sound in dark blue/light blue earlier), which determined the response windows used for mean SCR calculation in panel b. (B) Mean SCR by US, calculated within each modality’s response window. On average, SCR is not significantly different between modalities; differences arise between intensities, and in the interaction of modality and intensity (see text for parameters). (C) Mean SCR by US and PE type. Over all modalities and intensities, differences arise between each PE type. Within specific modality/intensity combinations, differences between no PEs and intensity PEs only arise in the high sound condition. (D) Mean SCR in and around reversal trials. Within trials, data are pooled over all modalities, intensities, and expectations, i.e., does not consider whether participants correctly predicted the subsequent stimulus. The dashed vertical line indicates contingency reversal, with relative trial number 0 as the reversal trial. SCR rises sharply after reversal, but quickly adapts postreversal to a stable level. Data used to produce the figure can be found at https://www.doi.org/10.17605/OSF.IO/7JBV3 . intPE, intensity prediction error; modPE, modality prediction error; noPE, correct prediction; PE, prediction error; SCR, skin conductance response; US, unconditioned stimulus.

The major question concerning SCR results were whether any differences between the US arose and how the different PE types would be reflected in this psychophysiological measure of nonspecific characteristics or processes like arousal, salience, or surprise. SCR following sound has a faster onset than that following heat pain stimuli ( Fig 4A ; see Materials and methods concerning the different response windows). The average amplitude of pain-related SCR was higher than the average of sound-related SCR, but this difference only showed a trend toward significance (main effect modality, t[4399] = −1.7228, p = 0.08499; random intercept linear mixed model predicting each participant’s and each trial’s SCR). Instead, the difference is subsumed by a larger difference between low and high stimuli in the pain modality, as compared to that in the sound modality (modality*intensity, t[4399] = −2.9739, p = 0. 0029567). On average, higher stimuli lead to larger amplitude as well (main effect intensity, t[4399] = 8.2743, p = 1.7 × 10 −16 ). Investigating this difference only in correctly predicted trials shows a similar effect on SCR (modality, t[2674] = −1.4379, p = 0.1506; intensity, t[2674] = 8.0081, p = 2 × 10 −15 ; modality*intensity, t[2674] = −4.6669, p = 3 × 10 −6 ) ( S3 Fig , S1 Table ).

Using a linear mixed effects model with reversal type (modality versus intensity reversals) and trial as categorical predictors, we find no mean difference of reversal type (p = 0.94) but of trial (p = 1.2 × 10 −7 ). Post hoc contrasts indicate that the trial effect is driven exclusively by the reversal trial (compared to all other trials, all p < 1 × 10 −169 ), whereas none of the nonreversal trials is different from each other (all p > 0.1).

The next behavioral question was whether the participants learned the CS–US contingencies. Fig 3B depicts mean performance in predicting the US currently associated with the CS, in relation to the reversals of the association. Combining reversal types and comparing performance at the single trials prior to reversal, at reversal, and after reversal, we find prereversal performance to be above chance level (t[79] = 13.8, p ≈ 0), at reversal performance below chance (t[79] = −15.9, p ≈ 0), and postreversal performance back above chance (t[79] = 19.5, p ≈ 0).

(A) Results for low and high unconditioned pain and sound stimuli; aggregate ratings of all pain and sound trials. Circles with error bars show the mean ± standard errors over all participant means. Participant means are displayed as smaller circles. Violin plots aggregate over participant means. The gray dashed line is the “intended” rating as per calibration (VAS 25 for low and VAS 75 for high intensities). (B) Performance pre- and postreversals, aggregated over all participants. Circles indicate the performance during (peri)reversal trials, first averaged within and then between participants (mean ± standard errors). The dashed horizontal line marks chance level (25%, i.e., 1 of 4 options). The dashed vertical line indicates contingency reversal, with relative trial number 0 as the reversal trial. Note that no difference arose between trials preceding and following modality versus intensity reversals (also see Fig 2 for aspects concerning contingency reversals). Furthermore, the steep increase in performance after trial number 0 indicates, on average, rapid learning of the new contingency. Data used to produce the figure can be found at https://www.doi.org/10.17605/OSF.IO/7JBV3 .

The first question concerning the behavioral data was whether ratings corresponded to the calibrated intensities (supposed to yield VAS of 25 and 75, respectively). Actual low pain ratings were at 15.4 ± 14.8 VAS and high pain ratings at 66.8 ± 21.3 VAS; low sound ratings were at 29.2 ± 21.0 VAS and high sound ratings at 63.3 ± 19.4 VAS ( Fig 3A ; see S2B Fig for individual ratings per participant).

(A) Set of CS; 2 were randomly selected for each participant (constraint: stimuli in row 2 could never both be selected due to high similarity). (B) Possible US associated with a CS at any particular trial (low pain, high pain, low sound, and high sound). Arrows indicate possible reversals; notably, no combined intensity and modality (cross)reversals occurred. (C) Example for contingencies of CS1 (black solid line) and CS2 (white solid line) for their 32 trials per session each. Vertical dotted lines indicate reversals, with light dotted lines for modality reversals and dark dotted lines for intensity reversals. (D) Example for an actual trial sequence of 64 trials with interspersed CS1 (black diamonds) and CS2 (white diamonds) and their associated US (rows). Data used to produce the figure can be found at https://www.doi.org/10.17605/OSF.IO/7JBV3 . CS, conditioned stimuli; US, unconditioned stimuli.

(A) Overall structure of the experiment. Calibration took approximately 15 minutes, each session approximately 20 minutes. (B) Devices used for heat stimulation (thermode) and sound stimulation (headphones), with standardized locations on the left arm for pain calibration and either of the 2 experimental sessions. (C) Trial structure with associated durations. After displaying CS, participants were asked to choose which US they expected to follow. The US was then applied and rated in terms of its painfulness (for pain)/unpleasantness (for sound). EDA, electrodermal activity; CS, conditioned stimuli; US, unconditioned stimuli.

In 2 sessions with 64 trials each, 47 participants learned associations of 2 CS (fractal pictures) with individually calibrated US (2 painful heat intensities and 2 loud sound intensities) ( Fig 1A and 1B ). In each trial, either CS appeared, followed by symbols of all 4 US, from which participants selected the US they expected ( Fig 1C ). One of the US was then applied. CS–US associations were deterministic, but importantly, associations frequently reversed and had to be relearned over the course of the experiment ( Fig 2 ). Reversals occurred unannounced after a randomized number of trials. Reversals could occur along the modality dimension or the intensity dimension, but not both simultaneously (e.g., no low heat to high sound reversals). See Materials and methods and S1 Fig for further details concerning design and protocol.

Discussion

Using a Pavlovian learning paradigm with frequent reversals within and across aversive modalities in combination with SCR recordings and high-resolution fMRI, we were able to investigate signed and unsigned representations of PEs in the human brain. The data showed an unsigned representation of intensity PEs in the anterior insula indistinguishable for pain and aversive sounds, supporting a role of the anterior insula in coding unspecific arousal or salience. In addition, the same part of the anterior insula also strongly activated for PEs concerning stimulus modality. Most importantly, we could identify a circumscribed part of the dorsal posterior insula representing a signed PE for pain only, collocated with areas processing pain intensity per se.

A signed representation of an intensity PE for pain is a crucial teaching signal in reinforcement learning, as it is important to dissociate a low threat from a high threat stimulus. Such a representation for pain could plausibly be located in an area adjacent the anterior insula part representing unsigned intensity PEs and modality PEs. Alternatively, this representation could be located closer to representations of pain intensity: Coding of signed intensity PEs within areas coding for stimulus intensity per se was observed using a similar Pavlovian transreinforcer paradigm in the olfactory domain [26]. Indeed, our data show that a signed intensity PE for pain is represented in a part of the dorsal posterior insula [22,35]. Interestingly, we also identified a similar representation of a signed intensity PE for aversive sounds in or adjacent to primary auditory cortices [43,44], namely the middle temporal gyrus and temporal operculum. It also seems indicative of the more general involvement of the insula in pain perception [45] that the signed intensity PE in pain has little to none sound-related activation at all, whereas the signed intensity PE in sound includes some pain intensity-related activation.

We have replicated findings concerning pain-related activation in the dorsal posterior insula/parietal operculum and sound-related activation in the superior temporal gyrus [22]. Previously, these areas showed a clear main effect of pain and sound stimulation, respectively, but a crucial pain and sound rating-related increase in activation that is shallower or absent in nonnoxious intensities. In contrast to the previous study, we see a stronger correlation of the BOLD response to sound ratings, possibly owing to the higher intensities employed here.

Also, in agreement with previous studies, we observed an unsigned intensity PE for pain in the anterior insula [12,19,21]. The novel contribution is the fact that stimuli in different modalities (i.e., pain and aversive sounds) lead to the same activations in the anterior insula, with similar magnitudes. To our surprise, strong activation in the anterior insula was also observed for modality PEs (expect pain and receive sound and vice versa). fMRI signals for unsigned intensity PEs and modality PEs were very similar in magnitude. This disconfirms our hypothesis that at the level of the insula, modality PE carries less difference in salience between the expected and the real outcome, as compared to an unsigned intensity PE. Rather, it seems that surprise from unexpected sensory modalities is as much a source of anterior insula activation as from unexpected intensities. Notably, activation following modality PEs in either modality is characterized by the overlap with the other, with little differential involvement of structures dedicated to either modality or discriminating functions such as spatial orientation (also see S10 and S11 Figs). Instead, differences appear to be a matter of degree in the spread of activation, without substantial involvement of unimodally different structures. Our findings suggest that modality and unsigned intensity PEs are largely modality neutral and support findings that the anterior insula is richly interconnected part of the salience and attentional network involved in decision-marking, error recognition, and generally the guidance of flexible behavior [31,46–49]. Indeed, the large-scale activation following modality PEs and unsigned intensity PEs themselves does not correspond to any single network description, but seems to involve all of the above; possibly, different dynamics are at play over the course of the stimulation, which do not allow for the disentangling of single networks. In fact, recent meta-analytic evidence of resting-state functional connectivity points to the existence of a pain-related network centered on the anterior insula [50]. The activation associated with both pain-related (posterior insula) activation and that associated with PE-related (anterior insula) activation correspond well with connectivity gradients observed along the posterior–anterior axis [51–53].

Regions coding for aversiveness per se should exhibit overlaps in the respective rating-related activation across modalities. In the current data, this is the case, e.g., in the central operculum (with high proximity to the anterior insula) and—as per whole brain analysis—the anterior cingulate cortex (Fig 5B and 5C, S6 Fig). This overlap, while relatively sparse, is in line with previous results using similar supramodal paradigms [19,22,23] and correspond to known correlates of suffering [54]. As has been pointed out before [22,55], studies exhibiting a large modality-independent activation predominantly use comparatively brief stimuli [23,56], with functional imaging results potentially emphasizing the salience-/orienting-related activation. In the future, it could be worthwhile to consider remaining differences between the modalities, such as the focus on predicting target stimuli instead of passive perception. For example, spatial location is a parameter relevant for the painful stimuli only, and laterality effects or even stimulation more proximal to the ear could explain further nonspecific variance in the cerebral signatures.

Furthermore, as exemplified by the predominance of unsigned over signed PEs, aversiveness is confounded with salience: It is possible that the intentionally similar salience between the 2 inherently aversive modalities had overshadowed some modality-specific and supramodal mechanisms. It seems promising that future protocols include equisalient appetitive conditions to tease out these mechanisms. For example, the involvement of different structures for reward and punishment has been demonstrated in studies using instrumental learning tasks [57,58] including intracranial recordings [30] or lesion studies [59]. In theory, the paradigm presented here also includes similar aspects, such as relief in the form of negative PEs (when more pain is expected, receiving less may be experienced as rewarding). Still, specifics of the protocol may have prevented more widespread overlaps between the modalities even though such supramodal parallels exist. Such specifics include a presumptive focus on contingencies as opposed to passive sensory perception or differences related to Pavlovian versus instrumental learning designs incorporating different decision-making processes, experiences of reward and punishment, and forms of feedback.

One of the strengths of the current paradigm include the parallel assessment of SCR, behavioral ratings for both expectation and outcome, as well as fMRI recordings which allowed us to investigate PEs in a multimodal fashion. Previous studies investigated PEs using cue-based pain paradigms [12,19,21,60]. In these paradigms, a cue predicts a stimulus intensity with a certain probability. However, the probability also determines the number of trials in which a PE occurs. This can lead to unbalanced designs in which certain PEs occur much more frequently than others. In addition, the fixed association of a specific cue with an outcome risks that specific features of the cue influence PE processing. Adopting a Pavlovian transreinforcer paradigm ameliorates these shortcomings and requires frequent relearning of contingencies and thus generates frequent PEs [25,26]. By defining a Markovian transition structure, we also controlled the nature of reversals; we confined our experiment to within-intensity/between-modality and between-intensity/within-modality reversals. Finally, introducing 2 CS in our task increased task difficulty. Even though we would have desired a more gradual learning curve for more fine-grained PEs, we have to attest a quick average learning performance (Fig 3B). Going forward, there are several options to increase task difficulty, such as using probabilistic instead of deterministic contingencies, adding intensities, or reducing discriminability.

We explicitly included expectation ratings, which allowed us to use the difference between the US and its expectation as a rating-derived PE [26]. Compared to model-derived PEs, this can account for within-subject differences in learning and can also capture PEs in erratic behaviors difficult to model in formal reinforcement learning models.

Although we aimed to perfectly match salience between stimulus modalities, high-intensity painful stimuli lead to higher SCR activation compared to low pain or either sound intensity (Fig 4), even though average SCR amplitudes between modalities were not statistically different. Technically, this is related to the fact that we were not able to increase sound pressure levels above a certain level [61] to avoid harm for the volunteers; this is a core obstacle when considering more sophisticated cross-modality matching procedures. However, the fMRI signal changes in the anterior insula for unsigned intensity PEs were similar for pain and sound, suggesting that the residual differences in SCR did not affect our results (Figs 6–8). In addition, previous accounts [62] have indicated that higher salience enhances memory performance. We tested this and observe no such effect: Learning performance did not substantially differ between any of the US groups (S14 Fig).

It is known that SCR predominantly shows arousal and similar effects, but is relatively insensitive concerning valence [27–29,63,64]. Here, SCR following unsigned or signed intensity PEs was little different from SCR following no PEs, while SCR following modality PEs was much higher. This might indicate that modality PEs provide a highly salient teaching signal even in the absence of intensity differences (S3 Fig).

Due to the task-inherent structure, signed pain intensity PEs can be correlated with actual pain ratings [57]. This collinearity can be remedied by orthogonalizing regressors in the general linear model used for fMRI analysis. However, this arbitrarily assigns the shared variance to either of the 2 correlated regressors, depending on the order of the serial orthogonalization [65]. Therefore, we refrained from any orthogonalization in our analysis and thus only reveal areas that show unique variance tied to the regressors, including the signed intensity PEs for pain. This may have been a contributing factor to the relatively sparse activation following signed intensity PEs—this limitation could also be addressed by increasing task difficulty (see Discussion above).

At most, the clear spatial dissociation of intensity PEs for pain and sounds furthermore indicates a specificity of the signal; at least, it stands in marked contrast with the large overlap of activation for unsigned intensity and modality PEs in the anterior insula. Powerful learning models can utilize both a signed PE to update their predictions and an unsigned PE to update their learning rate [10,17,18]. Our results provide a neuronal basis for these models as we were able to reveal the simultaneous representation of both a signed and unsigned PE signal in spatially distinct regions of the insula.

In conclusion, our data provide clear evidence of anterior insula-centered, modality-independent unsigned PEs, not only concerning mismatched stimulus intensities across modalities, but also across sensory modalities themselves. Equally important, signed intensity PEs were associated with activation in or adjacent to sensory areas highly dedicated to unimodal processing. Neuronal data from both sources are the basis for reinforcement learning and further enhance our understanding of the functional synergies within the insula. Importantly, pathological learning mechanisms [1,9] and abnormalities in anterior insula-related function have been reported in chronic pain [50,66]. Our data therefore offers the possibility that a misrepresentation of PEs constitutes a potential mechanism in pain persistence.

[END]

[1] Url: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3001540

(C) Plos One. "Accelerating the publication of peer-reviewed science."
Licensed under Creative Commons Attribution (CC BY 4.0)
URL: https://creativecommons.org/licenses/by/4.0/


via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/

You are viewing proxied material from magical.fish. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.