(C) PLOS One
This story was originally published by PLOS One and is unaltered.
. . . . . . . . . .



The contribution of the basal ganglia and cerebellum to motor learning: A neuro-computational approach [1]

['Javier Baladron', 'Department Of Computer Science', 'Chemnitz University Of Technology', 'Chemnitz', 'Departamento De Ingeniería Informática', 'Universidad De Santiago De Chile', 'Santiago', 'Julien Vitay', 'Torsten Fietzek', 'Fred H. Hamker']

Date: 2023-05

Motor learning involves a widespread brain network including the basal ganglia, cerebellum, motor cortex, and brainstem. Despite its importance, little is known about how this network learns motor tasks and which role different parts of this network take. We designed a systems-level computational model of motor learning, including a cortex-basal ganglia motor loop and the cerebellum that both determine the response of central pattern generators in the brainstem. First, we demonstrate its ability to learn arm movements toward different motor goals. Second, we test the model in a motor adaptation task with cognitive control, where the model replicates human data. We conclude that the cortex-basal ganglia loop learns via a novelty-based motor prediction error to determine concrete actions given a desired outcome, and that the cerebellum minimizes the remaining aiming error.

In this study, we aimed to better understand motor learning using a neuro-computational approach. While previous work emphasized different learning regimes of the various components, the main novelty of our study is the interplay of its components. Notably, we show that the model accounts well for motor adaptation data of experiments that involve a cognitive strategy.

Funding: This study was funded by German Research Foundation (DFG, 416228727) - SFB 1410 Hybrid Societies awarded to F.H. Parts of the salary for Javier Baladron and Torsten Fietzek were covered by the fund above. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

We introduce here a systems-level computational model that includes the basal ganglia, motor cortex, cerebellum, and brainstem. The focus of our study is the potential division of labor and learning in motor coordination, particularly in reaching and motor adaptation tasks. However, we do not aim to develop a rigorous implementation of neuro-biological details for each subsystem, given the still relatively poor understanding of the neural circuits in these brain parts.

Another part of the brain heavily involved in motor execution are the central pattern generators (CPGs) in the brainstem and spinal cord [ 49 – 52 ], that are not only involved in locomotion but also reaching [ 53 – 55 ]. In mice, stimulation of brainstem neurons in the lateral rostral medulla leads to complex forelimb reaching and grasping behavior, where different populations of neurons trigger different patterns of behavior [ 56 ]. The large diversity of specialized motor-related neurons in the brainstem integrates information from the cortex, thalamus, cerebellum, and basal ganglia [ 57 ]. CPGs became very popular in the research field of neurorobotics leading to sophisticated demonstrations of complex motor actions [ 58 – 61 ]. However, CPGs need some form of more high-level control when recruited for goal-directed behavior.

In addition to the rather theoretical frameworks discussed above, the interaction of brain areas relevant to motor tasks can be explored by means of computational models. Very few computational models have included both, the cerebellum and the basal ganglia. An early approach [ 45 ], in which the cerebellar and basal ganglia circuitry was modeled by means of simple feedforward neural networks and combined with the DIRECT-model for motor reaching [ 46 ], aimed at explaining the behavioral difference between Parkinsonian patients and controls in a motor adaptation task. According to this model, when learning in the basal ganglia is deactivated to mimic the neurodegeneration of dopaminergic nigrostriatal neurons, continuous erratic movements occur. This compares well to data from patients who show only a crude adaptation. Recently, Caligiore et al. [ 36 ] designed a basal ganglia-cerebellar-thalamo-cortical system to explain the development of tics in Tourette. Although the model can recreate changes in the firing rates of cells in animal models of the disease, it does not implement a motor task. Capirchio et al., [ 47 ] used a system-level model to simulate a reaching task, which requires to reach three targets from a home position. In this model, the basal ganglia are represented by an actor-critic reinforcement learning account and the cerebellum as a feed-forward perceptron. Lesions to the cerebellum part showed effects observed in patients with cerebellar ataxia. Another recent model by Todorov et al. [ 48 ] focused on the role of the cerebellum and basal ganglia in motor adaptation. The basal ganglia implement action selection of a cortical motor program representing a movement trajectory. It is trained by the difference of successive reward prediction errors to support learning when performance improved and suppress the recent action when performance decreases. The cerebellum computes a small correction to the cortical motor program by means of a neural network trained with error backpropagation. In their model, any cerebellum-induced change in performance activates learning in the basal ganglia creating a credit assignment problem about the source of a gain or decline in performance. They therefore propose the existence of a critic somewhere in the brain that determines when each component participates in learning.

Shadmehr and Krakauer [ 39 ] proposed a theory based on the framework of optimal feedback control. Rather than action selection, the basal ganglia are given a higher-level involvement in planning with respect to the cost and reward structure of the task. The cerebellum implements a forward model to predict the sensory consequences of motor action [ 40 ]. Recently, Haar and Donchin [ 41 ] combined Houk’s approach [ 34 ] with Shadmehr and Krakauer’s optimal control theory. They emphasize the distributed nature of the cortical network. The cortex-cerebellum loops are assumed to implement a predictive error correction of the cortical activity. However, their theory assumes the concept of parallel and segregated cortex-basal ganglia loops and thus, underestimates recent evidence for a hierarchical organization of cortex-basal ganglia loops [ 37 , 42 – 44 ].

Houk and colleagues [ 34 , 35 ] proposed a conceptual framework that suggests distributed processing modules. It places the cerebral cortex at the center, and independent loops with the BG and cerebellum feed back to the cortex. The cortico-basal ganglia loops make an initial course selection that is then narrowed down (or refined) by the corresponding cortico-cerebellar loop. For example, in a reaching task, the cerebellum may use a prediction error to compute an online corrective movement. Thus, although the cerebellum, cortex, and basal ganglia may use different learning paradigms, they implement an interactive system capable of handling a diversity of tasks [ 36 – 38 ].

Influential theories regarding the interaction between different motor systems emphasize that each system operates with a different type of learning mechanism, with the cerebellum implementing supervised learning, the basal ganglia reinforcement learning, and the cortex unsupervised learning [ 32 ]. An extended version of this idea, the super-learning hypothesis, proposes that the three learning mechanisms form an integrated system and act in synergy [ 33 ]. Results from one system may influence another through multiple neural pathways or neuromodulators.

The cerebellum is crucial for maintaining accuracy across multiple movements [ 18 , 19 ]. Individuals with cerebellar damage have deficits when required to adapt a well-known behavior to a sudden change in environmental conditions [ 20 ]. Its role in motor adaptation has been further confirmed in imaging studies [ 21 , 22 ]. Cerebellar pathology has also been hypothesized to be related to the Developmental Coordination Disorder, which in children manifests as a reduced motor performance [ 23 , 24 ]. The cerebellum may implement a forward model, an inverse model, or both. Like with the basal ganglia, most models of the cerebellum focus on internal dynamics but are rarely applied to complex motor tasks. Those computational models of the cerebellum involved in motor tasks have been mainly developed in the context of neuro-robotics, often abstract much from biological detail and typically implement a closed-loop motor control network [ 25 – 31 ].

Although most common tasks addressed by computational models of the basal ganglia only require choosing a correct action among other actions, e.g. selecting a button as a response to sensory input [ 9 – 13 ], some models addressed the role of the BG in a broader context of the motor system. Magdoom et al. [ 14 ] proposed a computational model where the BG makes corrections to movements controlled by the motor cortex. The model was used to show how reduced dopamine signals in Parkinson’s disease can produce abnormal movements. In an extended version of this model, the BG amplified low amplitude input signals through stochastic resonance to produce movements [ 15 ]. Kim et al. [ 16 ] proposed a model where the BG selects a muscle activation pattern demonstrated by a 2-dimensional reaching task. In the motor control framework proposed by Manella and Baldasarre [ 17 ], the BG modulates the dynamics of a cortical reservoir that implements a movement. The authors reproduce three different periodic behaviors of a two-joint arm.

A commonly assumed role for the motor basal ganglia (BG) is action or motor program selection [ 1 – 6 ]. The basal ganglia integrate sensory evidence arguing for a particular decision and disinhibit the corresponding action plan. Such motor program selection involves a focal removal of tonic neural activity in the output nuclei of the BG to activate the desired movement while increasing other neuronal activity to avoid the execution of unwanted programs [ 7 , 8 ]. However, how proper actions are discovered and represented is still unclear.

Results

Model design The model was designed in an open-loop control framework (Fig 1) in order to study its potential and limitations. In an open-loop control framework, the CPGs already provide movement dynamics but need to be under top-down or feedback control [58]. Our plant is a robotic arm with four degrees of freedom. In the reaching task, the shoulder’s yaw, pitch and roll, and the elbow were each controlled by an independent CPG network following the implementation of Nassour et al. [58, 62]. Each CPG network is formed by three layers: a rhythm generation layer that can generate multiple activity patterns, a pattern formation layer that shapes the generated pattern, and a motor neuron layer that drives the joint. While we do not neglect the existence of feedback pathways and closed-loop control, we start here with a model that does not include feedback except for learning. Thus, further upstream motor centers have to provide parameters that manipulate the movement dynamics of the CPG. Our model determines those parameters from two components. The motor cortex-basal ganglia interactions select concrete actions while the cerebellum fine-tunes those actions. The existent network in the brain is of course more complicated. For example, output neurons of the basal ganglia that project to the thalamus have collaterals that target different regions of the brainstem [56]. The term concrete action refers to the observation that movements can be decomposed into a finite set of elementary movements [63] and that activation of the motor cortex produces a limited set of muscle activations [64]. Action selection (BG) and action refinement (cerebellum) are learned through different biologically plausible mechanisms. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 1. Design of the model. A goal position, that may be determined by the pre-motor cortex-basal ganglia loop, has to be reached. This goal informs both, a motor cortex-basal ganglia loop and the cerebellum. The motor cortex-basal ganglia loop selects a concrete action, which determines the parameters of the CPG in the brainstem. Learning occurs when an achieved hand position is novel through dopamine-modulated Hebbian plasticity that reinforces the association between the executed action and the reached hand position. The cerebellum produces small adjustments to the CPG parameters that reduce the distance between the goal and the achieved position in the current task. Learning occurs through perturbation-based learning using the distance between the goal and the reached position as an error signal. https://doi.org/10.1371/journal.pcbi.1011024.g001 A recent hypothesis about the functional structure of the cerebellum is that the recurrent connectivity in the cerebellar cortex implements a reservoir of dynamic activities [65–67] instead of the classically hypothesized feedforward structure. Inputs from the cerebral cortex enter via the mossy fibers a strongly connected recurrent network formed by granule and Golgi cells in the cerebellar cortex, allowing complex patterns to evolve over time even after the inputs have stopped [68]. These spatio-temporal patterns in the reservoir can then be detected by the Purkinje cells to produce appropriate responses [69]. In order to benefit from this dynamical function of the cerebellum, we use the reward-modulated reservoir framework proposed by Miconi [70] as a model of the cerebellum. While the model of [70] is agnostic with respect to localizing the reservoir in any particular area of the brain, it has been used to control a musculoskeletal model of the human arm with four degrees of freedom and 16 muscles in a reaching task with two fixed targets. The reservoir learns by means of a perturbation learning rule, where random perturbations are individually applied to the neurons of the reservoir with varying amplitude and fixed frequency during a trial. At the end of a trial, the reached location is compared to the intended location to compute an aiming error signal. Depending on whether this error decreased or increased compared to the last similar trial, the weights inside the recurrent network are adapted depending on the occurrence of a perturbation (which is maintained by an eligibility trace) and the improvement or worsening of the aiming error. Perturbation learning is an alternative to error backpropagation and is considered more biologically plausible as all computations are local to the neurons. Although the reservoir network of [70] is not related to the particular structure of the cerebellum, its neurons can be divided into two groups, depending on whether they are output cells or not. Following the interpretation of the cerebellum as a reservoir computing machine [66, 67], output neurons would correspond to the Purkinje cells and non-output neurons to the granular and Golgi cells. Cerebellar parallel fibers implement therefore the readout connections, and recurrent connections between granule and Golgi cells provide the necessary dynamic behavior. However, there is no explicit distinction between excitatory granule cells and inhibitory Golgi cells in the version of the model that we use. The cortex-basal ganglia component is inspired by recent ideas regarding a hierarchical organization of the basal ganglia and cortex [42, 43]. Specifically, we proposed that the brain achieves goal-directed behavior through a cascade of decisions made by the multiple cortico-basal ganglia loops, each creating an intermediate objective at a different abstraction level [44]. Planning starts in the ventral or limbic loop with the desire for a particular internal or external reward known to be achievable given the current state. The dorsomedial or associative domain then determines the state needed to be reached in order to obtain the reward. The desired state is transformed into a motor goal by a further loop, e.g., by moving the hand to a particular location to satisfy the objective of reaching the object. Finally, the motor goal is transformed into a concrete action plan that may be executed by an open loop model, e.g. central pattern generators (CPGs). Let’s summarize the above concept with an example from everyday life: Our limbic system signals the need for water and we decide to reach for a glass of water, which in turn determines the motor goal in form of the spatial coordinates x,y,z, or the corresponding joint angles. The motor cortex-basal ganglia loop will then select a concrete action that moves the arm to the motor goal. The advantage of our hierarchical approach is that the motor goal is task-independent. After a decision about the target object is determined by the premotor loop, the reaching action does not need further information about those decisions made by the earlier loops. As we have already shown how such a set of decisions could be learned by dopamine-modulated plasticity [44, 71], we focus here on the motor loop only and how a motor goal is transformed into a concrete action and its final execution. We have also recently demonstrated that learning in multiple cortex-basal ganglia loops cannot rely on a single prediction error signal being identical for all loops [44]. While a reward prediction error is well suited for the limbic loop, the motor loops should be trained by different signals to make them specific to the motor content, independent of the planning and motivational aspects of the task. We use here a dopamine response that indicates the novelty of the achieved movement [72]. A further implication of our framework is that the goal location coming from the pre-motor cortex has initially no meaning. The meaning of such internal signals must be first discovered by active exploration via the environmental act-and-sense loop. Learning occurs after the motor action by sensing its outcome—the reached location—in the premotor cortex. Thus, the outcome is linked to the action that leads to the outcome, providing meaning to the goal signals from the premotor cortex. In our motor loop, actions are initially randomly activated and a phasic increase of dopamine indicates the novelty of the achieved movement, modulating plasticity in the motor striatum to connect outcomes to concrete actions. Supported by the ideomotor theory [73–75], we assume that this active exploration via the environmental act-and-sense loop is a necessary step that takes place prior to goal-directed behavior—but may continue during the lifetime—as the brain has initially no representation of the body kinematics (and dynamics).

Reaching with the cerebellum alone As a reference, we initially test the reservoir model from Miconi [70] to mimic cerebellar learning. Following the procedure introduced by Miconi, the activity of all cells in the reservoir is randomly initialized to a small value at the beginning of each trial, the corresponding input is set, and the network is simulated for 200 milliseconds. The input is then deactivated and the network relaxes its activity for 200 additional milliseconds. The mean activity in the last 200 milliseconds of the reservoir’s output cells is then transformed linearly into the six parameter values of each CPG layer (4 joints, therefore 24 output values). Thus, the reservoir encodes the values for the full arm movement, i.e. all joints. The network has to learn reaching movements towards 8 different arbitrary targets within the arm’s workspace. The perturbation learning rule used in the reservoir depends strongly on three parameters: the learning rate (η) or step size, the perturbation frequency (f) which determines how often the activity of the cells is perturbed, and the perturbation amplitude (A) which determines the size of the perturbation. Therefore, f and A control the level of noise in the network. Models with a small learning rate or low noise parameters decrease the error only by a small amount (see Fig 2A). Models with intermediate levels of noise or learning rate are able to solve the task but converge to different error levels. Models with faster learning become unstable: the distance to the goal initially decreases, reaches an asymptotic value, and then increases again. The same network configuration does not become unstable in a simpler version of the task in which only 2 goals are required to be learned (see Fig 2B). Results of an exhaustive parameter variation are given in S1 Fig. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 2. Reservoir’s performance. Performance of the reservoir with different parameter configurations: eta is the learning rate, f is the frequency of the perturbation and A is the amplitude of the perturbation. For each configuration, 50 different simulations are run, each with a different random seed producing different initial conditions, goals, and noise values. On all plots, the Euclidean distance between the goal and reached location over all simulations and including different goals is shown. A: The reservoir sets the parameters of a CPG network controlling each joint. The system is expected to learn 8 goals. Slow-learning networks hardly reduce the error. Fast-learning networks are unstable: They initially appear to learn the task, but then networks tend to forget previous knowledge. B: The same network is used but asked to only learn 2 goals. Configurations that were unstable with 8 goals are stable in this simpler version of the task. C: The output of the reservoir is transformed directly into joint angles (no CPGs are used). The performance of this network is worse than when including the CPGs. Shaded area next to each curve show the standard deviation of the mean. https://doi.org/10.1371/journal.pcbi.1011024.g002 On a further control configuration, CPGs are removed and the activity of the reservoir’s output cells is directly linked to the change in the 4 joint angles. Those angles are transformed into a resulting hand position using a kinematic model. Networks with less noise are weaker than those including the CPGs (see Fig 2C). Fast networks become unstable, similarly to the model that includes the CPG. Thus, the CPG component is rather beneficial and does not account for the observed limitation of the reservoir when asking it to learn movements to a larger set of goal locations. In summary, motor learning by the reservoir alone is sensitive to learning parameters, particularly when multiple target movements are required.

Reaching with the cerebellum and basal ganglia In order to test if the division of labor between the basal ganglia and the cerebellum can avoid instabilities, we tested our full neuro-computational model (see Fig 3 for a more detailed view of the model), involving both components, on the same reaching task as before. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 3. Detailed view on the computational model. Arrows indicate excitatory synaptic connections between neurons. Red arrows indicate plastic connections. Lines ending with a circle indicate inhibitory connections. The closed motor cortex-basal ganglia loop has as many stripes as concrete actions. The direct pathway within the basal ganglia selects one of 120 possible concrete actions. This large number of actions ensures sufficient movement diversity within the reaching space of the arm. Each action is represented in a discrete channel connecting the corresponding cortical, striatal, substantia nigra pars reticulata (SNr), and thalamic cells. Each discrete action activates multiple sets of neurons representing possible CPG parameter values. Each CPG is formed by three layers: RG is the rhythm-generator layer, PF is the pattern formation layer and MT are the motor neurons. The 6 parameters per CPG being adapted are: the time constant τ m , a shape parameter for the current–voltage curve of the fast current σ f , the potassium conductance normalized to the leak conductance σ s and the injected current i inj of the rhythm generator neurons of the CPGs. Further, α 0 and θ 0 which are the slope of the sigmoid and the center of the curve of the pattern formation layer of the CPGs. The final parameter value associated with each action is computed by integrating the activity of parameter cells weighted by their preferred parameter value. The cerebellum receives as input an abstract representation of the current goal (no position), one cell per possible goal. In the brain, that position may be encoded within the thalamus of the premotor loop. 24 of the 400 cells in the reservoir project outside (6 parameter values x 4 CPGs) and their activity contributes to the final CPG parameters. Only a single set of neurons for just one CPG is shown in the figure. https://doi.org/10.1371/journal.pcbi.1011024.g003 The possible concrete actions are encoded by a neural population called the motor cortex, which is part of a motor cortex-basal ganglia loop. Each cortical cell projects to a set of neurons that use a population code to represent the CPG parameter values (see Fig 3). Each cell in these parameter populations is assigned a preferred parameter value. The final parameter value is decoded by computing a sum over the preferred parameter values, weighted by the activity of the corresponding cell. The weights of the connections from the action encoding population to the parameter encoding populations are fixed and random. The basal ganglia network is a simplified version of our previous model [44, 76, 77], including only a direct pathway (striatum → substantia nigra pars reticulata → thalamus → cortex). Selection occurs when the constant inhibition exerted by the substantia nigra on the thalamus is removed by a corresponding activation in the striatum, allowing a specific cell in the thalamus to get activated and increase the firing rate of the corresponding concrete action. Despite some agreement on the functional role of different basal ganglia pathways there is nevertheless some variability particularly with respect to the indirect and hyperdirect pathway [4]. For the purpose of our study, we only need an intact function of the direct pathway and thus keep the model simple to save computation time. However, more complex motor tasks may benefit from considering additional basal ganglia pathways. Dopamine-modulated Hebbian learning in the striatum links the input from the goal-encoding cells to the motor program. Novelty-based learning in the basal ganglia works as follows: After every movement, the input activity of the dopamine cell is increased from its baseline to 1, triggering plasticity in striatal neurons. The activity reached by the dopamine cells is however limited by a prediction obtained from the inhibition produced from the striatum, which is also subject to plasticity. The dopamine level reaches its maximum value only when an action is executed for the first time as the striatal inhibition increases after each movement. The same dopamine signal reaches all cells. Unlike previous action-selection models of the BG, we only implement plasticity between the premotor cortex and the basal ganglia. It is common in computational models to assume that the BG implement a winner-take all mechanism between input action channels [6]. In classical action-selection models, the main inputs to the BG loop are the available actions and the BG must select one of them, usually the most salient one. In those models, the BG does not implement any transformation of the input information, it only removes the less salient action channels. Plasticity is then implemented in the connections within the loop (motor cortex) to assure a proper action selection. Based on our previous models [9, 44, 77, 78], we instead assume that each BG loop learns a goal-response map, which links objectives to appropriate actions. The input to the loop is different than action-selection models as it results from the information processing in previous loops. For selecting concrete actions, plasticity is then required at the projections from the premotor cortex, not necessarily at the projections from the motor cortex. The cerebellum is modeled as a pool of 400 randomly connected cells. The projections within the pool are plastic and follow a perturbation-based learning rule [70]. 24 of those 400 project outside (6 parameters per joint). The activity of these output cells is added to the parameter value encoded in the parameter cells before they are set in the CPGs. The basal ganglia are trained prior to the task simulation until the model replicates a randomly selected outcome for three times in a row. The main goal of this process is for the basal ganglia to create a map between outcomes (final hand positions) and concrete actions. During training, 120 actions are activated randomly, the outcome is observed and finally the association strength between the outcome and the action is increased. This creates a meaning for the pre-motor cortex neurons, which do not have one until activated by an observation. On each simulation a different set of 120 actions are defined, each associated with a random set of CPG parameters. Later, the outcome-action map is be used to select an action based on a desired outcome (Fig 4). The BG therefore are not trained on the goals of the task, but develop knowledge about the possible actions to choose from. Activity of the BG during an example trial is shown in S2 Fig. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 4. Basal ganglia training. The initial training of the basal ganglia is performed by randomly activating desired outcomes. A: Learned trajectories of 120 concrete actions of an example simulation. Each of the 120 lines in the plot represents the trajectory of the hand after selecting one action starting from the same position in one simulation. The basal ganglia can therefore select one among 120 trajectories. B: Result of learning in the basal ganglia by exploration via the environmental act-and-sense loop. At the beginning of every training trial, a random goal (desired hand position) is activated. Then, if no action cell had a strong enough firing rate, a random action is activated by setting its activity to 1. The basal ganglia learn to map the reached position with the activated action. Thus, learning associates the outcome with the action that leads to the outcome (act-and-sense). The plot shows that, over time, intended outcomes become associated with an action that closely reaches it. The blue line represents the mean distance over 50 simulations and the orange line is the average of the mean distance with a time window of 10 trials. https://doi.org/10.1371/journal.pcbi.1011024.g004 We simulated the same reaching task with 2 and 8 goals. We used in the cerebellum a learning rate η = 0.8 and noise parameters f = 9 and A = 20. These parameters correspond to a fast network, which produced an unstable behavior when learning the task directly. Our simulations show that, with the full model, both tasks can be learned without any problem of stability. The reason is that learning is simpler as the BG introduce an initial solution through a concrete action and only small adjustments are produced by the cerebellum (see Fig 5). Not surprisingly, learning is also much faster than with the cerebellum alone. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 5. Training the full model in a reaching task. The full model includes a cortex-basal ganglia component that has been pretrained to allow the selection of a concrete action to a given arbitrary goal. The full model is then tested with either 2 or 8 random goal positions and required to learn to execute a movement to the given goal. The full model is not unstable when the number of goals is increased from 2 to 8. The shaded area next to each curve shows the standard deviation. https://doi.org/10.1371/journal.pcbi.1011024.g005

Visuomotor adaptation task After demonstrating the model’s basic functionality, we now investigate its ability to explain observations in motor adaptation. Motor adaptation refers to a particular type of motor learning in which a well-known action is modified to maintain performance after a change in the environment or the body [79]. One common way to study adaptation in an experimental setting is to impose a visuomotor rotation [80]. In such experiments, participants are seated in front of a screen and are required to move a cursor toward a target location with a straight inward-outward movement [81]. The cursor is not visible throughout the whole trajectory. During the movement, the cursor remains initially at its starting position and then indicates the movement reversal point. Thus, subjects only obtain visual feedback about their movement outcome with respect to its endpoint. After several baseline trials, the cursor’s coordinate system is rotated with respect to the coordinate system of the hand movement space. As participants are not informed about the manipulation and only observe the outcome, they slowly alter their behavior to cope with this perturbation. Errors are reduced trial by trial suggesting that it is controlled by an implicit learning process. Once the perturbation is removed, an aftereffect is observed: The participants initially overcompensate and then slowly, trial by trial, return to normal movements [80]. However, when participants were instructed about the nature of the perturbation and an instruction to compensate for it, they immediately applied it and had almost no error in the trial after the information has been given [80]. We confront our model with the visuomotor adaptation task used by Mazzoni and Krakauer [80]. After initial training on the baseline trials on two random goals, the coordinate system of the cursor is rotated by 45 degrees. As with the participants, we have three types of model simulations: in a first simulation, the model receives no information about the perturbation (rotation group); in a second simulation, the model is forced to adopt an explicit cognitive strategy by instructing it to direct the movement 45 degrees counterclockwise (rotation + strategy group); and in a third simulation the model is also instructed to direct the movement 45 degrees counterclockwise but the cursor is not perturbed (strategy group). The perturbation is simulated in our model by rotating the final outcome of the hand movement by 45 degrees, as also human subjects have no visual feedback of their arm trajectory. Thus, after the rotation is introduced, the models make a 45 degree error (in Fig 6 at trial 100). The manipulation leads to an error signal in the cerebellum, which shows a strong increase once the rotation is introduced, but it does not induce novelty-based learning in the BG. In the strategy condition, the model is instructed to counter the perturbation, as with human subjects in the original experiment of Mazzoni and Krakauer. The instruction to counter the perturbation is given to our model as a change in the goal represented in the premotor cortex. The new goal corresponds to a position rotated from 45 degrees with respect to the initial one. The new input triggers the BG to select a different concrete action, one that moves the arm closer to the new goal direction. As with the participants, the instructed model immediately reduces the error close to zero (trial 103 in the Fig 6). This rapid change in movement direction, similarly to what was observed in humans, is in our model proposed by action selection at the BG level, as the cerebellum outputs only gradual corrections and requires multiple repetitions to adapt. In the following trials, the new motor goal is maintained and therefore the basal ganglia continues selecting the same concrete action. The change in the motor goal due to the instruction also affects the error computed at the level of the cerebellum, as the observed position of the pointer is compared to the intended motor outcome (aiming error, not task error). Importantly, as observed in human subjects, this explains why the model shows increasingly large directional errors over the following trials, over-adapting to the perturbation. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 6. Visuomotor adaptation. Test of the model with a visuomotor rotation task [80]. After initial training on baseline trials, the coordinate system of the cursor is rotated by 45 degrees. Then, after 200 trials interacting in the perturbed environment, the conditions return to the baseline. The first row shows the performance of models that, after 2 trials in the perturbed experiments, are informed about the perturbation by changing the goal location and re-setting the goal location later on (ROTATION + STRATEGY). The second row shows models that are not informed about the perturbation (ROTATION). The third row shows models that are provided with the new goal, but the environment was not perturbed (STRATEGY). https://doi.org/10.1371/journal.pcbi.1011024.g006 In the original experiment of [80], after over-adapting to the perturbation, participants were instructed to stop using the explicit strategy. We give our model this information by a change in the goal, setting it back to the initial position, changing therefore again the concrete action, and as a consequence the error at the cerebellum. The new concrete action produces an immediate change in the direction, as observed in humans (see the increase in the error in Fig 6 rotation+strategy group around trial 300). When the perturbation is finally removed (10 trials after the last instruction), models and subjects show an after-effect and the error slowly declines. During this last period there is no further change in the motor goal and the corrections are therefore only produced by the cerebellum. Our simulations of the rotation group (no instruction) show no immediate direction change. Like the human subjects, the model slowly adapts to the perturbation reducing the error trial by trial. Once the perturbation is removed, an aftereffect is again observed: A change in the direction of the error and a slow return to zero. The simulations of the group that was instructed, but not perturbed, show no slow change in the error and no aftereffect. The change in the concrete action moves the arm toward the new desired direction and only very small changes are introduced by the cerebellum, as errors are computed according to the new instructed motor goal (aiming error). Thus, no after-effect occurs, similar to the data from human subjects. A comparison of the error signal in the Cerebellum under the three conditions can be observed in S3 Fig. When we remove the cerebellum such that it provides no contribution to the CPG, in the rotation+strategy condition the BG compensates for the perturbation and the over-adaptation observed in the full model does not occur (S4 Fig). Concluding, our model can replicate the main properties of the data of [80]. However, we spotted also small differences such that the model’s implicit learning process is slower than those of the participants. This could be because in the experiment of Mazzoni and Krakauer, the subjects were expected to make wrist movements of only 2.2cm, much shorter than in our setup.

[END]
---
[1] Url: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1011024

Published and (C) by PLOS One
Content appears here under this condition or license: Creative Commons - Attribution BY 4.0.

via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/

You are viewing proxied material from magical.fish. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.