There are major similarities between the basal ganglia and reinforcement learning actor/critic networks. Dopamine neurons respond in ways that might encode reward-prediction errors, as required of the teaching signal in reinforcement learning models. The longer-term effects of dopamine on corticostriatal synap-tic input to striatal spiny projection neurons are consistent with three-factor rules required for reinforcement learning. There is also evidence that dopamine neurons represent motivational properties and may serve as a neural substrate to encode incentive salience. The more immediate and reversible actions of dopamine may be linked to initiation of movements, brought about by facilitation of striatal output by anticipatory firing of dopamine cells in response to incentive cues. Cholinergic interneurons, or TANs, encode instructed motivational contexts. The input to TANs is also subject to regulation by dopamine. The output of TANs may, in turn, modulate the effects of the dopamine signal, as well as have direct effects on the spiny projection neurons. There is evidence to support, in principle, the basic idea that dopamine and acetylcholine are involved in the activation of selected corticostriatal pathway circuits by incentive stimuli and strengthening by positive reinforcement. Many important questions and controversial issues remain: How good is the agreement between the predictions of machine learning theory regarding the teaching signal and neural activity in do-pamine and acetylcholine neurons? In the context of the basal ganglia, what is encoded in the inputs and outputs from the striatum, which in machine learning terms correspond to the stimulus and response? Finally, how well do the cellular actions of dopamine and acetylcholine translate into the learning rules required for adaptive actor/critic behavior?
Was this article helpful?