Summary: Two distinct groups of neurons have been identified that aid the mind in evaluating risk and reward in decision-making. These cells, located in the lateral brain, individually approach better-than-expected and worse-than-expected results. In research with animals, silencing these cells altered their expectation of returns, affecting their decision-making behavior.
The research suggests that the mind tracks a whole range of possible benefits, rather than just an ordinary, which aligns with machine-learning versions of decision-making. If human tests are confirmed, the results might help to understand limitations in weighing challenges in conditions like addiction and depression. This mental hardware will be analyzed in future studies.
Important Facts:
- Two Neurological Groups: One class processes better-than-expected results, while another songs worse-than-expected ones.
- Decision-Making Mechanism: The mind represents a whole range of possible benefits, not only averages.
- Prospective Clinical Impact: The findings may help explain the limitations of risk assessment in situations like addiction and depression.
Origin: Harvard
Every time, our brains makes thousands of choices, big and small. Any of these choices, from the least significant to the more crucial ones like choosing a restaurant, could have better or worse results.  ,
How does the brains interpret these calls as risk and reward? Researchers continue to wonder what the answer to this question is, but a recent study conducted by Harvard Medical School and Harvard University provides amazing answers.
The study,  , published Feb. 19 in , Nature , and supported in part by federal money, included machine-learning aspects into mouse tests to analyze the brain wiring that supports reward-based decisions.
The researchers found two types of brain cells in animals: one that teaches them about above-average effects and the other that is associated with sub-average outcomes. These cells, the experiments showed, provide the head with a comprehensive view of the range of potential rewards that come with a decision.
” Our findings suggest that animals— and by extension, another animals — seem to be representing more fine-grained information about risk and reward than we thought before”, said co-senior author , Jan Drugowitsch, equate professor of neuroscience in the Blavatnik Institute at Harvard Medical School.
If confirmed in humans, the findings could serve as a framework for understanding how the human brain makes reward-based decisions and what happens to the ability to assess risk and reward when reward circuitry fails.
Machine learning illuminates reward-based decisions
How the brain uses previous experiences to make new decisions has long been a topic of conversation for neuroscientists. However, according to Drugowitsch, many traditional theories about such decision-making fail to capture the complexity and nuance of real-world behavior.
When choosing a restaurant, according to Drugowitsch, you might want to play it safe by choosing one with a menu that your experience consistently recommends as being good, and you might want to take a chance by choosing one that offers a mix of excellent and subpar food.
Existing neuroscience theories assume that the restaurants in the above example have the same range of offerings, but they are all comparable when averaged, predicting an equal likelihood of choosing either.  ,
We are aware that this is not how animals and humans behave; we can choose to play safe while taking risks, according to Drugowitsch. We feel that our choices are more than just average expected rewards.
Researchers studying machine learning have recently created a theory of decision-making that more accurately captures the full range of potential rewards associated with a choice.
In Atari video games and a range of other tasks where each choice has multiple possible outcomes, they incorporated this theory into a new machine-learning algorithm that outperformed alternative algorithms.
” They basically asked what happens if rather than just learning average rewards for certain actions, the algorithm learns the whole distribution, and they found it improved performance significantly”, Drugowitsch said.
In a , 2020 , Nature , paper,  , Naoshige Uchida, professor of molecular and cellular biology at Harvard University, and colleagues reanalyzed existing data to explore whether this machine-learning theory applied to neuroscience, in the context of decision-making in rodent brains.
The analysis showed that in mice, activity of the neurotransmitter dopamine — which plays a role in reward-seeking, pleasure, and motivation — corresponded to reward-learning signals predicted by the algorithm.
In other words, according to Drugowitsch, the study suggested that the new algorithm was more effective at explaining dopamine activity.
how the brains of mice represent a range of benefits
To advance the study further, Drugowitsch and co-senior author Uchida collaborated to create a new study. Together, they created mouse experiments to examine how this process operates in a brain region known as the ventral striatum, which contains data on potential rewards associated with a choice.
” Dopamine activity only provides the learning signal for expected rewards,” said Drugowitsch, but we wanted to find representations of these learned rewards directly in the brain.
The researchers taught mice the range of possible outcomes of a choice by teaching them to associate various odors with rewards of varying magnitudes. The mice were then given odors, and they observed licking ( mice lick more to get better rewards ) while recording ventral striatum neural activity.
The team identified two distinct groups of neurons in the brain: One that helps a mouse learn about better-than-expected outcomes and another tied to worse-than-expected outcomes.
You can imagine having both an optimist and a pessimist in your head, both offering you advice on what to do next, according to Drugowitsch.
When the researchers silenced the “optimistic” neurons, the mouse exhibited behavior suggesting that it anticipated a less appealing reward. Conversely, when the researchers silenced the “pessimistic” neurons, the mouse behaved as if it expected a higher value treat.
According to Drugowitsch,” These two groups of brain cells combine to provide a representation of the full range of potential rewards for a decision.”
The researchers anticipate that their research will have many directions in the future, including how the brain makes decisions when there is more uncertainty about what each initial option represents and how their findings apply to more general-purpose logical reasoning about the world.
More research is required, according to Drugowitsch, and more work is needed to adapt the conclusions to the breadth of human decision-making. He thinks the work may already have some insight into how humans assess risk in decisions and why people with certain conditions like depression or addiction may struggle with these assessments based on the similarities between mouse and human brains.
Authorship, funding, disclosures
Additional authors on the paper include Adam Lowet, Qiao Zheng, Melissa Meng, and Sara Matias.
Funding: The study was funded by the National Institutes of Health ( R01NS116753, F31NS124095 ), the Human Frontier Science Program ( LT000801/2018 ), the Harvard Brain Science Initiative, and the Brain &, Behavior Research Foundation.
About this decision-making and neuroscience research news
Author: Dennis Nealon
Source: Harvard
Contact: Dennis Nealon – Harvard
Image: The image is credited to Neuroscience News
Original Research: Closed access.
Jan Drugowitsch and colleagues ‘” Distributive reinforcement learning in a rival striatal circuit” Nature
Abstract
Distributive reinforcement learning in a rival striatal circuit
By expanding the learning target from mean rewards to entire probability distributions of rewards, a method known as distributional reinforcement learning ( RL), machine learning research has achieved significant performance gains on a wide range of tasks.
Although it is thought that the mesolimbic dopamine system updates a representation of mean value in the striatum to underlie RL in the mammalian brain, little is known about whether, where, and how neurons in this circuit encode information about higher-order moments of reward distributions.
Here, to fill this gap, we used high-density probes ( Neuropixels ) to record striatal activity from mice performing a classical conditioning task in which reward mean, reward variance and stimulus identity were independently manipulated.
We found strong evidence for abstract encoding of variance in the striatum, in contrast to conventional RL accounts. These distributional representations in the striatum were disorganized by a synthetic ablation of dopamine inputs without affecting mean value coding.
The two main classes of striatal medium spiny neurons, D1 and D2, were identified by two-photon calcium imaging and optogenetics, which preferentially encoded the right and left tails of the reward distribution, respectively.
We combine these findings with a novel model of the striatum and mesolimbic dopamine that exploits the computational advantages of distributional RL by utilizing the opponency between D1 and D2 medium spiny neurons .
