Problems with working out how the brain works from fMRI scans

More analysis problems with brain scanning research have come to light in a new study just released in Nature Neuroscience and expertly covered by the BPS Research Digest. It demonstrates that the common practice of using the same data set to identify an area of interest and then home in on this area to test further ideas can lead to misleading results.
This usually occurs when brain activation is compared between two conditions where participants are doing different tasks. A whole brain analysis looks for statistically significant differences at every point in the brain.
It’s very complete, but because of the large amount of data, but the data also contains a large amount of noise, so it’s hard to find areas which you can confidently say are more active in one condition than the other.
An alternative approach is to only look at activation in one area of the brain, perhaps an area where it is most likely to occur based on what we already know about how the brain works. This is called region of interest analysis (often done with the wonderfully named ‘MarsBaR‘ tool) and because the data set is much smaller, it is more likely to find a reliable difference.
However, some studies do a whole brain analysis to find likely areas, and then home in using region of analysis tools to examine them ‘more closely’. This ‘magnifying glass’ metaphor seems intuitive, but because your using the same data set to create and test hypothesis, it can be problematic.
It’s like shooting arrows randomly into a wall and then drawing a target around ones which landed together. Someone looking at wall afterwards might think the archer was a good shot, but this impression is caused by the after-the-event painting of the target, and the same problem could affect these brain imaging studies.
After the recent furore over the ‘voodoo correlations’ study, this new study is markedly more measured in its language and doesn’t list individual offenders.
Indeed, the ‘Voodoo Correlations in Social Neuroscience’ paper was actually retitled on publication to ‘Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition’, presumably to avoid stirring the pot any further.
However, this new study takes a similar tack, demonstrating through several careful simulations that ‘double dipping’ a data set is likely to distort the results just due to statistical problems.
From the BPS Research Digest:

Nikolaus Kriegeskorte and colleagues analysed all the fMRI studies published in Nature, Science, Nature Neuroscience, Neuron and Journal of Neuroscience, in 2008, and found that 42 per cent of these 134 papers were guilty of performing at least one non-independent selective analysis – what Kriegeskorte’s team dub “double dipping”.
This is the procedure, also condemned by the Voodoo paper, in which researchers first perform an all-over analysis to find a brain region(s) that responds to the condition of interest, before going on to test their hypothesis on data collected in just that brain region. The cardinal sin is that the same data are used in both stages.
A similarly flawed approach can be seen in brain imaging studies that claim to be able to discern a presented stimulus from patterns of activity recorded in a given brain area. These are the kind of studies that lead to “mind reading” headlines in the popular press. In this case, the alleged statistical crime is to use the same data for the training phase of pattern extraction and the subsequent hypothesis testing phase.

Link to BPS Research Digest on the fMRI analysis problems.
Link to PubMed entry for study.

Amygdala hijacking - Don't let it happen to you! An excitable presentation style!

Evidence based ideas for behaviour change

Count very carefully......

The power of metaphor From All In The Mind (BBC Radio 4, 5.07.11)

Lera Boroditsky talks to Claudia Hammond about the power of metaphor to change what we think.

Claudia Hammond (CH): In 1990 a schoolgirl was attacked in Buffalo, New York. Before the attacker was captured 15 months later, another 10 girls had been assaulted by the same man. And some scholars believe that these girls were not only the victims of crime but the way we use language to frame a problem – that the police saw their role as hunting the man down in secrecy, rather than taking steps to protect the community from him while he was still at large. Now new research from the University of Stanford has found that something as simple as describing crime as a ‘beast’ or as a ‘virus’ can change the way we think about crime and the solutions we suggest to tackle it. But if simple words can make such a difference, what implications does this have for the social policy decisions that affect us all? The author of the research is Assistant Professor of Psychology, Lera Boroditsky. In a minute we’ll hear more about her specific work on metaphor and crime. But before that I asked her how more broadly language can affect both what we think and what we know in everyday situations.

Lara Boroditsky (LB): Research from my lab and from many others has shown that language isn’t just a way of expressing your thoughts – it shapes the very thoughts you wish to express. In some cases, languages even give people skills or cognitive abilities that other people don’t have. One of my favourite examples comes from work on language and spatial abilities, navigational abilities. So there are folks around the world who, instead of using words like ‘left’ and ‘right’, use words like ‘north, east, south and west’. One consequence of speaking like this – always using ‘north, east, south and west’ – is you have to say things like “There’s an ant on your south west leg” or “Pass the cup to the north-north east a little bit”. In Kuuk Thaayorre [an Australian Aboriginal language], for instance, to say “Hello”, you say “Where are you going?” and the answer should be something like, “North-north east in the far distance. How about you?” As a consequence, folks who speak languages like this are incredibly good at staying oriented. And even a five-year-old can point correctly south east when asked, without hesitation. That’s something that most American college students or most American professors can’t do.

CH: And your recent work has been looking at metaphor and how people think about crime. Why did you pick crime?

LB: The answer is that I wanted to work on something that is a real-world problem. It started with the observation that political speeches are just suffused with metaphors, and sometimes with metaphors that really strongly reorganise one’s ideas.

CH: So in the lab you’ve been looking at the different ways crime might be described and compare what happens if you talk about it as a virus or as beast. What did you do there?

LB: In our studies we told people about an increasing crime situation in a fictional city and we gave them lots of statistics about crime. But half of the people were told that crime was a beast ravaging the city and the other half were told that crime was a virus ravaging the city. Both groups got the same factual information, but in each case they got this innocent little one-word metaphor: “It’s a virus” or “It’s a beast”. And what we predicted was that if people were thinking about crime as a virus, then their solutions to treating the crime problem would be similar to the way you would treat a virus: they might want to diagnose the problem, they might want to institute preventative measures – kind of inoculate the community, institute social reforms, maybe improve the educational system, things like that. Whereas if they thought that crime was a beast, they would try to treat it as if it were a real beast attack. So you would try to hunt it down, you would throw people in jail, you would enforce harsher sentences, things like this.

And this is exactly what we found. People who were told crime was a beast, given the ‘beast’ metaphor, were much more likely to come up with enforcement and punishment solutions to the problem. Whereas people who were given the ‘virus’ metaphor were much more likely to say “Let’s try to improve the economic situation in the community, let’s try to improve the educational situation” – kind of restore the health of the community so that the problem doesn’t continue.

CH: It seems extraordinary that even when you mention this word ‘beast’ or ‘virus’ just once, it had this big impact on what people gave as their solutions.

LB: What we found was is that people assimilate information into their metaphorical frames, so even if it is just one word, it’s a very powerful word that activates a whole knowledge network, a whole framework of thinking; and that all the further information people learn about the city, all the crime statistics, is kind of assimilated and shaped into the knowledge framework that’s activated by that word.

CH: And you found it only worked if the word was near the beginning, not at the end of the description they had about the crime problem. Were people aware that this was happening, that this was influencing them?  

LB: That was one of the most interesting parts of what we found. We asked people after they gave us their solution: “What influenced you in your solution, what made you give the answer that you gave?” And nearly no-one chose the metaphor as the thing that influenced them. People thought they were influenced by the statistics, by the objective facts, whereas they were giving very different solutions depending on the metaphor that they got. So the metaphor was invisible to them, they didn’t think it was important, and yet it had formed their whole impression of the situation.

CH: And this must have effects way beyond crime because metaphors are used so often. I mean you think about people talking about floods of migrants coming in, or influxes or invasions – all those sorts of words are going to bring up certain sorts of things for people, aren’t they?

LB: Absolutely. In fact it’s impossible to talk about anything that’s abstract or complex without using metaphor. Partially this is just because we have a very finite set of words in our language and we have an infinite set of things we want to talk about, so we constantly have to re-use words and expressions from old knowledge to talk about new stuff. So we’re inevitably using metaphor. But certainly, whenever we’re trying to conceptualise something complex and abstract, and any societal problem fits into this category, we’re using huge numbers of metaphors. And those metaphors really frame how we shape the issues and then how we try to solve them, how we act on them in the end.

CH: Lara Boroditsky from Stanford University. 

The Optimism Bias: Why we're wired to look on the bright side by Tali Sharot

We like to think of ourselves as rational creatures. We watch our backs, weigh the odds, pack an umbrella. But both neuroscience and social science suggest that we are more optimistic than realistic. On average, we expect things to turn out better than they wind up being. People hugely underestimate their chances of getting divorced, losing their job or being diagnosed with cancer; expect their children to be extraordinarily gifted; envision themselves achieving more than their peers; and overestimate their likely life span (sometimes by 20 years or more).

The belief that the future will be much better than the past and present is known as the optimism bias. It abides in every race, region and socioeconomic bracket. Schoolchildren playing when-I-grow-up are rampant optimists, but so are grown-ups: a 2005 study found that adults over 60 are just as likely to see the glass half full as young adults.

You might expect optimism to erode under the tide of news about violent conflicts, high unemployment, tornadoes and floods and all the threats and failures that shape human life. Collectively we can grow pessimistic – about the direction of our country or the ability of our leaders to improve education and reduce crime. But private optimism, about our personal future, remains incredibly resilient. A survey conducted in 2007 found that while 70% thought families in general were less successful than in their parents' day, 76% of respondents were optimistic about the future of their own family.

Overly positive assumptions can lead to disastrous miscalculations – make us less likely to get health checkups, apply sunscreen or open a savings account, and more likely to bet the farm on a bad investment. But the bias also protects and inspires us: it keeps us moving forward rather than to the nearest high-rise ledge. Without optimism, our ancestors might never have ventured far from their tribes and we might all be cave dwellers, still huddled together and dreaming of light and heat.

To make progress, we need to be able to imagine alternative realities – better ones – and we need to believe that we can achieve them. Such faith helps motivate us to pursue our goals. Optimists in general work longer hours and tend to earn more. Economists at Duke University found that optimists even save more. And although they are not less likely to divorce, they are more likely to remarry – an act that is, as Samuel Johnson wrote, the triumph of hope over experience.

Even if that better future is often an illusion, optimism has clear benefits in the present. Hope keeps our minds at ease, lowers stress and improves physical health. Researchers studying heart-disease patients found that optimists were more likely than non-optimistic patients to take vitamins, eat low-fat diets and exercise, thereby reducing their overall coronary risk. A study of cancer patients revealed that pessimistic patients under 60 were more likely to die within eight months than non-pessimistic patients of the same initial health, status and age.

In fact, a growing body of scientific evidence points to the conclusion that optimism may be hardwired by evolution into the human brain. The science of optimism, once scorned as an intellectually suspect province of pep rallies and smiley faces, is opening a new window on the workings of human consciousness. What it shows could fuel a revolution in psychology, as the field comes to grips with accumulating evidence that our brains aren't just stamped by the past. They are constantly being shaped by the future.

Hardwired for hope?

I would have liked to tell you that my work on optimism grew out of a keen interest in the positive side of human nature. The reality is that I stumbled onto the brain's innate optimism by accident. After living through 9/11, in New York City, I had set out to investigate people's memories of the terrorist attacks. I was intrigued by the fact that people felt their memories were as accurate as a videotape, while often they were filled with errors. A survey conducted around the country showed that 11 months after the attacks, individuals' recollections of their experience that day were consistent with their initial accounts (given in September 2011) only 63% of the time. They were also poor at remembering details of the event, such as the names of the airline carriers. Where did these mistakes in memory come from?

Scientists who study memory proposed an intriguing answer: memories are susceptible to inaccuracies partly because the neural system responsible for remembering episodes from our past might not have evolved for memory alone. Rather, the core function of the memory system could in fact be to imagine the future – to enable us to prepare for what has yet to come. The system is not designed to perfectly replay past events, the researchers claimed. It is designed to flexibly construct future scenarios in our minds. As a result, memory also ends up being a reconstructive process, and occasionally, details are deleted and others inserted.

To test this, I decided to record the brain activity of volunteers while they imagined future events – not events on the scale of 9/11, but events in their everyday lives – and compare those results with the pattern I observed when the same individuals recalled past events. But something unexpected occurred. Once people started imagining the future, even the most banal life events seemed to take a dramatic turn for the better. Mundane scenes brightened with upbeat details as if polished by a Hollywood script doctor. You might think that imagining a future haircut would be pretty dull. Not at all. Here is what one of my participants pictured: "I was getting my hair cut to donate to Locks of Love [a charity that fashions wigs for young cancer patients]. It had taken me years to grow it out, and my friends were all there to help celebrate. We went to my favourite hair place in Brooklyn and then went to lunch at our favourite restaurant."

I asked another participant to imagine a plane ride. "I imagined the takeoff – my favourite! – and then the eight-hour-long nap in between and then finally landing in Krakow and clapping the pilot for providing the safe voyage," she responded. No tarmac delays, no screaming babies. The world, only a year or two into the future, was a wonderful place to live in.

If all our participants insisted on thinking positively when it came to what lay in store for them personally, what does that tell us about how our brains are wired? Is the human tendency for optimism a consequence of the architecture of our brains?

The Human time machine

To think positively about our prospects, we must first be able to imagine ourselves in the future. Optimism starts with what may be the most extraordinary of human talents: mental time travel, the ability to move back and forth through time and space in one's mind. Although most of us take this ability for granted, our capacity to envision a different time and place is in fact critical to our survival.

It is easy to see why cognitive time travel was naturally selected for over the course of evolution. It allows us to plan ahead, to save food and resources for times of scarcity and to endure hard work in anticipation of a future reward. It also lets us forecast how our current behaviour may influence future generations. If we were not able to picture the world in a hundred years or more, would we be concerned with global warming? Would we attempt to live healthily? Would we have children?

While mental time travel has clear survival advantages, conscious foresight came to humans at an enormous price – the understanding that somewhere in the future, death awaits. Ajit Varki, a biologist at the University of California, San Diego, argues that the awareness of mortality on its own would have led evolution to a dead end. The despair would have interfered with our daily function, bringing the activities needed for survival to a stop. The only way conscious mental time travel could have arisen over the course of evolution is if it emerged together with irrational optimism. Knowledge of death had to emerge side by side with the persistent ability to picture a bright future.

The capacity to envision the future relies partly on the hippocampus, a brain structure that is crucial to memory. Patients with damage to their hippocampus are unable to recollect the past, but they are also unable to construct detailed images of future scenarios. They appear to be stuck in time. The rest of us constantly move back and forth in time; we might think of a conversation we had with our spouse yesterday and then immediately of our dinner plans for later tonight.

But the brain doesn't travel in time in a random fashion. It tends to engage in specific types of thoughts. We consider how well our kids will do in life, how we will obtain that sought-after job, afford that house on the hill and find perfect love. We imagine our team winning the crucial game, look forward to an enjoyable night on the town or picture a winning streak at the blackjack table. We also worry about losing loved ones, failing at our job or dying in a terrible plane crash – but research shows that most of us spend less time mulling over negative outcomes than we do over positive ones. When we do contemplate defeat and heartache, we tend to focus on how these can be avoided.

Findings from a study I conducted a few years ago with prominent neuroscientist Elizabeth Phelps suggest that directing our thoughts of the future toward the positive is a result of our frontal cortex's communicating with subcortical regions deep in our brain. The frontal cortex, a large area behind the forehead, is the most recently evolved part of the brain. It is larger in humans than in other primates and is critical for many complex human functions such as language and goal setting. Using a functional magnetic resonance imaging (fMRI) scanner, we recorded brain activity in volunteers as they imagined specific events that might occur to them in the future. Some of the events that I asked them to imagine were desirable (a great date or winning a large sum of money), and some were undesirable (losing a wallet, ending a romantic relationship). The volunteers reported that their images of sought-after events were richer and more vivid than those of unwanted events.

This matched the enhanced activity we observed in two critical regions of the brain: the amygdala, a small structure deep in the brain that is central to the processing of emotion, and the rostral anterior cingulate cortex (rACC), an area of the frontal cortex that modulates emotion and motivation. The rACC acts like a traffic conductor, enhancing the flow of positive emotions and associations. The more optimistic a person was, the higher the activity in these regions was while imagining positive future events (relative to negative ones) and the stronger the connectivity between the two structures.

The findings were particularly fascinating because these precise regions – the amygdala and the rACC – show abnormal activity in depressed individuals. While healthy people expect the future to be slightly better than it ends up being, people with severe depression tend to be pessimistically biased: they expect things to be worse than they end up being. People with mild depression are relatively accurate when predicting future events. They see the world as it is. In other words, in the absence of a neural mechanism that generates unrealistic optimism, it is possible all humans would be mildly depressed.

Can optimism change reality?

The problem with pessimistic expectations, such as those of the clinically depressed, is that they have the power to alter the future; negative expectations shape outcomes in a negative way. How do expectations change reality?

To answer this question my colleague, cognitive neuroscientist Sara Bengtsson, devised an experiment in which she manipulated positive and negative expectations of students while their brains were scanned and tested their performance on cognitive tasks. To induce expectations of success, she primed college students with words such as smart, intelligent and clever just before asking them to perform a test. To induce expectations of failure, she primed them with words like stupid and ignorant. The students performed better after being primed with an affirmative message.

Examining the brain-imaging data, Bengtsson found that the students' brains responded differently to the mistakes they made depending on whether they were primed with the word clever or the word stupid. When the mistake followed positive words, she observed enhanced activity in the anterior medial part of the prefrontal cortex (a region that is involved in self-reflection and recollection). However, when the participants were primed with the word stupid, there was no heightened activity after a wrong answer. It appears that after being primed with the word stupid, the brain expected to do poorly and did not show signs of surprise or conflict when it made an error.

A brain that doesn't expect good results lacks a signal telling it, "Take notice – wrong answer!" These brains will fail to learn from their mistakes and are less likely to improve over time. Expectations become self-fulfilling by altering our performance and actions, which ultimately affects what happens in the future. Often, however, expectations simply transform the way we perceive the world without altering reality itself. Let me give you an example. While writing these lines, my friend calls. He is at Heathrow waiting to get on a plane to Austria for a skiing holiday. His plane has been delayed for three hours already, because of snowstorms at his destination. "I guess this is both a good and bad thing," he says.

Waiting at the airport is not pleasant, but he quickly concludes that snow today means better skiing conditions tomorrow. His brain works to match the unexpected misfortune of being stuck at the airport to its eager anticipation of a fun getaway.

A cancelled flight is hardly tragic, but even when the incidents that befall us are the type of horrific events we never expected to encounter, we automatically seek evidence confirming that our misfortune is a blessing in disguise. No, we did not anticipate losing our job, being ill or getting a divorce, but when these incidents occur, we search for the upside. These experiences mature us, we think. They may lead to more fulfilling jobs and stable relationships in the future. Interpreting a misfortune in this way allows us to conclude that our sunny expectations were correct after all – things did work out for the best.

The role of the caudate nucleus

How do we find the silver lining in storm clouds? To answer that, my colleagues – renowned neuroscientist Ray Dolan and neurologist Tamara Shiner – and I instructed volunteers in the fMRI scanner to visualise a range of medical conditions, from broken bones to Alzheimer's, and rate how bad they imagined these conditions to be. Then we asked them: If you had to endure one of the following, which would you rather have – a broken leg or a broken arm? Heartburn or asthma? Finally, they rated all the conditions again. Minutes after choosing one particular illness out of many, the volunteers suddenly found that the chosen illness was less intimidating. A broken leg, for example, may have been thought of as "terrible" before choosing it over some other malady. However, after choosing it, the subject would find a silver lining: "With a broken leg, I will be able to lie in bed watching TV, guilt-free."

In our study, we also found that people perceived adverse events more positively if they had experienced them in the past. Recording brain activity while these reappraisals took place revealed that highlighting the positive within the negative involves, once again, a tête-à-tête between the frontal cortex and subcortical regions processing emotional value. While contemplating a mishap, like a broken leg, activity in the rACC modulated signals in a region called the striatum that conveyed the good and bad of the event in question – biasing activity in a positive direction.

It seems that our brain possesses the philosopher's stone that enables us to turn lead into gold and helps us bounce back to normal levels of wellbeing. It is wired to place high value on the events we encounter and put faith in its own decisions. This is true not only when forced to choose between two adverse options (such as selecting between two courses of medical treatment) but also when we are selecting between desirable alternatives. Imagine you need to pick between two equally attractive job offers. Making a decision may be a tiring, difficult ordeal, but once you make up your mind, something miraculous happens. Suddenly – if you are like most people – you view the chosen offer as better than you did before and conclude that the other option was not that great after all. According to social psychologist Leon Festinger, we re-evaluate the options post-choice to reduce the tension that arises from making a difficult decision between equally desirable options.

In a brain-imaging study I conducted with Ray Dolan and Benedetto De Martino in 2009, we asked subjects to imagine going on vacation to 80 different destinations and rate how happy they thought they would be in each place. We then asked them to select one destination from two choices that they had rated exactly the same. Would you choose Paris over Brazil? Finally, we asked them to imagine and rate all the destinations again. Seconds after picking between two destinations, people rated their selected destination higher than before and rated the discarded choice lower than before.

The brain-imaging data revealed that these changes were happening in the caudate nucleus, a cluster of nerve cells that is part of the striatum. The caudate has been shown to process rewards and signal their expectation. If we believe we are about to be given a paycheck or eat a scrumptious chocolate cake, the caudate acts as an announcer broadcasting to other parts of the brain, "Be ready for something good." After we receive the reward, the value is quickly updated. If there is a bonus in the paycheck, this higher value will be reflected in striatal activity. If the cake is disappointing, the decreased value will be tracked so that next time our expectations will be lower.

In our experiment, after a decision was made between two destinations, the caudate nucleus rapidly updated its signal. Before choosing, it might signal "thinking of something great" while imagining both Greece and Thailand. But after choosing Greece, it now broadcast "thinking of something remarkable!" for Greece and merely "thinking of something good" for Thailand.

True, sometimes we regret our decisions; our choices can turn out to be disappointing. But on balance, when you make a decision – even if it is a hypothetical choice – you will value it more and expect it to bring you pleasure.

This affirmation of our decisions helps us derive heightened pleasure from choices that might actually be neutral. Without this, our lives might well be filled with second-guessing. Have we done the right thing? Should we change our mind? We would find ourselves stuck, overcome by indecision and unable to move forward.

The puzzle of optimism

While the past few years have seen important advances in the neuroscience of optimism, one enduring puzzle remained. How is it that people maintain this rosy bias even when information challenging our upbeat forecasts is so readily available? Only recently have we been able to decipher this mystery, by scanning the brains of people as they process both positive and negative information about the future. The findings are striking: when people learn, their neurons faithfully encode desirable information that can enhance optimism but fail at incorporating unexpectedly undesirable information. When we hear a success story like Mark Zuckerberg's, our brains take note of the possibility that we too may become immensely rich one day. But hearing that the odds of divorce are almost one in two tends not to make us think that our own marriages may be destined to fail.

Why would our brains be wired in this way? It is tempting to speculate that optimism was selected by evolution precisely because, on balance, positive expectations enhance the odds of survival. Research findings that optimists live longer and are healthier, plus the fact that most humans display optimistic biases – and emerging data that optimism is linked to specific genes – all strongly support this hypothesis. Yet optimism is also irrational and can lead to unwanted outcomes. The question then is, How can we remain hopeful – benefiting from the fruits of optimism – while at the same time guarding ourselves from its pitfalls?

I believe knowledge is key. We are not born with an innate understanding of our biases. The brain's illusions have to be identified by careful scientific observation and controlled experiments and then communicated to the rest of us. Once we are made aware of our optimistic illusions, we can act to protect ourselves. The good news is that awareness rarely shatters the illusion. The glass remains half full. It is possible, then, to strike a balance, to believe we will stay healthy, but get medical insurance anyway; to be certain the sun will shine, but grab an umbrella on our way out — just in case.

Tali Sharot is a research fellow at University College London's Wellcome Trust Centre for Neuroimaging

© 2011 Tali Sharot

The Human Givens

A link to information about the Human Givens and their importance for wellbeing:

http://www.hgi.org.uk/archive/In_our_own_words.htm