Why do researchers disagree so much about how well psychotherapy works?

On 28 September
2022, NRK, the Norwegian Broadcasting Corporation aired a programme that sent
shock waves into the offices of the country’s therapists.

Presenter
Andreas Wahl considered the effect of psychotherapy – treatment where
conversations with a psychologist are supposed to improve mental health – in
an episode of his tv-programme Folkeopplysningen (Public information).

The conclusion was that the treatment seemed rather ineffective. Only one in seven patients who go
to therapy is actually helped by it.

This triggered a flurry of reactions from psychologists and researchers. Several claimed that, on the contrary, the research shows that psychotherapy has very positive effects. 

How could they
come to such different conclusions?

A deeper dive
into the matter shows that the disagreement runs deep in professional circles.

Better chances of recovery with psychotherapy

The topic is
complicated.

However, based
on summaries of research and conversations with many Norwegian professionals,
we can state one general conclusion to begin with:

Psychotherapy
works. It is widely agreed that people generally have a greater chance of
recovering or getting better from a mental illness if they receive
psychotherapy. It is therefore a good idea to seek help if you are struggling with
your mental health, and there is no reason to stop a treatment that you feel
you benefit from.

The question,
however, is how big this treatment effect is. How many patients improve, and how
many make a full recovery?

That is what
professionals don’t agree on at all.

“I regard it as
essential to give people with mental health problems realistic information
about what they can expect, so that they can make informed choices,” says
Halvorsen.

He believes the
overall research available indicates that psychotherapy generally has a small
to moderate effect. Summaries of the research have stated that the standardized
effect size is between 0.3 and 0.4.

Standardized
effect size is a measurement researchers often use to state how much effect a
treatment has. If a treatment has an effect size of 0.2, it is considered to
have a small effect. An effect size of 0.5 is considered a medium effect, while
a figure above 0.8 is considered a large effect.

An effect of
0.3 is thus not particularly large.

Few patients experience positive effects

Researchers can
also describe the effect by stating how many patients need to receive treatment
in order for one person to experience a clear impact. This number is called NNT,
or the “Number-Needed-to-Treat.”

Halvorsen
believes research shows that the NNT for psychotherapy for depression is seven.
In other words: Only one in seven patients who go to therapy will actually
improve significantly from the treatment.

A similar
picture is presented by Arne Holte, a professor emeritus in health psychology
and former assistant director of the Norwegian Institute of Public Health.

Minority of patients experienced a positive effect

Holte recently
prepared a report for the organization Mental helse (Mental health), in
which he writes that about 40 per cent of patients had a positive treatment
effect after two months in psychotherapy. This treatment effect also applied to
15 per cent of the patients who did not receive talk therapy.

Just over 30
per cent of the patients who received treatment recovered, compared to around 10
per cent of patients who recovered without psychotherapy treatment. The mental
health of five per cent of the participants worsened.

Holte believes
that the matter is complicated, but that the modest effect of psychotherapy might
perhaps be part of the explanation for a paradox: Despite the fact that many
more people are receiving treatment for mental disorders, the incidence of
these ailments is not decreasing.

Psychotherapy has a great impact

Both Holte and
Halvorsen have derived their figures from research. But not everyone agrees
that this is what the research shows.

Ole André
Solbakken, a professor of psychology at the University of Oslo, is one of those
who have a different answer.

“Throughout the
entire history of therapy, across therapists, diagnostic criteria, directions
of therapy and thousands upon thousands of patients, the effect is around 0.8
on average,” he says.

In other words,
therapy has a big impact.

“The best
studies we have indicate that if you manage to deliver psychotherapy within the
best possible framework, with good guidance from the therapist, systematic
follow-up of the patients, and a selection process so that we provide treatment
to the patients who actually want to receive treatment – then we mostly end up
with about a 50 per cent recovery rate,” says Solbakken.

“And 20 per cent
improve, even if they don’t recover completely.”

Too conservative

Sverre Urnes
Johnsen, a professor and clinical psychologist at the University of Oslo and
Modum Bad, also believes that Halvorsen’s low figure for effect gives an
inaccurate picture.

“His low effect
figure is far too conservative,” he says.

“I think an
important message to the general population is that psychotherapy works well
and helps a lot people.”

But how do all
these viewpoints connect? Why do different professionals paint such a wide-ranging
picture of the effect of psychotherapy?

Researchers interpret data differently

The issue depends
on how different researchers interpret the available data.

Many thousands
of studies have been carried out on the effects of a number of different types
of psychotherapy. The studies include investigations into treatment for numerous
diagnoses, including depression and anxiety disorders, which are the diseases
that affect the most people.

When the
researchers report on the effect of therapy, they use systematic reviews of the results from
these individual studies.

And that sounds
good in theory: an average of all the individual results should give a good
estimate of how big the impact is. These systematic reviews are also often considered a
gold standard in research, meaning that these are the studies that should provide
the most reliable answers.

But there’s a
catch.

How good is the quality?

The quality of
the review depends on how good the individual studies are.

If each of the
studies is reliable and of good quality, the systematic review will give more reliable
answers. But in the opposite case, poor individual studies will give unreliable systematic reviews.

In a word:
Garbage in – garbage out.

This is where
one of the great discussions in psychology occurs.

In short, some
researchers believe that the methods that have been used in much of the psychological
research are of too poor quality and that many of the results therefore cannot
be trusted.

Weaknesses that reinforce positive findings

A striking
number of studies in psychology confirm the researcher’s hypotheses – they
typically show that the authors’ own type of treatment has a great effect.

Critics believe
this is not because the researchers are exceptionally accurate, but rather
because many weaknesses in the studies contribute to positive results being
amplified, while the negative ones are understated. (You can read more about
this in a forthcoming article in this series.)

In this way,
the studies may indicate excessively positive results.

And when these
studies are systematically reviewed, the impression can be that the treatments work much
better than what is actually the case.

These are the types
of figures that Halvorsen is referring to when he argues that psychotherapy
generally has a rather modest effect.

Holte also
believes that the studies exaggerate the effects.

But here the
professionals do not agree.

Other
researchers – in Norway and
internationally – believe that Cuijper’s criteria are far too strict and thus
exclude important results. When these other researchers do systematic reviews of study
findings, the results show an effect of around 0.7.

Hope

So how well
does psychotherapy work?

At the moment,
neither systematic reviews nor conversations with experts can provide a uniform
answer to this question.

There is also
no agreement on how this should be communicated to patients.

“In the wake of
Folkeopplysningen, I was told that I’m stealing hope from patients,” says
Halvorsen.

Research
suggests that hope for recovery and faith in the treatment are important
factors for the patients to get better.

“But if
patients come to psychotherapy with the impression that now everything is going
to be so much better, and then it turns out that they are among the many who see
no particular effect, then what are they left with?” Halvorsen asks. 

Realistic expectations

Halvorsen says
he is not aware of research into how expectations affect patients, but says he
has met several patients who think it is their own fault that they are not
benefiting from the treatment.

He believes it
is important to give patients realistic expectations.

“I’m a bit
confused that it’s possible to ignore the quality of research to such a large extent,”
he says.

“I think it
could be that psychologists have spent many years of their lives on this and
want to help, and then we’re presented with results showing that the treatment
works worse than what we’re experiencing ourselves. Taking that in could be
difficult,” says Halvorsen.

Different effects for different diseases

Halvorsen
believes it is also important to mention that expectations should probably be
adjusted according to the type of mental disorder a person has.

There is a lot
of evidence that psychotherapy has different effects on different illnesses.

“Psychotherapy
has a good effect when you consider specific treatment for various anxiety
disorders. A lot of people experience significant benefit. But for a number of
other disorders, like depression, the effect is much less,” says Halvorsen.

Perhaps asking
a general question about how well psychotherapy works is not that interesting.
For example, it wouldn’t be very relevant to ask whether cancer treatment works,
because the answer varies dramatically from cancer to cancer.

No worse than a lot of medical treatment

And on the
topic of medications, the discussions surrounding the effect of psychotherapy
can easily give the impression that this form of treatment is much less
effective than somatic treatment – that is, treating physical ailments with
drugs or other medical therapy.

Not all somatic
treatment has a very good average effect either. Neither painkillers nor
allergy medicines work well for everyone who takes them.

“A lot of people
have the funny idea that when they go to their doctor and pick up some medication
it always has a positive effect. But that isn’t true,” Solbakken says.

He believes
there’s no reason to believe that psychotherapy generally has less effect than
medical treatment. And here Halvorsen agrees.

“Overall,
psychotherapy isn’t any less effective than a lot of somatic treatment,” he
says.

——

Translated by: Ingrid P. Nuse

Read the Norwegian version of this article on forskning.no 

Pictures:

Illustration
photo of psychologist and patient: Megaflopp/Shutterstock/NTB
Photo by Joar Øveraas Halvorsen: Frode Nikolaisen
Illustration
photo in black and white of a man: KieferPix/Shutterstock/NTB