Predictors of Long-Term Improvement Following Cognitive Remediation in a Sample With Elevated Depressive Symptoms

fpsyg-11-02232 September 9, 2020 Time: 22:5 # 1

ORIGINAL RESEARCH

published: 11 September 2020

doi: 10.3389/fpsyg.2020.02232

Edited by:

Peter L. Fisher,

University of Liverpool,

United Kingdom

Reviewed by:

Tobias Kube,

Harvard Medical School,

United States

Brooke Schneider,

University Medical Center

Hamburg-Eppendorf, Germany

*Correspondence:

Bjørn Ingulfsvann Hagen

[email protected]

Specialty section:

This article was submitted to

Psychology for Clinical Settings,

a section of the journal

Frontiers in Psychology

Received: 28 April 2020

Accepted: 10 August 2020

Published: 11 September 2020

Citation:

Hagen BI, Landrø NI, Lau B,

Koster EHW and Stubberud J (2020)

Predictors of Long-Term Improvement

Following Cognitive Remediation in a

Sample With Elevated Depressive

Symptoms. Front. Psychol. 11:2232.

doi: 10.3389/fpsyg.2020.02232

Predictors of Long-Term

Improvement Following Cognitive

Remediation in a Sample With

Elevated Depressive Symptoms

Bjørn Ingulfsvann Hagen

, Nils Inge Landrø

, Bjørn Lau

, Ernst H. W. Koster

and

Jan Stubberud

1,2

Department of Research, Lovisenberg Diaconal Hospital, Oslo, Norway,

Department of Psychology, University of Oslo,

Oslo, Norway,

Department of Experimental Clinical and Health Psychology, Ghent University, Ghent, Belgium

Objective: Cognitive remediation (CR) techniques (interventions to enhance cognitive

functioning) have proven moderately effective in improving cognition and daily

functioning in major depressive disorder (MDD). However, baseline predictors of

treatment response are lacking. The present study aimed to identify factors inﬂuencing

long-term CR outcomes in a sample with current or previous, mild or moderate MDD

and with self-reported cognitive deﬁcits.

Methods: Forty-two completers of group-based CR (strategy learning or drill-and-

practice), were pooled into one sample. Based on change scores from baseline to

6-month follow-up, participants were categorized as “improvers” or “non-improvers”

using reliable change index calculations. Measures included a questionnaire of everyday

executive functioning and a neuropsychological test of attention. Finally, improvers and

non-improvers were compared in terms of various sociodemographic, psychological,

illness-related, and neuropsychological baseline variables.

Results: Seventeen participants improved reliably in everyday executive functioning,

and fourteen demonstrated a reliable improvement in attention. No statistically signiﬁcant

differences emerged between improvers and non-improvers.

Conclusion: No major predictors of CR were identiﬁed. Importantly, the current ﬁndings

are insufﬁcient to guide clinical decision-making. Large-scale studies with a priori

hypotheses are needed to make advances in the future.

Keywords: depression, cognitive remediation, treatment predictors, executive functions, attention

INTRODUCTION

Major depressive disorder (MDD) is characterized by deﬁcits in cognitive functions, including

attention, memory, and executive functions (EF) (Snyder, 2012; Ahern and Semkovska, 2017).

However, the heterogeneity of the cognitive proﬁle in MDD appears to be large, with distinct

neurocognitive subgroups (Pu et al., 2018). For those experiencing cognitive diﬃculties, deﬁcits

often persist into remission (Rock et al., 2014) and have a deleterious eﬀect on everyday

functioning (Baune et al., 2010). Cognitive deﬁcits, particularly in EF, are additionally associated

Frontiers in Psychology | www.frontiersin.org 1 September 2020 | Volume 11 | Article 2232

fpsyg-11-02232 September 9, 2020 Time: 22:5 # 2

Hagen et al. Predictors of Cognitive Remediation Outcome

with unfavorable depression outcomes, such as impaired

long-term recovery (Vicent-Gil et al., 2018) and reduced

antidepressant medication eﬀectiveness (Groves et al., 2018).

Hence, cognitive functioning represents a potential treatment

target in MDD (Kaser et al., 2017).

Nonetheless, both antidepressant medication and cognitive–

behavioral therapy (CBT) show limited eﬀectiveness in alleviating

cognitive deﬁcits (Porter et al., 2016; Shilyansky et al.,

2016). Cognitive remediation (CR) interventions – speciﬁcally

targeting cognitive dysfunction – intended to produce lasting

improvements in everyday functioning are thus emerging.

Although the heterogeneity of treatment approaches labeled

“CR” is large (Motter et al., 2016), these interventions can be

divided into either bottom-up drill-and-practice approaches or

top-down approaches focusing on strategy learning. Bottom-up

approaches typically consist of computerized cognitive training

(CCT) tasks intended to improve basic cognitive processes

through the process of neuroplasticity (Motter et al., 2016). In

contrast, top-down approaches consist of learning compensatory

strategies for wide appliance in daily living, to compensate for

the cognitive diﬃculties. Findings indicate moderate eﬀectiveness

in improving cognition and everyday functioning following CR,

but there is a paucity of evidence for improved EF or on its long-

term eﬀects (for a meta-analysis, see Motter et al., 2016). In this

context, it has been argued that the substantial heterogeneity in

the cognitive proﬁle of MDD may inﬂuence the eﬀectiveness of

CR (Koster et al., 2017).

The lack of factors associated with successful treatment

outcomes may be a barrier to improving CR eﬀectiveness

(Motter et al., 2016; Koster et al., 2017). The identiﬁcation of

pretreatment predictors could improve eﬃcacy by facilitating

individualized clinical decision-making. Moreover, it may be

helpful for the development of future treatments by providing

insight into the mechanisms of CR interventions (Koster et al.,

2017). The investigation of CR predictors and moderators

in MDD is scarce, but a meta-analysis covering a range of

CCT interventions found decreased treatment eﬀectiveness with

increased age (Motter et al., 2016). Additional variables, such

as gender and receiving concurrent treatment (antidepressant

medication or psychotherapy), did not signiﬁcantly inﬂuence

outcomes. In a recent study dedicated to identifying the

predictors in a CR intervention consisting of both CCT and

strategy learning, Listunova et al. (2020) observed a shorter

duration of illness to be the only factor associated with

improvement on a neuropsychological measure of attention in

a partially remitted MDD sample. Several sociodemographic,

neurocognitive, psychopathological, and training-speciﬁc factors

thus failed to predict outcomes. However, the study was

limited by its exploratory approach, modest sample size, and

exclusive focus on the attention domain (Listunova et al.,

2020). Interestingly, CR ﬁndings diverge from psychotherapy

research in MDD, where illness characteristics such as greater

depression symptom severity, younger age at onset, and more

previous episodes have all been associated with poorer responses

in relation to depressive symptom alleviation (Hamilton and

Dobson, 2002). Moreover, in schizophrenia research, where

predictors and moderators of CR eﬀectiveness have been

more frequently studied, most of the factors reviewed fail to

signiﬁcantly inﬂuence CR treatment response (Reser et al.,

2019; Seccomandi et al., 2020). However, in a selection

of studies, better baseline performance in several cognitive

domains predicted both cognitive and functional improvement,

while increased chronicity and severity of schizophrenia has

been associated with worse CR outcomes (Medalia and

Richardson, 2005; Kurtz et al., 2009; Vita et al., 2013;

Lindenmayer et al., 2017).

The main aim of the present study was to explore whether

a selection of sociodemographic, neuropsychological, illness-

related, and psychological variables could predict long-term

CR outcomes in a sample with current or previous mild or

moderate MDD. Data were collected as part of a single-blind

randomized controlled trial (RCT) comparing the eﬀectiveness

of a strategy-based CR approach, Goal Management Training

(GMT), with drill-and-practice CCT, in improving EF (Hagen

et al., 2020). Both groups improved on measures of EF in daily

life, neuropsychological tests of EF and attention, and depression

symptom severity following CR. That is, no signiﬁcant diﬀerences

emerged between groups in the original study, although within-

group changes in everyday EF and depression symptom severity

were only signiﬁcant following GMT. Owing to the limited

number of previous studies examining predictors of CR in MDD,

the present study applied an exploratory approach with no

a priori hypothesis.

MATERIALS AND METHODS

The original RCT was preregistered at clinicaltrials.gov with the

identiﬁer NCT03338413, and the study protocol was approved by

the Regional Committee for Medical and Health Research Ethics,

South-Eastern Norway (2017/666). The study was conducted

following the World Medical Association’s Declaration of

Helsinki, and all participants gave their written informed consent.

For more detailed information on the methodological approach

of the original RCT study, see Hagen et al. (2020).

Participants

The sample (n = 63) included participants diagnosed with mild

or moderate MDD according to International Classiﬁcation

of Diseases – 10th Revision (ICD-10) criteria (World Health

Organization, 2004), either as a primary or as a secondary

diagnosis. All participants had undergone a diagnostic

evaluation and completed treatment at the Return-to-Work

clinic at Lovisenberg Diaconal Hospital within 2 years

before inclusion. The Return-to-Work clinic oﬀers short-

term outpatient psychotherapeutic treatment to patients

with mental health issues of mild to moderate severity, at

risk of receiving sick leave because of mental illness. In

addition, to be included, participants had to be between 18

and 60 years of age and to have self-reported everyday EF

deﬁcits (e.g., diﬃculties with memory, organizing/planning,

emotional regulation, and/or concentration) in a custom-

made telephone interview. All participants were additionally

asked to conﬁrm that depression symptoms represent a

Frontiers in Psychology | www.frontiersin.org 2 September 2020 | Volume 11 | Article 2232

fpsyg-11-02232 September 9, 2020 Time: 22:5 # 3

Hagen et al. Predictors of Cognitive Remediation Outcome

major mental health complaint. Exclusion criteria included

comorbid neurological conditions, ongoing alcohol or substance

abuse, and severe cognitive problems or mental disorders

(psychotic disorders and severe personality disorders).

No cutoﬀ scores were speciﬁed for current depressive

symptom severity. There were no restrictions on participants

concerning additional concurrent psychotherapeutic or

antidepressant treatment.

Study Design and Blinding

All participants completed a baseline assessment (T1) before

being randomly assigned to nine sessions of either GMT

or CCT, using computer-generated simple randomization.

Participants were reassessed immediately following treatment

completion (T2) and at a 6-month follow-up (T3). The

assessments consisted of neuropsychological tests and self-

report rating scales. Assessments were not blind, because

the person responsible for data collection also acted as a

therapist in both interventions. To compensate, an external

assessor performed a limited set of blind T3 assessments

(n = 5) for comparison. However, the study was single-

blind because participants were not informed whether they

had been allocated to the condition considered to be the

active treatment.

Cognitive Remediation Interventions

Goal Management Training

GMT is a manual-based CR intervention to improve everyday

EF (Levine et al., 2011). A central element of GMT is to learn

and internalize strategies for wide application in daily living,

promoting goal-directed behavior through increased executive

control and improved problem-solving capacities. Strategies

consist of a self-instruction to stop ongoing behavior, check

the current content of working memory, state and deﬁne

goals, apply a systematic approach to problem-solving, and

monitor performance.

In the present study, the Norwegian translation of the

standard GMT protocol was employed (Stubberud et al.,

2013; Tornås et al., 2016). A clinical psychologist and a

neuropsychologist delivered GMT in groups of ﬁve to seven

participants in 9 weekly 2 h sessions. In-class exercises included

practicing the use of the compensatory strategies (e.g., practice a

systematic approach to problem-solving by arranging a ﬁctitious

wedding party). Mindfulness exercises (Kabat-Zinn, 1990),

intended to enhance attentional control, were also practiced in

class. Sessions emphasized group discussions addressing personal

examples of dysexecutive behavior. Between-session assignments

included monitoring EF-related errors, mindfulness exercises,

and the application of learned strategies in daily life (Table 1).

Computerized Cognitive Training

The CCT consisted of seven exercises from the BrainHQ

platform. Repetition is the hallmark feature of CCT, and

neuroplasticity is its theoretical foundation (Siegle et al., 2007).

Cognitive improvements, including EF, have been identiﬁed

using BrainHQ or similar exercises (Morimoto et al., 2014;

Lewandowski et al., 2017), and these studies established the

empirical basis for the selection of exercises. In addition, exercise

selection was based on the provider’s description

The CCT consisted of nine twice-weekly 1 h sessions by groups

of three participants. Exercises targeted attention, memory,

processing speed, and EF. To ensure appropriate levels of mastery

and frustration, the platform adapted diﬃculty levels to the

individual participants’ performance, keeping the success rate

at 80% throughout. A clinical psychologist acted as a therapist

and gave participants positive feedback on their eﬀorts. The ﬁrst

session included psychoeducation, with the therapist introducing

the concept of neuroplasticity, typical cognitive deﬁcits in

depression, and the importance of cognitive processes in diﬀerent

everyday situations. Participants had online access to the training

platform and were encouraged to practice for at least 30 min

between each session (Table 1).

Completer Sample

Participants had to attend a minimum of six training sessions

and complete the 6-month follow-up assessment (T3) to be

included in the completer sample. Forty-two completers from

both groups were pooled into one sample. The pooling of

participants receiving diﬀerent treatments was done to increase

the sample size (Figure 1).

Outcome Measures

The Global Executive Composite (GEC) from the Behavior

Rating Inventory of Executive Function – Adult version (BRIEF-

A) (Roth et al., 2005) was applied as an outcome of self-reported

everyday EF. The BRIEF-A GEC consists of 70 items and nine

non-overlapping subscales (Inhibit, Self-Monitor, Plan/Organize,

Shift, Initiate, Task Monitor, Emotional Control, Working

Memory, Organization of Materials), tapping the frequency of

everyday dysexecutive behavior (item range 1–3, total range 70–

210). The psychometric properties of the GEC are acceptable,

with a 1-month test–retest reliability of 0.94 and a Cronbach’s

alpha of 0.96 (Roth et al., 2005).

The Conners Continuous Performance Test – Third edition

(CPT-3) (Conners, 2015) was applied as a neuropsychological

measure of attention. The CPT-3 is a 14 min go/no-go

test of visual attentiveness, response inhibition, and sustained

attention. The number of commission errors (Commissions;

response to “no-go” targets: attentiveness and inhibition) and

hit reaction time standard deviation (HRT SD: response

consistency/sustained attention) subscales were included as

outcome measures. The corrected test–retest reliabilities (1–5

weeks) of the included subscales are 0.68 (HRT SD) and 0.85

(Commissions) (Conners, 2015). A neuropsychological measure

of attention was selected as an outcome to facilitate comparison

with the only previous study that we know of that investigated the

predictors of CR response in MDD (Listunova et al., 2020).

Predictors of Treatment Effect

Sociodemographic Factors

Sociodemographic factors included age, gender, years of

education, and employment status, all self-reported in a

https://www.brainhq.com

Frontiers in Psychology | www.frontiersin.org 3 September 2020 | Volume 11 | Article 2232

fpsyg-11-02232 September 9, 2020 Time: 22:5 # 4

Hagen et al. Predictors of Cognitive Remediation Outcome

TABLE 1 | GMT sessions, CCT tasks, and objectives.

GMT session Objective CCT task Objective

The present and the absent mind Tracking absentmindedness/Practice mindfulness techniques Double decision Attention

Inattentive Errors Condition for, and consequences of, absentmindedness Divided attention Attention/Inhibition

The Automatic Pilot How automatic behavior leads to inappropriate responding Target tracker Attention

Stop the automatic pilot Make a habit of stopping/Bring attention to the present Syllable stacks Working memory

The mental blackboard Check the content of working memory Scene crasher Working memory

State your goal State goals to facilitate goal maintenance Face-to-Face Processing Speed/ Social Cognition

Making decisions Goal conﬂict in decision-making/Making to-do lists Sound sweeps Processing speed

Splitting tasks into subtasks Split overwhelming tasks into subtasks/Step-by-step approach

Checking (STOP!) Adapting behavior to situational change/Summary of training

GMT, Goal Management Training; CCT, computerized cognitive training.

custom-made interview. Employment status was transformed

into a dichotomous variable with the categories “full-time

employment/full-time student” and “other” (including

“part-time employment/part-time student,” “sick leave,” and

“looking for a job”).

Illness-Related Factors

Illness-related factors included current depressive symptom

severity in addition to the self-reported number of previous

depressive episodes, age of onset, duration of illness, and current

antidepressant medication use. Depressive symptom severity was

assessed with the Beck Depression Inventory (BDI) (Beck et al.,

1961), which has satisfactory internal consistency (Beck et al.,

1988). The remaining illness-related factors were self-reported

during a custom-made interview. The number of previous

episodes was transformed into a dichotomous variable based on

the categories of “one episode” and “more than one episode,” as

this was considered a theoretically meaningful subdivision of the

highly skewed original continuous variable.

Psychological Factors

Psychological factors included overall psychological distress and

a tendency to ruminate. The Clinical Outcomes in Routine

Evaluation – Outcome Measure (CORE-OM) (Barkham et al.,

2001) was applied as a measure of overall psychological distress.

The CORE-OM clinical score was calculated as a mean of

completed items multiplied by 10 (range: 0–40). Rumination was

assessed using the Ruminative Response Scale (RRS) (Treynor

et al., 2003). Both questionnaires have acceptable internal

consistency (Barkham et al., 2001; Treynor et al., 2003).

Neuropsychological Factors

Neuropsychological factors included estimated IQ, assessed with

the two-subtest form (Matrix reasoning; Vocabulary) of the

Wechsler Abbreviated Scale of Intelligence (WASI) (Wechsler,

1999). Performance on the Delis–Kaplan Executive Function

System (D-KEFS) (Delis et al., 2001) Trail-Making Test was

applied as a measure of processing speed (condition 2) and

EF/shifting (condition 4). Memory was assessed using the

California Verbal Learning Test–Second edition – Short form

(CVLT-II SF) (Delis et al., 2000). The digit span forward

(attention span) and digit span backward (working memory)

subtests of the Wechsler Adult Intelligence Scale – Fourth edition

(WAIS-IV) (Wechsler, 2014), were also employed.

Other Factors

Received intervention (GMT or CCT) was included as a training-

speciﬁc factor. In addition, baseline performance scores on the

outcome variables were included as predictors.

Statistical Analysis

Calculation of Reliable Change Index

The reliable change index (RCI) (Jacobson and Truax, 1991)

was calculated for everyday EF (BRIEF-A GEC) and the

neuropsychological measure of attention (CPT-3: Commissions,

HRT SD). The RCI analysis is a statistical approach to

identify individuals with statistically reliable improvement,

given the scale reliability. Thus, the approach is sensitive to

individual participant improvements potentially lost in group-

level statistical analysis (Jacobson and Truax, 1991). To calculate

the RCI, a change in individual raw score (BRIEF-A) or T-score

(CPT-3) between T1 (X

) and T3 (X

) was divided by the

standard error of the diﬀerence (SE

diﬀ

) using the formula:

RCI = (X

− X

)/SE

diﬀ

was derived from the standard error of measurement (S

calculated using the test–retest reliability (r

) of the instrument,

and the standard deviation (SD) using the following formulas:

= S

(1 − r

) and SE

diﬀ

2 (S

)

An RCI smaller than –1.96 (because of measurement

direction) was required to be considered as a reliable

improvement. A change surpassing the ±1.96 threshold

occurs by chance in only 5% of cases. Information on test–retest

reliability and standard deviation was collected from the test

manuals (Roth et al., 2005; Conners, 2015). The decision to

calculate change scores from baseline (T1) to the 6-month

follow-up (T3) was because long-term outcomes were regarded

as most clinically relevant. For the CPT-3, a reliable improvement

on one of the two subscales (Commissions or HRT SD) was

required to count as an improvement. Finally, participants

improving reliably on one subscale were not included if there

was a reliable deterioration on the other.

Frontiers in Psychology | www.frontiersin.org 4 September 2020 | Volume 11 | Article 2232

fpsyg-11-02232 September 9, 2020 Time: 22:5 # 5

Hagen et al. Predictors of Cognitive Remediation Outcome

FIGURE 1 | CONSORT diagram (Schulz et al., 2010).

Comparison of Improvers and Non-improvers

For each of the two outcome measures, participants were

categorized as either “improvers” or “non-improvers” based

on their RCI score. Improvers were compared with non-

improvers in the pooled completer samples using the non-

parametric Mann–Whitney U test and chi-square test, for

pairs of continuous and dichotomous variables, respectively.

In addition, the T3 results on the outcome measures were

compared between assessors (blind/non-blind), and T1 results

between completers and non-completers, using the Mann–

Whitney U test. All tests were two-tailed, and to partially

account for multiple testing, the signiﬁcance level was set

to 0.01. Values between 0.01 and 0.05 were interpreted

as trends. SPSS version 24.0 for Windows was applied

for all analyses.

RESULTS

The completers (n = 42) had a median age of 41 years (range = 28–

59) and a median of 15 years of education (range = 9–

18). The majority were female (79.1%), and 76.7% did not

Frontiers in Psychology | www.frontiersin.org 5 September 2020 | Volume 11 | Article 2232

fpsyg-11-02232 September 9, 2020 Time: 22:5 # 6

Hagen et al. Predictors of Cognitive Remediation Outcome

TABLE 2 | Comorbid ICD-10 diagnoses in the completer sample.

Disorder Frequency

Generalized anxiety disorder (F 41.1) 6

Panic disorder (F 41.0) 3

Post-traumatic stress disorder (F 43.1) 1

Agoraphobia (F 40.0) 1

Social phobia (F 40.1) 1

Anxiety disorder, unspeciﬁed (F 41.9) 1

Nonorganic insomnia (F 51.0) 2

Mental and behavioral disorders due to

psychoactive substance use (F 10–F 19)

Number of participants with comorbidities, n = 14. Multiple comorbid diagnoses

are possible for each participant. ICD-10 = International Classiﬁcation of Diseases –

10th Revision.

currently use antidepressant medication. Furthermore, their

average depression symptom severity was in the mild range (BDI:

median = 17.0, range = 4.0–34.0) (Beck et al., 1988). Fourteen

completers had been diagnosed with a comorbid ICD-10 mental

or behavioral disorder (Table 2).

The sample reported substantial executive dysfunction in daily

living (BRIEF-A GEC T-score: median = 64, range = 44–80)

but performed in the normal range for the included CPT-

3 subscales at baseline (Commissions T-score: median = 48,

range = 35–73; HRT SD T-score: median = 44, range = 33–

75). At follow-up, completers reported overall fewer EF

deﬁcits in daily life (BRIEF-A GEC T-score: median = 59.5,

range = 36–78) and performed better on the measure of attention

(CPT-3: Commissions T-score, median = 44, range = 25–

71; HRT SD T-score, median = 41, range = 31–56). No

statistically signiﬁcant diﬀerences emerged for any of the

outcome measures at follow-up between the blinded and non-

blinded assessors. Finally, the completers were not signiﬁcantly

diﬀerent from the non-completers (n = 21) on any of the

included variables.

Comparison of Everyday Executive

Functioning of Improvers and

Non-improvers

Seventeen participants (40.5%) were identiﬁed to improve

reliably on the BRIEF-A GEC between T1 and T3. For

participants to surpass the critical value for improvement,

a 13-point reduction in BRIEF-A GEC raw score was

required. The mean BRIEF-A GEC raw score of improvers

at follow-up was 107.9 (SD = 17.9). No statistically signiﬁcant

diﬀerences emerged between improvers and non-improvers

for any of the predictors. However, improvers had a

higher estimated IQ than non-improvers at trend level

(p = 0.044) (Table 3).

Comparison of Attention by Improvers

and Non-improvers

Fourteen participants (33.3%) were identiﬁed as improvers on

the measure of attention. Twelve participants improved on the

Commissions subscale and four on the HRT SD subscale, while

TABLE 3 | Characteristics of improvers and non-improvers in everyday EF.

Variable Improvers Non-improvers p-value

(n = 17) (n = 25)

Categorical variables

Gender (female) 14 19 0.622

Work status (full-time) 11 18 0.616

Antidepressant use (yes) 2 8 0.131

Dep. ep. (1 ep.) 5 9 0.754

Received intervention (GMT) 10 13 0.663

Continuous variables Mean (SD) Mean (SD)

Age (years) 44.9 (8.7) 40.8 (8.5) 0.098

Years of education 15.5 (2.0) 14.4 (2.1) 0.071

Age of onset (years) 32.7 (12.9) 26.0 (10.3) 0.087

Illness duration (years) 12.2 (10.4) 14.8 (9.7) 0.317

WASI (IQ estimate) 115.3 (6.7) 108.8 (12.1) 0.044

TMT 2 (seconds) 30.8 (10.2) 31.4 (9.7) 0.488

TMT 4 (seconds) 71.2 (21.6) 84.1 (34.6) 0.109

CVLT-II SF Long delay 8.4 (1.2) 8.3 (0.8) 0.299

WAIS-IV DS Forward 9.7 (2.4) 9.1 (1.5) 0.539

WAIS-IV DS Backward 10.0 (2.3) 9.8 (1.8) 0.815

BDI 19.0 (6.7) 15.2 (7.1) 0.063

RRS 56.3 (10.5) 55.0 (12.2) 0.710

CORE-OM 14.7 (5.5) 13.4 (6.0) 0.522

CPT-3 Commissions (T-score) 50.2 (11.3) 51.5 (8.4) 0.336

CPT-3 HRT SD (T-score) 46.6 (9.8) 44.8 (9.8) 0.293

BRIEF-A GEC 137.3 (17.9) 128.6 (15.2) 0.124

SD, standard deviation, Dep., depression; ep., episode; GMT, Goal Management

Training; WASI, Wechsler Abbreviated Scale of Intelligence; TMT, Trail-Making

Test; CVLT-II SF, California Verbal Learning Test–Second edition – Short form;

WAIS-IV, Wechsler Adult Intelligence Scale – Fourth edition; DS, digit span; BDI,

Beck Depression Inventory; RRS, Ruminative Response Scale; CORE-OM, Clinical

Outcomes in Routine Evaluation – Outcome Measure; CPT, Conners Continuous

Performance Test; HRT, hit reaction time; BRIEF-A GEC, Behavior Rating Inventory

of Executive Function – Adult version Global Executive Composite; EF, executive

functions. All scores reported are raw scores, except when otherwise speciﬁed.

Higher scores indicate greater impairment, except for WASI, CVLT-II, and WAIS-IV.

two participants improved on both subscales. For participants to

surpass the critical value for improvement, a change in T-score

of 10 (Commissions) or 13 (HRT SD) was required. At follow-

up, the mean T-score for the improver group (n = 14) was 42.2

(SD = 6.5) on the Commission subscale and 39.9 (SD = 7.5)

on the HRT SD subscale. No statistically signiﬁcant diﬀerences

emerged between improvers and non-improvers. However, at

trend level, fewer of the improvers compared with the non-

improvers (p = 0.011) had experienced only one previous

depressive episode (Table 4). The mean number of self-reported

previous depressive episodes was 4.8 (SD = 3.3) for improvers

and 3.4 (SD = 3.2) for non-improvers (values > 10 were

recoded > 10 = 10).

Overlap Between Improvers Across

Outcomes

Fifteen participants (35.7%) improved on neither measure, and

four participants (9.5%) improved reliably on both the BRIEF-A

GEC and the CPT-3.

Frontiers in Psychology | www.frontiersin.org 6 September 2020 | Volume 11 | Article 2232

fpsyg-11-02232 September 9, 2020 Time: 22:5 # 7

Hagen et al. Predictors of Cognitive Remediation Outcome

TABLE 4 | Characteristics of improvers and non-improvers in attention.

Variable Improvers Non-improvers p-value

(n = 14) (n = 28)

Categorical variables

Gender (female) 11 22 0.999

Work status (full-time) 10 19 0.813

Antidepressant use (yes) 1 9 0.073

Dep. ep. (1 ep.) 1 14 0.011

Received intervention (GMT) 7 16 0.661

Continuous variables Mean (SD) Mean (SD)

Age (years) 42.9 (9.4) 42.3 (8.5) 0.915

Years of education 15.1 (2.1) 14.8 (2.1) 0.539

Age of onset (years) 29.3 (13.6) 28.4 (11.0) 0.957

Illness duration (years) 13.6 (9.3) 13.9 (10.4) 0.947

WASI (IQ-estimate) 114.9 (6.4) 109.6 (12.0) 0.194

TMT 2 (seconds) 30.3 (7.6) 31.5 (10.8) 0.904

TMT 4 (seconds) 79.1 (21.4) 79.2 (34.3) 0.566

CVLT-II SF Long delay 8.3 (0.9) 8.3 (1.0) 0.801

WAIS-IV DS Forward 10.1 (2.1) 9.0 (1.7) 0.092

WAIS-IV DS Backward 10.5 (1.8) 9.6 (2.0) 0.151

BDI 18.8 (6.5) 15.7 (7.3) 0.169

RRS 58.4 (10.7) 54.1 (11.7) 0.279

CORE-OM 16.5 (4.9) 12.6 (5.9) 0.056

CPT-3 Commissions (T-score) 54.8 (9.3) 49.0 (9.3) 0.050

CPT-3 HRT SD (T-score) 49.5 (13.3) 43.5 (6.7) 0.224

BRIEF-A GEC 137.6 (14.5) 129.4 (17.3) 0.157

SD, standard deviation; Dep., depression; ep., episode; GMT, Goal Management

Training; WASI, Wechsler Abbreviated Scale of Intelligence; TMT, Trail-Making

Test; CVLT-II SF, California Verbal Learning Test–Second edition – Short form;

WAIS-IV, Wechsler Adult Intelligence Scale – Fourth edition; DS, digit span; BDI,

Beck Depression Inventory; RRS, Ruminative Response Scale; CORE-OM, Clinical

Outcomes in Routine Evaluation – Outcome Measure; CPT, Conners’ Continuous

Performance Test; HRT, hit reaction time; BRIEF-A GEC, Behavior Rating Inventory

of Executive Function – Adult version Global Executive Composite. All scores

reported are raw scores, except when otherwise speciﬁed. Higher scores indicate

greater impairment, except for WASI, CVLT-II, and WAIS-IV.

DISCUSSION

The present study aimed to identify factors predicting long-term

treatment outcomes following CR in an MDD sample. None of

the variables emerged as major predictors of change in either

everyday EF or attention. The lack of factors associated with CR

improvement is generally consistent with previous research on

MDD (Motter et al., 2016; Listunova et al., 2020).

Even though none of the illness-related factors emerged

as major predictors, surprisingly, a reliable improvement in

attention was associated at trend level with having experienced

more than one previous depressive episode. Recurrence of

episodes arguably indicates greater illness severity and chronicity,

previously associated with reduced CR eﬀectiveness for both

MDD (Listunova et al., 2020) and schizophrenia (Medalia and

Richardson, 2005; Vita et al., 2013; Lindenmayer et al., 2017).

This result thus diverges from a selection of previous ﬁndings.

However, contrary to conclusions from systematic reviews in

MDD (Rock et al., 2014; Ahern and Semkovska, 2017), the

present sample did not display objective attention deﬁcits at

baseline. Additionally, previous research has suggested distinct

neurocognitive subgroups for MDD, with a majority showing

near-normative performance on neuropsychological tests (Pu

et al., 2018). Participants in the Return-to-Work program

report less overall illness severity and are more likely to hold

a job compared with other outpatients (Victor et al., 2016).

In addition, IQ estimates were above average in the present

sample. Such sample characteristics could have contributed

to normal performance on the cognitive measures (Elgamal

et al., 2010; Venezia et al., 2018). Furthermore, owing to the

weak correlations between self-reported and neuropsychological

measures of cognition in MDD, the inclusion based on subjective

deﬁcits may have also resulted in a subgroup of cognitively

unimpaired participants (Petersen et al., 2019). Notably, at

baseline, the non-improver group performed even better than

improvers on the outcome measure of attention and may as

such represent a part of the sample without actual attention

deﬁcits. The non-improvers were thus less likely to surpass the

threshold for improvement, because their baseline left little room

for further gains. Indeed, not accounting for individual diﬀerence

in baseline performance is a limitation of the RCI approach

(Duﬀ, 2012). Moreover, although overall results are mixed, some

previous ﬁndings indicate that MDD recurrence is related to

impaired performance on measures of cognition (Hasselbalch

et al., 2011), and this could potentially explain why recurrent

episodes were associated with attention improvement in our

sample when members of this group had the opportunity to

improve their attention as a result of the interventions.

The present study failed to replicate the single signiﬁcant

ﬁnding from a previous meta-analysis investigating moderators

of CR outcomes in MDD, namely, that treatment eﬀectiveness

decreases with increasing age (Motter et al., 2016). Excluding

participants above the age of 60 years restricted the age range and

reduced the sample variability of the present study, potentially

limiting the prospect of obtaining signiﬁcant results. Nonetheless,

the overall available evidence does not indicate that age is a

reliable predictor of CR outcomes (Reser et al., 2019; Listunova

et al., 2020; Seccomandi et al., 2020). However, in accordance

with previous research on MDD (Motter et al., 2016; Listunova

et al., 2020), neither gender nor receiving concurrent treatments

predicted improvements in cognition or everyday EF, with the

latter indicating no additive eﬀect on cognition of combining CR

with antidepressant medication.

Cognitive performance at baseline did not predict outcomes

for attention or EF in our sample. This ﬁnding is contrary to

a selection of ﬁndings regarding schizophrenia (Reser et al.,

2019) but in accordance with MDD research (Listunova et al.,

2020). One notable exception was that higher IQ estimates

were associated with improvement in everyday EF at the trend

level. Theoretically, a greater general ability may contribute to

reaching one’s potential for applying learned strategies in daily

living, thereby increasing CR eﬀectiveness (Velligan et al., 2006).

Hence, the role of IQ in CR should be further investigated

in future studies.

Delivering CR therapies to patients with MDD hinges on the

theoretical assumption that cognitive deﬁcits impair everyday

Frontiers in Psychology | www.frontiersin.org 7 September 2020 | Volume 11 | Article 2232

fpsyg-11-02232 September 9, 2020 Time: 22:5 # 8

Hagen et al. Predictors of Cognitive Remediation Outcome

functioning and act as risk factors for depressive symptoms.

Thus, it is striking that cognitive performance at baseline

(i.e., the degree of the cognitive deﬁcits) lacks support as a

moderator of CR outcome (Koster et al., 2017). No established

neuropsychological proﬁle exists for MDD (Marazziti et al.,

2010), and this heterogeneity could limit the chance of detecting

reliable pretreatment cognitive predictors. Furthermore, the

relationship of cognitive factors with outcomes may be non-

linear, exerting diﬀerent inﬂuences at diﬀerent levels of each

variable. To illustrate, higher baseline cognitive performance may

be conceptualized both as facilitating CR gains and as restricting

improvement potential (Twamley et al., 2011; Vita et al., 2013).

Rumination is proposed to have a bidirectional relationship

with EF (Davis and Nolen-Hoeksema, 2000; Philippot and

Brutoux, 2008), and in CCT interventions speciﬁcally addressing

EF processes (i.e., cognitive control training), rumination has

been found to mediate depressive symptom outcomes (Quinn

et al., 2014). To our knowledge, no previous study has

investigated whether baseline rumination predicts CR outcomes

in cognition or functioning. Our ﬁndings suggest that baseline

rumination is not a major predictor of improvements in attention

or everyday EF. However, important subcomponents of the

rumination construct have been identiﬁed (Treynor et al., 2003)

but were not presently investigated.

Pooling participants who received CR interventions that

diﬀered in content and theoretical foundation was necessary to

obtain an acceptable sample size. This may have obscured the

eﬀect of predictors on each treatment. Nevertheless, the number

of improvers was similar across interventions for both outcome

measures in the present study; moreover, a previous meta-

analysis of schizophrenia has indicated that diﬀerent approaches

produce similar overall eﬀects on measures of cognition (Wykes

et al., 2011), suggesting commonalities between treatments.

Clinical Implications and Future

Directions

The percentage of improvers in the present study (33.3–

40.5%) and previous studies (34.2%) (Listunova et al., 2020)

indicates the potential to increase CR eﬀectiveness in MDD. The

heterogeneity of cognitive deﬁcits in depression suggests that

individualized interventions may be required, and understanding

why participants achieve diﬀerent outcomes represents a critical

hurdle to individualizing CR. However, identifying easily

available major predictors of treatment outcomes has proven to

be a challenge, and current ﬁndings are insuﬃcient to guide

clinical decision-making. Moreover, no consistent barriers to

improvement have been identiﬁed to date, so these ﬁndings

suggest that MDD patients have the potential to improve

following CR, regardless of their baseline characteristics. For

advances in the ﬁeld, large-scale and ﬁne-grained investigations

with a priori hypotheses are required (Reser et al., 2019;

Seccomandi et al., 2020). In addition, as has been suggested

for schizophrenia, we may need to go beyond generic

demographic and clinical factors to predict CR outcomes reliably

(Reser et al., 2019).

The present study focused on identifying baseline predictors

that could be easily disseminated into clinical practice. Hence, it

did not include several factors identiﬁed as potential mediators

or moderators of CR in MDD or schizophrenia, such as

the number of training sessions (Buonocore et al., 2017),

motivation/engagement with training (Medalia and Richardson,

2005; Siegle et al., 2014), therapist characteristics (e.g., clinical

experience) (Medalia and Richardson, 2005), and patient–

therapist working alliance (Huddy et al., 2012), all candidates for

further investigation.

Strengths and Limitations

The study was based on data from a single-blind RCT, applied

a multimodal selection of outcome measures, and used a

stringent approach to deﬁne improvement. In addition, the study

attempted to extend on previous research by applying a long

follow-up period as an endpoint, aiming to identify predictors of

durable change following CR. However, the following limitations

should be considered when interpreting the above ﬁndings.

No a priori hypotheses were generated, and the analyses were

exploratory, calling for caution in the interpretation of results.

Another notable limitation was the modest sample size, reducing

statistical power and increasing the risk of type II errors.

Furthermore, multiple testing inﬂated the risk of type I errors,

even if partially accounted for by lowering the signiﬁcance level.

The neuropsychological measure of attention was not

corrected for practice eﬀects (Chelune et al., 1993). Although

the overall practice eﬀects on the CPT-3 are reported to be

small-to-moderate across the included subscales [T = 2.9 for

Commissions; T = 0.2 for HRT SD; Conners, 2015], correcting

for these would still provide reliable improvement in attention

that is very hard to achieve for a substantial proportion of

the sample, given the conservative RCI threshold and baseline

performance in the normal range. Moreover, the lack of an

adequate sample for comparison (i.e., non-intervention control

or comparable norm population) restricted the advantages of

applying more sophisticated statistical approaches to overcome

some of the above issues.

A selection of variables was transformed into dichotomous

categories, which may result in a loss of information and power

(MacCallum et al., 2002). Notably, this included the “number

of previous episodes” variable, found to diﬀer between groups

at trend level. The follow-up assessments lacked blinding for

most of the sample, increasing the risk of biased responding.

However, results from a small number of blind assessments were

not signiﬁcantly diﬀerent from those of non-blind assessments.

Finally, the illness-related variables (e.g., age of onset, number of

episodes) were self-reported and thus more susceptible to bias,

including memory biases.

CONCLUSION

In the present study, no major predictors of long-term

improvement in attention or executive functioning following CR

emerged. The results are consistent with previous research, which

mostly failed to identify predictors of CR treatment. Importantly,

the current ﬁndings are insuﬃcient to guide clinical decision-

making, and there is a need for large-scale and ﬁne-grained

investigations to extend current knowledge.

Frontiers in Psychology | www.frontiersin.org 8 September 2020 | Volume 11 | Article 2232

fpsyg-11-02232 September 9, 2020 Time: 22:5 # 9

Hagen et al. Predictors of Cognitive Remediation Outcome

DATA AVAILABILITY STATEMENT

The datasets for this article are not publicly available because of

restrictions speciﬁed in the study consent-form, and conditions

for approval from the local ethics committee, concerning

patient conﬁdentiality and participant privacy. Requests to

access the datasets should be directed to Jan Stubberud,

[email protected].

ETHICS STATEMENT

The studies involving human participants were reviewed

and approved by the Regional Committee for Medical and

Health Research Ethics, South-Eastern Norway (2017/666). The

patients/participants provided their written informed consent to

participate in this study.

AUTHOR CONTRIBUTIONS

BH completed the data collection and data curation,

analyzed the data, and wrote the manuscript draft. The

study was part of a doctoral thesis by BH. JS provided

supervision, conceptualized the original trial, acted as principal

investigator, and contributed with revisions of the manuscript

draft. BL and NL contributed to the conceptualization of

the original trial and revision of the manuscript draft.

EK contributed with revisions of the manuscript draft.

All authors contributed to the article and approved the

submitted version.

FUNDING

This study was funded by the South-Eastern Norway Regional

Health Authority (Grant No. 2019120) and the research fund at

the Lovisenberg Diaconal Hospital. The funding sources were not

otherwise involved in the research.

ACKNOWLEDGMENTS

We would like to thank all participants for their contributions.

Also, we thank Milada Cvancarova Småstuen (Lovisenberg

Diaconal Hospital, Oslo Metropolitan University) for

contributions to the statistical analysis and the results section

of the manuscript.

REFERENCES

Ahern, E., and Semkovska, M. (2017). Cognitive functioning in the

ﬁrst-episode of major depressive disorder: a systematic review and

meta-analysis. Neuropsychology 31, 52–72. doi: 10.1037/neu00

00319

Barkham, M., Margison, F., Leach, C., Lucock, M., Mellor-Clark, J., Evans,

C., et al. (2001). Service proﬁling and outcomes benchmarking using

the CORE-OM: toward practice-based evidence in the psychological

therapies. J. Consult. Clin. Psychol. 69, 184–196. doi: 10.1037/0022-006x.69.

2.184

Baune, B. T., Miller, R., McAfoose, J., Johnson, M., Quirk, F., and Mitchell,

D. (2010). The role of cognitive impairment in general functioning in

major depression. Psychiatry Res. 176, 183–189. doi: 10.1016/j.psychres.2008.

12.001

Beck, A. T., Steer, R. A., and Carbin, M. G. (1988). Psychometric properties of the

Beck Depression Inventory: twenty-ﬁve years of evaluation. Clin. Psychol. Rev.

8, 77–100. doi: 10.1016/0272-7358(88)90050-5

Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., and Erbaugh, J. (1961). An

Inventory for Measuring Depression. Arch. Gen. Psychiatry 4, 561–571. doi:

10.1001/archpsyc.1961.01710120031004

Buonocore, M., Bosia, M., Bechi, M., Spangaro, M., Cavedoni, S., Cocchi, F.,

et al. (2017). Is longer treatment better? A comparison study of 3 versus 6

months cognitive remediation in schizophrenia. Neuropsychology 31, 467–473.

doi: 10.1037/neu0000347

Chelune, G. J., Naugle, R. I., Lüders, H., Sedlak, J., and Awad, I. A. (1993).

Individual change after epilepsy surgery: practice eﬀects and base-rate

information. Neuropsychology 7, 41–52. doi: 10.1037/0894-4105.7.1.41

Conners, C. K. (2015). Conners’ Continuous Performance Test 3rd

editionTM(Conners’ CPT 3TM), Conners’ Continuous Auditory Test of Attention

(Conners’ CATATM) Manual. Canada: Multi-Health Systems Incorporated.

Davis, R. N., and Nolen-Hoeksema, S. (2000). Cognitive inﬂexibility

among ruminators and nonruminators. Cogn. Ther. Res. 24,

699–711.

Delis, D. C., Kaplan, E., and Kramer, J. H. (2001). Delis-Kaplan Executive Function

System (D-KEFS). San Antonio, TX: Psychological Corporation.

Delis, D. C., Kramer, J. H., Kaplan, E., and Ober, B. A. (2000). CVLT-

II: California Verbal Learning Test: Adult Version. San Antonio, TX:

Psychological Corporation.

Duﬀ, K. (2012). Evidence-based indicators of neuropsychological

change in the individual patient: relevant concepts and methods.

Arch. Clin. Neuropsychol. 27, 248–261. doi: 10.1093/arclin/

acr120

Elgamal, S., Denburg, S., Marriott, M., and MacQueen, G. (2010).

Clinical factors that predict cognitive function in patients with major

depression. Canad. J. Psychiatry 55, 653–661. doi: 10.1177/0706743710055

01004

Groves, S. J., Douglas, K. M., and Porter, R. J. (2018). A systematic

review of cognitive predictors of treatment outcome in major

depression. Front. in Psychiatry 9:382. doi: 10.3389/fpsyt.2018.

00382

Hagen, B. I., Lau, B., Joormann, J., Småstuen, M. C., Landrø, N. I., and Stubberud,

J. (2020). Goal Management Training as a cognitive remediation intervention

in depression: a randomized controlled trial. J. Aﬀect. Disord. 275, 267–277.

doi: 10.1016/j.jad.2020.07.015

Hamilton, K. E., and Dobson, K. S. (2002). Cognitive therapy of depression:

pretreatment patient predictors of outcome. Clin. Psychol. Rev. 22, 875–893.

doi: 10.1016/S0272-7358(02)00106-X

Hasselbalch, B. J., Knorr, U., and Kessing, L. V. (2011). Cognitive

impairment in the remitted state of unipolar depressive disorder: a

systematic review. J. Aﬀect. Disord. 134, 20–31. doi: 10.1016/j.jad.2010.

11.011

Huddy, V., Reeder, C., Kontis, D., Wykes, T., and Stahl, D. (2012). The eﬀect

of working alliance on adherence and outcome in cognitive remediation

therapy. J. Nerv. Ment. Dis. 200, 614–619. doi: 10.1097/nmd.0b013e31

825bfc31

Jacobson, N. S., and Truax, P. (1991). Clinical signiﬁcance: a statistical approach to

deﬁning meaningful change in psychotherapy research. J. Consult. Clin. Psychol.

59, 12–19. doi: 10.1037/0022-006x.59.1.12

Kabat-Zinn, J. (1990). Full Catastrophe Living: The Program of the Stress Reduction

Clinic at the University of Massachusetts Medical Center. New York, NY:

Delacorte Press.

Frontiers in Psychology | www.frontiersin.org 9 September 2020 | Volume 11 | Article 2232

fpsyg-11-02232 September 9, 2020 Time: 22:5 # 10

Hagen et al. Predictors of Cognitive Remediation Outcome

Kaser, M., Zaman, R., and Sahakian, B. J. (2017). Cognition as a treatment

target in depression. Psychol. Med. 47, 987–989. doi: 10.1017/s0033291716

003123

Koster, E. H. W., Hoorelbeke, K., Onraedt, T., Owens, M., and Derakshan, N.

(2017). Cognitive control interventions for depression: a systematic review of

ﬁndings from training studies. Clin. Psychol. Rev. 53, 79–92. doi: 10.1016/j.cpr.

2017.02.002

Kurtz, M. M., Seltzer, J. C., Fujimoto, M., Shagan, D. S., and Wexler,

B. E. (2009). Predictors of change in life skills in schizophrenia after

cognitive remediation. Schizophr. Res. 107, 267–274. doi: 10.1016/j.schres.2008.

10.014

Levine, B., Schweizer, T. A., O’Connor, C., Turner, G., Gillingham, S., Stuss, D. T.,

et al. (2011). Rehabilitation of executive functioning in patients with frontal

lobe brain damage with goal management training. Front. Hum. Neurosci. 5:9.

doi: 10.3389/fnhum.2011.00009

Lewandowski, K. E., Sperry, S. H., Cohen, B. M., Norris, L. A., Fitzmaurice,

G. M., Ongur, D., et al. (2017). Treatment to enhance cognition in

bipolar disorder (TREC-BD): eﬃcacy of a randomized controlled trial

of cognitive remediation versus active control. J. Clin. Psychiatry 78,

e1242–e1249.

Lindenmayer, J. P., Fregenti, S., Kang, G., Ozog, V., Ljuri, I., Khan, A.,

et al. (2017). The relationship of cognitive improvement after cognitive

remediation with social functioning in patients with schizophrenia and severe

cognitive deﬁcits. Schizophr. Res. 185, 154–160. doi: 10.1016/j.schres.2017.

01.007

Listunova, L., Bartolovic, M., Kienzle, J., Jaehn, A., Grützner, T. M., Wolf, R. C.,

et al. (2020). Predictors of cognitive remediation therapy improvement in

(partially) remitted unipolar depression. J. Aﬀect. Disord. 264, 40–49. doi:

10.1016/j.jad.2019.12.006

MacCallum, R. C., Zhang, S., Preacher, K. J., and Rucker, D. D. (2002). On the

practice of dichotomization of quantitative variables. Psychol. Methods 7, 19–40.

doi: 10.1037/1082-989X.7.1.19

Marazziti, D., Consoli, G., Picchetti, M., Carlini, M., and Faravelli, L. (2010).

Cognitive impairment in major depression. Eur. J. Pharmacol. 626, 83–86.

Medalia, A., and Richardson, R. (2005). What predicts a good response to cognitive

remediation interventions? Schizophr. Bull. 31, 942–953. doi: 10.1093/schbul/

sbi045

Morimoto, S. S., Wexler, B. E., Liu, J., Hu, W., Seirup, J., and Alexopoulos,

G. S. (2014). Neuroplasticity-based computerized cognitive remediation for

treatment-resistant geriatric depression. Nat. Commun. 5, 1–7. doi: 10.1038/

ncomms5579

Motter, J. N., Pimontel, M. A., Rindskopf, D., Devanand, D. P.,

Doraiswamy, P. M., and Sneed, J. R. (2016). Computerized cognitive

training and functional recovery in major depressive disorder: a

meta-analysis. J. Aﬀect. Disord. 189, 184–191. doi: 10.1016/j.jad.2015.

09.022

Petersen, J. Z., Porter, R. J., and Miskowiak, K. W. (2019). Clinical characteristics

associated with the discrepancy between subjective and objective cognitive

impairment in depression. J. Aﬀect. Disord. 246, 763–774. doi: 10.1016/j.jad.

2018.12.105

Philippot, P., and Brutoux, F. (2008). Induced rumination dampens executive

processes in dysphoric young adults. J. Behav. Ther. Exp. Psychiatry 39, 219–227.

doi: 10.1016/j.jbtep.2007.07.001

Porter, R. J., Bourke, C., Carter, J. D., Douglas, K. M., McIntosh, V. V. W.,

Jordan, J., et al. (2016). No change in neuropsychological dysfunction or

emotional processing during treatment of major depression with cognitive–

behaviour therapy or schema therapy. Psychol. Med. 46, 393–404. doi: 10.1017/

s0033291715001907

Pu, S., Noda, T., Setoyama, S., and Nakagome, K. (2018). Empirical evidence

for discrete neurocognitive subgroups in patients with non-psychotic major

depressive disorder: clinical implications. Psychol. Med. 48, 2717–2729. doi:

10.1017/s003329171800034x

Quinn, M. E., Keil, D. C., Utke, S., and Joormann, J. (2014). Trait rumination

moderates the eﬀect of executive control training. J. Exp. Psychopathol. 5,

289–301. doi: 10.5127/jep.038713

Reser, M. P., Slikboer, R., and Rossell, S. L. (2019). A systematic review of factors

that inﬂuence the eﬃcacy of cognitive remediation therapy in schizophrenia.

Austral. N. Z. J. Psychiatry 53, 624–641. doi: 10.1177/00048674198

53348

Rock, P. L., Roiser, J. P., Riedel, W. J., and Blackwell, A. D. (2014). Cognitive

impairment in depression: a systematic review and meta-analysis. Psychol. Med.

44, 2029–2040. doi: 10.1017/s0033291713002535

Roth, R. M., Isquith, P. K., and Gioia, G. (2005). Behavioral Rating Inventory

of Executive Function–Adult Version. Lutz, FL: Psychological Assessment

Resources.

Schulz, K. F., Altman, D. G., and Moher, D. (2010). CONSORT 2010 statement:

updated guidelines for reporting parallel group randomised trials. BMC Med.

8:18. doi: 10.1186/1741-7015-8-18

Seccomandi, B., Tsapekos, D., Newbery, K., Wykes, T., and Cella, M. (2020).

A systematic review of moderators of cognitive remediation response for

people with schizophrenia. Schizophr. Res. 19:100160. doi: 10.1016/j.scog.2019.

100160

Shilyansky, C., Williams, L. M., Gyurak, A., Harris, A., Usherwood, T., and Etkin, A.

(2016). Eﬀect of antidepressant treatment on cognitive impairments associated

with depression: a randomised longitudinal study. Lancet Psychiatry 3, 425–435.

doi: 10.1016/s2215-0366(16)00012-2

Siegle, G. J., Ghinassi, F., and Thase, M. E. (2007). Neurobehavioral therapies in

the 21st century: summary of an emerging ﬁeld and an extended example of

cognitive control training for depression. Cogn. Ther. Res. 31, 235–262. doi:

10.1007/s10608-006-9118-6

Siegle, G. J., Price, R. B., Jones, N. P., Ghinassi, F., Painter, T., and Thase,

M. E. (2014). You gotta work at it: pupillary indices of task focus are

prognostic for response to a neurocognitive intervention for rumination

in depression. Clin. Psychol. Sci. 2, 455–471. doi: 10.1177/2167702614

536160

Snyder, H. R. (2012). Major depressive disorder is associated with broad

impairments on neuropsychological measures of executive function: a

meta-analysis and review. Psychol. Bull. 139, 81–132. doi: 10.1037/a00

28727

Stubberud, J., Langenbahn, D., Levine, B., Stanghelle, J., and Schanke, A.-K. (2013).

Goal management training of executive functions in patients with spina biﬁda:

a randomized controlled trial. J. Int. Neuropsychol. Soc. 19, 672–685. doi:

10.1017/s1355617713000209

Tornås, S., Løvstad, M., Solbakk, A.-K., Evans, J., Endestad, T., Hol,

P. K., et al. (2016). Rehabilitation of executive functions in patients

with chronic acquired brain injury with goal management training,

external cuing, and emotional regulation: a randomized controlled

trial. J. Int. Neuropsychol. Soc. 22, 436–452. doi: 10.1017/s1355617715

001344

Treynor, W., Gonzalez, R., and Nolen-Hoeksema, S. (2003). Rumination

reconsidered: a psychometric analysis. Cogn. Ther. Res. 27, 247–259.

Twamley, E. W., Burton, C. Z., and Vella, L. (2011). Compensatory cognitive

training for psychosis: who beneﬁts? Who stays in treatment? Schizophr. Bull.

37(Suppl._2), 55–62. doi: 10.1093/schbul/sbr059

Velligan, D. I., Kern, R. S., and Gold, J. M. (2006). Cognitive rehabilitation for

schizophrenia and the putative role of motivation and expectancies. Schizophr.

Bull. 32, 474–485. doi: 10.1093/schbul/sbj071

Venezia, R. G., Gorlyn, M., Burke, A. K., Oquendo, M. A., Mann, J. J., and Keilp,

J. G. (2018). The impact of cognitive reserve on neurocognitive performance

in major depressive disorder. Psychiatry Res. 270, 211–218. doi: 10.1016/j.

psychres.2018.09.031

Vicent-Gil, M., Keymer-Gausset, A., Serra-Blasco, M., Carceller-Sindreu,

M., de Diego-Adeliño, J., Trujols, J., et al. (2018). Cognitive predictors

of illness course at 12 months after ﬁrst-episode of depression. Eur.

Neuropsychopharmacol. 28, 529–537. doi: 10.1016/j.euroneuro.2018.

02.001

Victor, M., Lau, B., and Ruud, T. (2016). Patient characteristics in a return to work

programme for common mental disorders: a cross-sectional study. BMC Public

Health 16:745. doi: 10.1186/s12889-016-3431-0

Vita, A., Deste, G., De Peri, L., Barlati, S., Poli, R., Cesana, B. M.,

et al. (2013). Predictors of cognitive and functional improvement

and normalization after cognitive remediation in patients with

schizophrenia. Schizophr. Res. 150, 51–57. doi: 10.1016/j.schres.2013.

08.011

Frontiers in Psychology | www.frontiersin.org 10 September 2020 | Volume 11 | Article 2232

fpsyg-11-02232 September 9, 2020 Time: 22:5 # 11

Hagen et al. Predictors of Cognitive Remediation Outcome

Wechsler, D. (1999). Manual for the Wechsler abbreviated intelligence scale (WASI).

San Antonio, TX: The Psychological Corporation.

Wechsler, D. (2014). Wechsler adult intelligence scale–Fourth Edition (WAIS–IV).

San Antonio, TX: Psychological Corporation.

World Health Organization (2004). ICD-10: International Statistical Classiﬁcation

of Diseases and Related Health Problems: Tenth Revision, 2nd Edn. Geneva:

World Health Organization.

Wykes, T., Huddy, V., Cellard, C., McGurk, S. R., and Czobor, P. (2011). A meta-

analysis of cognitive remediation for schizophrenia: methodology and eﬀect

sizes. Am. J. Psychiatry 168, 472–485. doi: 10.1176/appi.ajp.2010.10060855

Conﬂict of Interest: The authors declare that the research was conducted in the

absence of any commercial or ﬁnancial relationships that could be construed as a

potential conﬂict of interest.

article distributed under the terms of the Creative Commons Attribution License

(CC BY). The use, distribution or reproduction in other forums is permitted, provided

the original author(s) and the copyright owner(s) are credited and that the original

publication in this journal is cited, in accordance with accepted academic practice. No

use, distribution or reproduction is permitted which does not comply with these terms.

Frontiers in Psychology | www.frontiersin.org 11 September 2020 | Volume 11 | Article 2232