Running head: EVIDENCE-BASED PSYCHOSOCIAL PRACTICES
Bruce,
T.J. & Sanderson, W.C. (2004).
Evidence-Based psychosocial practices: The past, present, and future. In C. Stout & R. Hayes (Eds). Evidence-Based Practice for Mental Health
Professionals. New York: Wiley.
Evidence-based Psychosocial
Practices: The Past, Present, and Future
Timothy J. Bruce, Ph.D.
University of Illinois College
of Medicine, Peoria
William C. Sanderson, Ph.D.
Hofstra University
Evidence-based Psychosocial
Practices: The Past, Present, and Future
The
last decade has witnessed efforts to bring findings from clinical research on
psychosocial interventions into everyday clinical practice. These efforts have sought to identify
treatments for specific diagnoses that have shown efficacy through empirical
study. The identified treatments
have sequentially been termed empirically validated, empirically supported, and
evidence-based practices (EBPs), henceforth referred to as EBPs. The interventions described in this
volume are a few of the products of this effort. Several groups including government agencies, professional
experts, and health care organizations have designated certain treatments as
EBPs, but their means and rationale for doing so are not often clear.
In this chapter, we review how EBPs have been
identified by the primary professional groups involved, including the criteria
used and key scientific concepts underpinning them. We examine how these criteria compare to criteria for best
practice treatment guidelines. And
we explore some of the prominent controversies and criticisms of the EBP
movement. Future directions of the
movement known and speculative conclude the chapter. Despite the controversy it has created, the movement to
identify and disseminate EBPs will continue, providers, consumers, and payers
of mental health services will influence its course. Understanding the processes through which EBPs are
identified, their strengths and weaknesses, and the types of questions being
raised by the process informs decisions about their appropriate role in quality
mental health service delivery.
The initial, and perhaps the most influential, effort
to identify EBPs was undertaken by The Society of Clinical Psychology (Division
12) of the American Psychological Association (APA). In 1993, this division, under the charge of then-president
David H. Barlow, Ph.D., organized the Task Force on Promotion and Dissemination
of Psychological Procedures to review comprehensively the empirical literature
in search of psychosocial interventions that had proven efficacy. Using sets of criteria reflecting
different levels of empirical support, this group has published three reports
of its efforts (available at http://pantheon.yale.edu/~tat22/empirically_supported_treatments.htm). The first of these reports, (Task
Force, 1995), described the criteria used to select EBPs and a working list of
25 treatments, primarily for adult populations, that met those criteria. As of the last of the other two reports
(Chambless et al., 1996, 1998), 71 EBPs had been identified. In addition, the task force published a
series in the Division 12 journal ÒThe Clinical Psychologist.Ó In it, experts describe the EBPs, the
models underpinning them, the empirical work supporting them, and information
on protocols and training (Sanderson & Woody, 1995; Woody & Sanderson,
1998). Division 12 has also
disseminated this information to state psychological associations and
psychology training programs. The
task force was transformed into a standing committee of the Division in 1999 (i.e., the Committee on Science and Practice) and continues
its efforts today.
Other professional efforts to identify EBPs have since
been undertaken. For example,
another task force of Division 12, ÒThe Task Force on Effective Psychosocial
Interventions: A Lifespan Perspective,Ó identified EBPs and prevention programs
for children. Results can be found
in Spirito (1999) and in a special section of the Journal of Child Clinical
Psychology (Empirically supported, 1998).
The Division also commissioned a book, ÒA Guide to Treatments that
Work,Ó now in its second edition (Nathan & Gorman, 1998; 2002). In these edited volumes, various
content experts review and identify psychosocial and pharmacological treatments
meeting criteria independent of but similar to those used by the original
Division 12 task group. In a
special section of the Journal of Consulting and Clinical Psychology, Kendall
and Chambless (1998) report results of EBP reviews for adults, children,
marital, and family therapies. And
EBPs for problems common to the elderly have been offered by Gatz and
colleagues (1998), as have those for treatment of chronic pain conditions
(Wilson & Gil, 1996). Outside
the United States, EBP reviews have take place in England (Roth & Fonagy,
1996) and Canada (Hunsley et al., 1999).
Chambless and Ollendick (2001) review the results of each of these
efforts, the criteria used to make their determinations, and address putative
concerns raised by the efforts.
Any assessment of the strengths and weaknesses of the
EBP identification process is informed by an understanding of the scientific
processes that promising treatments usually undergo before being designate as
evidence-based. Over two decades
ago, Agras and Berkowitz (1980), updating Agras, Kazdin, and Wilson (1979),
offered a model of applied research that clarifies the processes used to
develop and evaluate promising treatments (Figure 1).
Figure 1
about here
As shown in this model, most eventual EBPs originate
from a clinical innovation, a new theoretical advance, or basic research
findings that have clinical implications.
After a promising intervention is developed sufficiently for use, often involving
initial case studies, it is typically tested in a small group comparison to a
no-treatment control condition.
This kind of study addresses the question of whether the treatment works
better than simply leaving the condition to run its natural course. It is possible that a problem seemingly
responding to treatment is in fact remitting on its own. A comparison to a no-treatment control
rules out this alternative explanation.
At this early stage of treatment development, other studies may be conducted
to identify what aspects of the treatment are actually contributing to its
outcome (e.g., a dismantling study).
Adjunctive procedures or parametric changes may be made to improve
outcome (e.g., investigating which schedule for therapy homework exercises
works best). Later, the critical
step of conducting comparative studies with a nonspecific or attention-control
condition are conducted to determine whether aspects unique to the new
treatment significantly contribute to therapeutic outcome over and above the
contributions made by variables common to all psychotherapies such as
attention, psychoeducation, and the like.
Comparative studies with established treatments may be conducted at
these stages. Initially, these
comparative studies are usual restricted to investigating the effects of active
therapy (i.e., pre-treatment to post-treatment comparisons). Studies of the treatmentÕs long-term
effects typically follow. Lastly,
Agras and Berkowitz indicated that studies should be conducted to determine if
the beneficial results observed in previous studies generalize to Òreal-worldÓ
applications.
The controlled comparative studies designed to test
whether a treatment works as a result of its unique or specific ingredients
have been termed efficacy studies.
Studies designed to test whether an already efficacious treatment works
in real world applications have been termed effectiveness studies (Hoagwood et
al., 1995; Moras, 1998). In
addition to differences in their purpose, efficacy and effectiveness studies
often differ in experimental design and in the types of conclusions that can be
drawn from their results.
Designs used to establish a treatmentÕs efficacy place
a strong emphasis on maximizing internal validity. Internal validity refers to the degree to which a design
rules out alternative explanations of results. Features of designs that increase internal validity include
random assignment, nonspecific or attention controls, large samples, blind and
independent assessment of outcome on psychometrically-sound measures, and
measures of whether the experimental and comparison treatments were delivered
distinctively and with treatment integrity and fidelity (generally referring to
the idea that they were delivered as intended). Efficacy studies often use specific inclusion and exclusion
criteria for enrollment of participants, in part so that the target of the
treatmentÕs efforts is consistently represented across individuals and the
problem(s) for which a benefit was derived can be specified. Independent replication of results is
desirable to rule out an allegiance effect, the possibility that the benefit
observed is specific to one or a few settings or providers such as experts or
advocates of the treatment.
The ideal single efficacy study would compare the
experimental treatment to a nonspecific control treatment that duplicates all
but the hypothesized active ingredients of the experimental intervention (a
psychological placebo).
Participants would be randomly assigned to groups to distribute any other
factors that might influence outcome.
If the ÒplaceboÓ group fails to perform as well as the experimental
treatment group, confidence is increased in the conclusion that the active
ingredients of the experimental treatment were responsible for the outcome. The closest realization of this ideal
is the randomized controlled trail (RCT).
A large-scale clinical replication series using controlled
single-subject designs is also capable of producing this type of evidence (see
Barlow, Hayes, & Nelson, 1984).
Within the Agras and Berkowitz model, efficacy studies fall under the
general rubric of outcome studies.
It is important to note that a treatment that has not been subjected to
efficacy evaluation may be producing its beneficial effects through factors
other than those unique to it, or only on specific or biased measures of
outcome, or be due to other uncontrolled factors.
Effectiveness studies, on the other hand, ask the
question of whether the efficacy a treatment has previously demonstrated will
transfer to real world applications.
The designs of effectiveness studies usually differ from those used to
evaluate efficacy in that they place high value on maximizing external or
ecological validity. External
validity refers to the extent to which experimental results can be generalized
to the larger population from which a client sample was drawn, including
different settings, providers, more complicated clinical presentations, and the
like. To a large degree, design
features of effectiveness studies attempt to mimic real world
applications. For example, random
assignment, a cardinal feature of efficacy studies, may be omitted from an
effectiveness study because patients in the real world are not randomly assigned
to treatment options (see Seligman, 1995). These studies may have clients participate in only one
treatment, as opposed to some efficacy studies that have them cross over into
comparison treatments. Inclusion
and exclusion criteria of effectiveness studies may be broader than some
efficacy studies so they can capture the complexities seen in real world
settings. An uncommon exception to
this is when the effectiveness study is testing an efficacious treatments
designed for a specific problem or type of patient.
It is important to note that one cannot draw the
conclusion from the typical effectiveness study that the outcome observed is
due to the unique effects of the treatment, because they typically lack the
controls needed to rule-out alternative explanations of the results. One can determine from them, however,
whether a treatmentÕs benefits witnessed in previous efficacy studies did or
did not generalize to the particular application tested. In interpreting the results of
uncontrolled effectiveness studies, it is assumed that the observed benefits
are due to the treatment in question even when that treatment has previously
demonstrated efficacy. Common
threats to external validity can be found in Kazdin (2003). Considerations for balancing the
internal and external validity of applied research designs can be found in
Miklowitz and Clarkin (1999).
Understanding the types of studies involved in
treatment development and the types of questions they ask and can answer
informs oneÕs evaluation of how EBPs are identified. Similarly, knowing the types of criteria recommended for
best practice treatment guidelines informs oneÕs evaluation of how well EBP
criteria conform to these standards and how well the literature supporting a
particular EBP conforms as well.
Treatment guidelines can be thought of as a specific form of best
practice recommendations. They
recommend treatment options available to patients that are likely to maximize
their chances for recovery.
Ideally, they are informed by the best
empirical evidence and by expert clinical consensus.
Despite the proliferation of treatment guidelines by
various groups, only recently has there been offered a comprehensive set of
criteria for judging how guideline authors should evaluate the treatment
options they recommend. This was
undertaken by a collaborative task force of the APA representing the Board of
Professional Affairs, the Board of Scientific Affairs, and the Committee for
the Advancement of Professional Practice within the organization. This group has published what is
recognized as a comprehensive and rigorous set of criteria for constructing or
evaluating treatment guidelines (APA, 2000). Although most relevant to psychosocial interventions, these
recommendations are applicable to healthcare interventions broadly.
The APA criteria for guidelines are organized around
two fundamental dimensions of evaluation that will ring familiar to readers:
treatment efficacy and clinical utility.
Treatment efficacy refers to essentially the same concept discussed
above, Òthe systematic and scientific evaluation of whether a treatment worksÓ
(p. 4). Clinical utility refers
to, Òthe applicability, feasibility, and usefulness of the intervention in the
local or specific setting where it is to be offered.Ó It also includes a Òdetermination of the generalizability of
an intervention whose efficacy has been establishedÓ (p. 4). Cost considerations are the final
domain of utility. Utility is a
concept similar to effectiveness as described above, although effectiveness
refers more to the type of studies often used to test utility.
The value placed on the demonstration of efficacy in
these guideline recommendations is evident in several of the criteria. For example, guideline crafters are
urged to base their recommendations on a broad and comprehensive evaluation of
the empirical literature supporting any treatment. They are urged to consider the rigor of the methodology
used, including the quality of comparison/control groups and the quality and
specificity of outcome assessments.
The authors state that without evidence of efficacy, providers are
forced to rely exclusively on their direct experience, which risks erroneous
conclusions when used as a sole source of knowledge.
An example of the benefit of scientifically
establishing efficacy is found in the history of the development of systematic
desensitization (SD), an EBP for specific phobias identified by all independent
review panels. Systematic
desensitization originally contained several procedures that were later found
to be unnecessary in reducing phobic fear and avoidance (see Barlow, 2002;
Bruce & Sanderson, 1998). For
example, the original version of SD involved first teaching participants to
relax deeply. This relaxed state
was then repeatedly paired with exposure to imagined phobic stimuli until no
significant distress was elicited upon exposure. Patients would systematically repeat this process moving
from low-fear to high-fear stimuli.
This pairing procedure was based on the theoretical notions of reciprocal
inhibition and counterconditioning.
The originator of SD, Joseph Wolpe (1958), hypothesized that phobic
stimuli had been conditioned to sympathetic nervous system arousal (e.g.,
increased heart rate, respiration, and tension). The treatment was designed to countercondition those phobic
stimuli through systematic, repeated, and exclusive pairing with a response
that would (reciprocally) inhibit sympathetic arousal, in this case the
parasympathetically-based relaxation response. Subsequent controlled studies found, however, that this
pairing was not a necessary condition for phobic fear reduction and that
exposure alone was sufficient.
Other studies showed for example that the hierarchical presentation of
stimuli, although more tolerable to more patients, was not a necessary
condition for fear reduction as well.
These studies not only helped identify key therapeutic procedures, but
advanced the profession's understanding of the mechanisms of fear reduction. Without the controlled studies leading to
the establishment of efficacy, these developments may not have taken
place. The APA guideline authors
cite other more serious consequences of excluding efficacy evaluation such as
popular treatments once believed to be effective but later shown to be ineffective
or even harmful.
In addition to an emphasis on efficacy, guideline
criteria emphasize consideration of the utility of interventions. This value is reflected in several
recommendations. For example, the
task group recommends consideration of a breadth of patient variables that
could influence outcome. Examples
include age, gender, race, and cultural background. They advise consideration of the complexities of clinical
presentations including comorbidity and severity. The consideration of feasibility is reflected in
recommendations to consider data on the effect of a practitionerÕs training,
skill and experience. The
evaluation of costs, to the patient and to the healthcare system, is
highlighted as well.
Of course, the degree to which an empirical literature
supportive of a treatment meets these criteria depends on how advanced it is
along the clinical research process.
Empirical literatures certainly vary in this regard. For example, the empirical support for
identified EBPs for depression has advanced to where efficacy has been strongly
demonstrated and initial effectiveness studies support the generalizability of
results. On the other hand, the
literature supporting identified EBPs for substance abuse and dependence have
not clearly established efficacy (see Chambless & Ollendick, 2001). Factors such as the complexity of the
target problem(s), the type of intervention under study, funding priorities,
and the like can influence the degree to which an empirical literature
advances. To address this
variability and fill the gaps left by the absence of empirical data, the APA
group encourages guideline developers to Òconsider clinical opinion,
observation, and consensus of experts representing the range of views in the
fieldÓ (p.5). Accordingly, the
relative degree to which empirical evidence or clinical opinion informs a
treatment recommendation will depend on the problem area and treatment in
question.
Table 1 about here
Division 12 categorized their criteria into different
ÒlevelsÓ of evidence based on the rigor and volume of the empirical studies
supporting a particular treatment.
The categories are termed well-established, probably efficacious, and
experimental. A close examination
of the criteria for the highest level of evidence, Òwell-established,Ó reflects
the degree to which they value efficacy and utility. The first criterion emphasizes the internal validity of
designs used to test the efficacy of the treatment in question. It requires that treatments have shown
either equivalence in outcome to an already established treatment or
superiority to a pill or psychotherapy placebo condition using a Ògood
between-groups designÓ in at least two separate studies. Alternatively, Criterion II allows acceptance
a large scale, rigorously conducted clinical replication series (i.e., multiple
single case design studies) involving a treatment comparison.
Criterion III requires the use of manuals in the
delivery of the treatment or an equivalent clear description of treatment, a
criterion that has met with praise (e.g., see Wilson, 1998) and criticism
(e.g., see Bohart, OÕHara, & Leitner, 1998). This criterion has been explained in a manner that
emphasizes the generalizability of efficacy study results. For example, Chambless and Olledick
(2001) state, ÒÉin terms of generalizable knowledge, it is meaningless to say
that a treatment works without being able to say what that treatment isÓ (p.
701). But some critics have argued
that manual use overly restricts necessary therapeutic flexibility (e.g.,
Silverman, 1996; Lambert, 1998) or may actually limit the likelihood of
generalizability because therapists in the field do not use them (Seligman,
1995). A survey of therapists on
this topic found that, indeed, many report not using manuals, but it also found
that many therapists do not know that EBP manuals exist (Addis & Krasnow,
2000). Interestingly, the few
effectiveness studies that have used manuals to guide treatment have reported
success. It remains an empirical
question as to whether manual use will facilitate generalizability for some or
all EBPs. It is unclear how many
treatments were excluded from consideration by Division 12 due to this
requirement.
It should be noted that although this criterion has
been explained in terms of generalizability, it has implications for internal
validity as well. For example,
providing a description of the principles and procedures of a treatment,
whether through use of a manual of other means, allows the assessment of treatment
integrity, fidelity, and distinctiveness of the compared treatments. In other words, it can be used to
assess whether treatments compared in efficacy studies were delivered as
intended and distinctively. From a
design perspective, this is the notion that one cannot conclude that an
independent variable, in this case the EBP, was responsible for the outcome
unless it is certain that it has been applied as intended and delivered those
elements that distinguish it from the control treatment. Some efficacy studies supportive of
EBPs report fidelity data. Others
do not.
Criteria IV and V for well-established treatments
place value on both internal and external validity. For example, Criterion IV requires that characteristics of
study participants are identified.
This helps specify the dependent variable(s), and it allows consumers of
the study to determine for what clients and problems the EBP may apply. Criterion V, which requires independent
replication of results, ensures that the results of a study used to support an
EBP were not a product of an allegiance effect. This requirement is relevant to internal validity in that it
rules out an alternative explanation of the results. It also emphasizes generalizability in that it strengthens
confidence that the benefits of the treatment are transportable to other sites,
providers, and circumstances.
These criteria show that, consistent with its charge, the Division 12
task force placed strong emphasis on the establishment of efficacy in the
criteria it used to identify EBPs.
Utility is not ignored, but nor is definitive demonstration of it
required. The other two categories
of EBPs, Òprobably efficaciousÓ and Òexperimental,Ó reflect lesser levels of
evidence. They are ÒlesserÓ in
that they do not require independently replicated studies involving comparisons
with an established treatment or psychological placebo.
As shown in Table 1, criteria that were used in other
reviews had a similar emphasis on efficacy, although not all used the same
categorical breakdown used by Division 12. Nathan and Gorman (1998), for example, required a randomized
prospective clinical trial and several specific statistical and design features
(e.g., blind assessments, adequate sample size to ensure statistical power
sufficient to reduce risks of Type I and Type II errors). Although the level of specificity
differs from that of Division 12Õs criteria, it is consistent with an emphasis
on internal validity. There are
other differences in the detail of criteria used in the reviews cited,
discussion of which is beyond the purpose of this chapter. Interested readers are referred to
Chambless and Ollendick (2001).
So, how far have the empirical literatures used to
support identified EBPs advanced through the clinical research process, and how
well do they compare to the criteria for treatment guidelines recommended by
the APA task group (APA, 2000)? At
last report, 22% of interventions identified by Division 12 had reached the
Òwell-establishedÓ level of evidence, strongly supporting their efficacy. Approximately 60% of these treatments
are for anxiety or depressive disorders.
Effectiveness studies, the furthest developmental advance in the
clinical research process and supportive of utility, have been published
primarily in these areas well, particularly in panic disorder and unipolar
depression (see Chambless & Ollendick, 2001). The majority of identified EBPs (78%) are supported at less
than well-established levels of efficacy, indicating that although data
supportive of efficacy exists, they are insufficient to rule-out with certainty
alternative explanations of results.
In several cases, it is an allegiance effect that has not been ruled
out, meaning that the studies supporting the EBP have not been independently
replicated. Although these
treatments may be as they are labeled, Òprobably efficacious,Ó the lack of
independent replication leaves questions about their transportability and
generalizability unanswered. Thus,
although it appears that psychosocial treatment development and outcome
research is well advanced for some identified EBPs, the current status of
literatures supporting most EBPs is at levels supporting efficacy, but not yet
demonstrating their generalizability, feasibility, and cost-effectiveness in
community application.
Division
12Õs interest in identifying EBPs is part of a larger worldwide movement to
make findings from applied medical research more available to clinicians in
practice, a movement known generally as evidence-based medicine (Sackett,
Richardson, Rosenberg, & Haynes, 1997). The primary aim of this effort is to improve the quality of
patient care by making it easier for practitioners to empirically inform their
services. Empirically informed
treatment guidelines have been a common means of disseminating this information. And if the APA criteria for evaluating
treatment guidelines serves as a model, then the criteria used to identify EBPs
are consistent them, particularly in regards to the demonstration of efficacy. It is also clear that the degree to
which current literatures supportive of EBPs meet these criteria varies and
depends on the problem area in question.
Even the most advanced bodies of work, those for anxiety and depressive
disorders, do not satisfy all recommended guideline criteria. Most have not reached a level of
evidence convincingly demonstrating the generalizability, feasibility, and
costs of treatment. Clearly, more
effectiveness studies are needed to address these issues, and federal funding
priorities acknowledge this need.
The next decade will witness a surge in these types of studies and
provide more direct evidence of the degree to which EBPs are capable of
improving the quality of current mental health care. As has been noted by advocates and critics of EBPs, any
mental health service provider that uses an EBP should be prepared to adapt it
to clients as needed using their best clinical judgement, a point true of
nearly all health care interventions.
As worthy as the intent of the EBP movement may seem, it, its products,
and, in particular, the use of these products has created controversy.
Selection bias. One criticism of EBPs is the claim that they are essentially products of a specific group of, primarily cognitive-behavioral, clinical researchers whose theoretical model and treatment approaches are more amenable than alternative treatments to the kinds of scientific evaluation being recommended as criteria for EBPs. It is true that cognitive-behavioral clinical researchers were represented on the Division 12 task force, that many cognitive-behavioral therapies (CBTs) have been manualized (a required criterion of some reviewers), and that CBT emphasizes measurable treatment goals. It is also true that therapies based on CBT models appear prominently on EBT lists and have been studied more than other approaches. Yet, several individuals and groups have undertaken the EBP identification task, not all were cognitive-behavioral in orientation, and they have come to similar conclusions regarding appropriate EBP criteria. Further, the APA task group for evaluating treatment guidelines, a group representing diverse professional allegiances, also generated criteria consistent with those of the independent EBP review panels--as have several other guideline authors (e.g., American Psychiatric Association). Although many CBTs for many problem areas have met EBP criteria, non-CBT therapies that have been subjected to empirical scrutiny have met them as well (e.g., brief psychodynamic therapy for depression). The ÒunfairnessÓ of excluding unstudied therapies from these lists may be more the issue here, and is discussed further in a subsequent section.
Capturing complexity. It has been argued that most studies supportive of EBPs fail to capture the complexity and uniqueness of patients seen in community practice, and by implication their applicability to the community patient is minimal. This argument has taken several forms. For example, it has been said that the criteria used to define samples in efficacy studies rely too heavily on the DSM, or that their selection criteria excludes common comorbidities, or that they have otherwise treated easier-to-treat clients than one finds in the community (e.g., Bohart, 1998; Henry, 1998; Levant, in press).
The argument regarding the DSM suggests that categorical diagnostics, such as those used in the DSM, are inadequate in capturing dimensional complexities with which clients commonly present. Although some task forces, such as Division 12Õs, used more than DSM diagnoses in evaluating studies (e.g., cut-off scores of questionnaires, specific behavioral targets of treatment), by and large the DSM has indeed guided most recent treatment outcome research. Of course, a categorical diagnosis does not preclude a treatment plan based on the categoryÕs dimensional features. It should also be noted that some DSM diagnostic categories do capture complex and difficult clinical presentations (e.g., Borderline Personality Disorders; BPD), and some identified EBPs have shown promise in treating them (e.g., Dialectic Behavior Therapy for BPD). But the essence of this argument is that the limitations of the DSM limits the applicability of efficacy studies to the community (a point to which we will return).
The study typically used to support the complexity argument is Westen and Morrison (2001), who reported that complexity, defined as syndromal comorbidity, is underrepresented in efficacy studies. These investigators reported that approximately two-thirds of potential participants in treatment studies of depression, panic disorder, and generalized anxiety disorder did not meet inclusion criteria due to comorbidity. Results of this study stand in contrast, however, to studies that have reported a high degree of similarity between samples used in efficacy studies and those drawn from the community, with a trend toward increased complexity in research samples (Lipsey & Wilson, 1993; Shadish, Matt, Navarro, & Phillips, 2000). In interpreting these mixed results, L. E. Beutler (personal communication, November 3, 2003) noted that increased emphasis is being place on improving the representativeness of research samples. The inclusion of earlier studies in the Westen and Morrison (2001) meta-analysis may explain its results.
An assumption of the complexity argument, that is not a given, is that an EBP treatment focused on one disorder is not applicable or will be ineffective for patients with comorbidity. This is an empirical question that has received some preliminary study, producing mixed results. The question under study has been whether treating the primary disorder of a comorbid profile (e.g., the most severe or initial) results in improvement in the primary and comorbid conditions. Tsao, Lewin, and Craske (1998) found that symptoms of comorbid social anxiety, generalized anxiety, and post-traumatic stress disorder decreased significantly following the EBP treatment of the primary panic disorder. Borkovec, Abel, and Newman (1995) found similar results in a group of patient with a primary diagnosis of generalized anxiety disorder. McLean, Woody, Taylor, and Koch (1998) found, however, that comorbid depressive symptoms remained relatively unchanged after a similar treatment for panic disorder. Those symptoms predicted impaired treatment response on some measures and warranted further treatment. Brown, Antony, and Barlow (1995) found decreased comorbidity following treatment of a primary panic disorder, but symptoms had returned to pretreatment level by the two-year follow-up. These mixed results suggest that there may be applications of current EBPs designed for single disorders to certain comorbidity profiles, but some comorbidities will not respond. Further studies inclusive of complex participants will help answer these questions.
Perhaps a more subtle consequence of using the DSM is the limitation of its axes-based system to guide psychosocial aspects of treatment plans, which often focus beyond the impact of symptoms on functioning. It is fair to say that the concept of Axis IV (psychosocial stress) and its relevance to treatment planning is underdeveloped in the guidance it offers for treatment planning. This does not negate testing psychosocial interventions related to Axis I or II diagnoses, but indirectly discourages conceptualizing and study client problems more inclusive than those diagnoses.
A final argument concerns the application of EBPs to complex individual clients, and is part of a larger criticism on the limitations of the RCT. This argument too has taken many forms. For example, it has been suggested that the data analyses commonly used in RCTs place inordinate emphasis on group trends (nomothetic level of analysis) over that of individual patient responses (idiographic analyses). Another version of the argument has been that RCTs favor quantitative research methods over qualitative one, suggesting the latter of which are more sensitive to individual responses (e.g., Bohart et al., 1998). Within most applied research models, qualitative studies are often seen as a source of hypotheses that may be tested in subsequent controlled studies. A strength of applied qualitative research is that it often provides data rich in individual participantsÕ responses to an intervention. A weakness of them though is that their designs often lack the internally validity sufficient to rule-out alternative explanations of their results. Without controlled internally valid designs, it is impossible to ascertain to what any participant is responding.
This emphasis on internal validity and efficacy in
most EBP criteria reflects more the value placed on systematic as opposed to
unsystematic observation. Yet,
many RCTs have favored designs and data analytic procedures that are largely
quantitative and nomothetic. For
instance, data from many RCTs are analyzed using traditional f- or t-tests of
group by time interactions. These
types of analyses are conducted using aggregated (group) data, results of which
may not predict a future individualÕs response to treatment. There is a trend toward describing some
RCT results in terms of the percentage of patients who meet varying levels of
improvement, often defined multidimensionally. But, idiographic analyses are often not done or reported in
RCTs. Recent trends in statistical
analyses are addressing these concerns and are discussed more fully under
ÒFuture Directions.Ó
Demonstrating
utility. The need for more
effectiveness studies of EBPs has been recognized since the movement to
identify EBPs began. In 1999, a
special issue of the online journal, ÒPrevention and TreatmentÓ published
several excellent articles addressing this need and considerations, theoretical
and applied. (available at http://journals.apa.org/prevention/volume2). Effectiveness studies are beginning to
appear and are likely to continue for some time. So, what have been the results of studies completed to
date? Have EBPs shown promise of
utility in the real world healthcare system? Have they worked in community agencies, with
difficult-to-treat clients, multiple demands, and limited resources? Although the body of this research is
small, the trend in existing evidence is supportive of generalization, not
uniformly though and not without some qualifications. For example, Wade, Treat, and Stuart (1998) reported
improvements comparable to benchmark efficacy studies in a large broad-based
sample of community mental health patients treated for panic disorder with or
without agoraphobia using a standard manual-based intervention (Barlow &
Craske, 1994). Drop-out rates were
lower than rates typically observed in this population, but higher than in
efficacy studies. Results
suggested improvement over standard care but short of complete generalization
across indices. This same
treatment was tested with a group of urban, low income, and primarily Latino
patients whose pre-treatment severity levels were higher than benchmark
efficacy studies (Sanderson, Raue, and Wetzler, 1998). Results showed statistically significant
improvement, but to post-treatment severity level that were also higher than
post-treatment efficacy benchmarks.
In a study of CBT for depression, improvements reaching benchmark levels
were achieved, but the number of therapy session needed to this had to be
extended over those used in efficacy studies (Persons, Bostrom, and
Bertagnolli, 1998).
These examples show the promise of EBPs in producing
good outcome, but efficacy benchmarks may not be achieved or need some
modifications may be needed to achieve them. Whether other identified EBPs will show effectiveness in
routine practice wait to be seen.
What the nature of the outcome will be, whether it will be found
throughout the range of patient complexities, what systemic obstacles may
arise, and what role is best played by the EBP in the treatment plan are
questions awaiting further study.
As noted, federal funding agencies are making such studies a priority,
and answers to these questions are forthcoming.
Treatment specificity and nonspecific factors. One consequence of the EBP identification movement has been to rekindle a long running debate about what type of factors actually produce the therapeutic outcome seen in efficacy studies and whether some therapies do this better than others. The former debate has been focused primarily on the relative contribution to therapeutic outcome of factors specific to the treatment (specific factors) versus those that are common to all psychotherapies (nonspecific factors). Critics have argued that a literature supporting the important role of nonspecific factors has been ignored by the focus on identifying specific treatments for specific disorders.
Perhaps the most concerted effort to identify and disseminate information on the role nonspecific factors has come from another APA task force, the Task Force on Empirically Supported Therapy Relationships, formed by Division 29 (Psychotherapy). This task force summarized a literature investigating factors related to the therapist, the therapeutic relationship, and non-diagnostic patient characteristics. Two reports from that work (i.e., Lambert & Barley, 2001; Norcorss, 2001) have been cited by critics of EBP lists because they concluded that factors specific to treatments account for no more than 15% of the variance in therapeutic outcomes, the same percentage accounted for by placebo and expectancy effects. The remaining 70% of variance was due to factors related to patients (15%), therapeutic relationships (40%), and changes outside the therapy (15%). These figures, however, are not consistent with other studies. It has been suggested that this may be due to the way in which they were calculated (L. E. Beutler, personal communication, November 3, 2003). Specifically, Beutler notes that they were not derived from direct calculations of effect size, the preferred method, but from estimates of what the total variance would have been if all the studies used in the analysis had employed analyses of variance statistics in which total variance can be calculated Studies of relationship variables, for example, that have used direct effect size calculations have yielded correlations ranging from .11 (Stevens, Hynan, & Allen, 2000) to .26 (Horvath & Symonds, 1991) with most falling near .24. When the variance accounted for by these factors is calculated it shows that relationship and treatment-specific factors make similar contributions to outcome approximating .10. This may be beside the point, however, because these types of studies may be artificially separating factors that may in practice interact in complex ways to influence outcome (Beutler et al., 2003). Alternative approaches to analyzing these interactions hold the promise of capturing this complex interplay and may help de-polarize the debate over what factors are the most influential (see ÒFuture DirectionsÓ below).
Even if specific and nonspecific factors interact inextricably to produce outcome, it is possible that certain treatments may still result in better outcomes than others, also known as treatment specificity. Chambless and Ollendick (2001) review evidence of treatment specificity and show that it is evident in the some literatures, particularly anxiety disorders and childhood depression. Interestingly, in the highly evolved literature on adult depression, evidence supports the conclusion that different psychotherapies (e.g., psychodynamic, interpersonal, and behavioral) can each produce desired outcome. Literatures for other problem areas vary in their capacity to address this question.
It should be noted that meta-analytic evidence showing that psychotherapy in generally effective is often cited as evidence against treatment specificity (e.g., Shapiro & Shapiro, 1982; Weisz, Weiss, Han, Granger, & Morton, 1995). This type of result, however, does not constitute evidence against the notion that some treatments are more effective than others for particular problems. The most direct evidence comes from direct comparative studies. Unfortunately, these types of studies are typically seen only in the more advanced literatures. Although not every psychotherapy can be tested in comparison to others, it seems that tests of specificity among commonly-used interventions may reveal differences in outcome that can inform treatment selection in ways that improve a client's chance of recovery. This is relevant given that several studies are finding that the majority of client who suffer disorders for which there are well-established EBPs are nevertheless not receiving them, and in many cases receiving therapies shown to be inferior (Taylor, King, & Margraf, 1989; Goisman, Warshaw, & Keller, 1999; Plante, Andersen, & Boccaccini, 1999; Adiss & Krasnow, 2000; Sanderson, Hiatt, & Schwartz, 2001).
Forced use. Perhaps the most disconcerting consequence of the EBP movement is one that has not to do with the identification process per se, but with the manner in which EBPs are being seen and used by payers of services. For example, some authors (e.g., Seligman and Levant, 1998; Silvermen, 1996) point out that managed care and insurance companies are beginning to restrict healthcare coverage to specific EBPs, not acknowledging the absence of data demonstrating their utility. Similar restrictions are emerging through some state, and federal funding agencies. Kovacs (1995) warns that this precedent may make practitioners who do not used identified EBPs vulnerable to malpractice lawsuits. A consequence of restricted coverage is that potentially valuable services developed in the community by licensed clinicians are rendered unavailable to clients because they have not been subjected to extensive empirical testing. Levant (in press) has made the point that restricting payment to EBPs over untested alternatives is tantamount to equating the latter with therapies that have demonstrated a lack of efficacy. These points argue against the forced and exclusive use of EBP, whether through funding incentives, legal deterrents, or other means.
Another objection to forced use is more philosophical. Some therapeutic models differ from others at the level of the world views supporting them, world views that may derive from contradictory epistemic values. Most experienced mental health professionals practicing today remember being taught psychotherapy in its relation to personality theory, and being taught personality theory in its relation to the philosophical schools of thought from which they derive. A scientific worldview and therapies consistent with it, many of which are identified EBPs, were taught as one of many options, others of which may not be bedded in the logical positivism and empiricism of science. It was common that students were instructed to choose a model, in part, based on how consistent it was with oneÕs personal worldview. It has been suggested that science is but one of several ÒfaithsÓ in this sense, and that treatments deriving from it should not be forced on practitioners (cf., Levant, in press).
Counterpoints to these arguments have essentially been practical in nature. They acknowledge that seasoned clinicians may indeed be providing effective services, but that without some demonstration of this, payers have to trust that they are not buying an ineffective approach. In the current atmosphere of increased accountability in healthcare service provision, payers are not likely to offer this trust a priori. To the more philosophical argument, the counterpoints have been similarly practical. Today, the practice of psychotherapy is not treated as a matter of choice in a faith, but as a healthcare treatment option subject to cost and benefit/risk analyses. In this atmosphere, for better or worse, the practitionerÕs option to choose what they offer based on philosophic grounds has become an intellectual luxury not affordable to those seeking third-party reimbursement for their services.
Unfortunately, there are few alternatives to resolving the problem of how to recognize potentially effective yet untested treatments apart from some form of scientific study. It is unlikely that reimbursement policies will return the days when licensure was sufficient to justify oneÕs choice of intervention. Practitioners now need to demonstration outcome. And outcome measurement, indeed, may be a means to addressing several of the concerns raised by the forced use issue. Demonstration of outcome may not only serve to justify reimbursement of services, but holds promise of demonstrating the effectiveness of current clinical practices and of EBPs. Although some therapeutic models may indeed be more difficult to operationalize, their desired outcome is potentially definable and measurable. To date, however, efforts to organize outcome measurement systems for these purposes are not concerted. The task is clearly challenging. Capturing major domains of client progress, such as symptom, function, or quality of life in a reliable, valid, relevant, and least burdensome manner is difficult and resource intensive. Despite its challenges, though, the use of outcome measurement systems to assess the effects of field-delivered interventions may be one of the few ways to preserving choice of interventions while satisfying the demand for accountability to payers. Of course, in terms of advancing the knowledge base of the field, this use of outcome measurement is akin an uncontrolled case study and leaves open the question of what is causing the improvement if witnessed (see below for further discussion).
Future Directions
Clearly, the movement to identify EBPs has potential risks and benefits. Decades of clinical research have led to the development of efficacious psychosocial treatments that have benefited thousands of clients and hold the promise of doing the same for future consumers. Improvement rates for many of these psychosocial treatments rival or exceed those of the primary alternative treatment, state-of-the-art medications (Nathan & Gorman, 2002). Initial studies of how these approaches work in community settings have been promising. Many of the objections to the dissemination of EBPs revolve around the risk of potential misuse by payers and by a legal system that may not appreciate the complexities of addressing community mental health needs and the appropriate roles of EBPs in that effort. The challenge of defining those roles and educating these systems in the appropriate use of EBPs use will be ongoing. It is likely to play out in several arenas including the public mental health system, court systems, funding agency policies, treatment guidelines, EBP toolkits, and the relevant professional literatures.
As noted, several studies have shown that the majority of people seeking mental health treatment are receiving therapeutic approaches of unknown efficacy or of efficacy known to be inferior to EBPs (Taylor et al., 1989; Goisman et al., 1999; Plante et al., 1999; Adiss & Krasnow, 2000; Sanderson et al., 2001). Use of EBPs for those clients is likely to improve care. Of course, EBPs are not universally effective with the disorders for which they were developed. Even experienced EBP providers know that some client problems are of a complexity or severity to be unamenable to an unmodified EBP protocol. The nature of these limits to the generalizability of EBPs will be better known as they are subjected to effectiveness studies over the next decade. Advocates and critics agree that use of EBPs still requires use of oneÕs clinical expertise and sensitivity to the clientÕs needs and wishes.
This chapter highlighted the risk that exclusive use of EBPs may prevent potentially effective, field-developed, or otherwise innovative treatments from being recognized. Science and practice are likely to move toward integration if a means can be found to bring the under-recognized voice of seasoned practitioners into the effort to define and shape best practices. Although there are several means to this end, one option highlighted in this chapter is the growing effort to collect, evaluate, and disseminate outcome data collected on community-based interventions.
Work has begun in this regard and its continued development may be fruitful. An example includes Practice-Research Networks, in which standardized outcomes are being collected across selected participating practices (e.g., Borkovec, Echemendia, Ragusea, & Ruiz, 2001). Fishman (1999) has also described a model system for organizing and disseminating case studies of adequate design. The call for the use of standardized measurement systems has been made in clinical research arenas to allow for better comparison of results across empirical studies. It seems equally applicable for demonstration projects of field-developed or EBP therapies. Although efforts like research networks are likely to be used to field test EBPs, field testing current community practices (e.g., between EBPs and treatment as usual) may improve understanding of the relative contributions of each approach. In addition, public payers, such as state divisions of mental health, as well as private insurers are in a position to contribute to this effort if they organize these types of projects and seek funding opportunities. Funding and outlets for the results of efforts are increasing as priorities are shifting toward effectiveness research. It waits to be seen if such efforts will be made.
The EBP movement has rekindled discussion and debate regarding which types of questions that should be asked in treatment outcomes studies. The debate regarding the relative contributions of the content (specific factors) and the process (nonspecific factors) to therapeutic outcome was highlighted and is likely to continue. Efforts to re-explore the integration of these seemingly separate aspects of psychosocial interventions are being witnessed in current research and may have far reaching consequences for the EBP movement and the field in general. As introduced previously, the conventional design of most RCTs has framed treatment as an independent variable exerting its effects on dependent variables (e.g., the patient/participant). Factors other than those specific to the treatment (the nonspecific factors) are controlled, usually by randomly distributing or duplicating them across all groups under study. Traditionally, outcome has been analyzed in part by isolating the magnitude of change on the dependent variable that is correlated exclusively with the application of the specific factors of the therapy.
This model is enriched when patient and therapist variables traditionally considered nonspecific (e.g., relationship quality, degree of fit between patient and treatment) are studied as potential moderating variables. These kinds of moderated relationships can be explored within traditional RCT designs by employing regression statistics, for example, which allow unique correlational variance to be ascertained. A recent example of this approach was published by Beutler et al. (2003). These investigators analyzed results of a RCT conventionally, evaluating group differences across time on the dependent variable. They also analyzed these same data using a regression analytic strategy that evaluated four sets of variables: 1) patient factors, 2) treatment procedures, 3) relationship quality, and 4) the degree of patient-procedure fit. These four classes of variables each added independent variance to the prediction of outcomes, accounting for from 53% to over 90% of the variance, depending on the outcome variable. Increased use of these types of analyses holds the promise of improving our understanding of the complexity of the mechanisms of therapeutic change, depolarizing the debate between nonspecific and specific factors, and changing the manner in which EBPs are conceptualized.
An organized example of
this move toward integration is the current effort by a large task force of the
APA that is exploring empirically supported ÒprinciplesÓ of therapeutic change,
as opposed to lists of specific treatments (Beutler & Costonguay, in
press). Four problem areas have
been the initial focus of this group: dysphoric disorders, anxiety disorders,
personality disorders, and habit disorders (broadly defined to include addictions). The principles under study are
reflective of traditional content and process emphases: those related to
treatment procedures (specific factors), treatment participants (e.g., the
patient and therapist), and the therapeutic relationship. This work represents a shift toward
thinking of specific and nonspecific factors as intertwined and operating in
complex relationships, perhaps differently in different individuals, to produce
outcome.
Should efforts such as
outcome measurement use and research on specific and nonspecific factors
produce results, they are still likely to encounter some challenges in
demonstrating accountability to payers.
For example, some funding agencies require the demonstration of fidelity
of the services they fund (assessments of whether the treatment is being
conducted as intended) in addition to demonstrations of outcome. Although adhering to the fidelity of
some EBPs seems important to achieving their outcome (e.g., assertive community
training), others allow more flexibility as long as the principle of the
treatment is honored (e.g., exposure-based therapies). The ÒprinciplesÓ approach described
above is inherently broader and more flexible in application than a specific
EBP protocol as might be some complex field-developed interventions the
effectiveness of which may be demonstrated through outcome assessment. Fidelity requirements would need to
flex as well.
As much as the EBP identification process has stirred controversy and debate, efforts to acknowledge, depolarize, and integrate science and practice perspectives are being seen as a result of it. Another arena in which this integration is possible is in the development of treatment guidelines. Although a representative APA task force has defined criteria for good treatment guidelines, their have been do direct attempts to contribute a set of working treatment guidelines based on them. Whether such an effort will be witnessed in the near future is unknown, but it could advance current conceptualizations of psychosocial treatment beyond the specific treatment approach. Empirically and clinically informed guidelines such as these might be as educational to the systems that might misuse EBPs as they would be practical to interested providers.
Translating evidenced-based treatment recommendations into practice requires acceptance and training of practitioners as well as acceptance by patients. In regards to EBPs, a Division 12 survey conducted of directors of APA-approved clinical training programs revealed that approximately 20% of programs do not teach them at a minimum level of coverage as defined by the survey authors (Crits-Christoph et al., 1995). Data showing that most patients do not receive EBP treatment suggests, albeit indirectly, that these treatments are a tool not many therapists possess or are unwilling to use. Weissman and Sanderson (2001) offer an example of a model for training EBPs applicable across disciplines.
Regarding patient acceptance, recent years have witnessed the adoption of the Òrecovery modelÓ by advocates of persons with severe mentally disorders. This model advances the strong value of placing responsibility and control over the recovery process in the hands of the sufferer of the condition. Accordingly, this places a need on providers to educate consumers about the risks and benefits of various service options available to them. It would seem that EBPs would stand in good stead in this regard, in that their risks and benefits have an empirical base. But, it is not uncommon in recovery circles to view science with suspicion and reject certain services regardless of the evidence supporting their contributions to recovery. An example of one way to close this gap between scientific and recovery model adherents has been offered by Frese, Stanley, Kress, and Vogel-Scibilia (2001). Other efforts are likely to be witnessed as well, and will be important to the dissemination effort.
Conclusion
In this chapter, we discussed the current status and
future directions of the EBP identification movement. We reviewed means used to identify EBPs and key scientific
concepts underlying that process.
We examined how EBP criteria compare to standards for best practice
treatment guidelines and found that those criteria overlap in many ways,
particularly in regard to efficacy.
We reviewed the degree to which literatures supporting particular EBPs
have advanced through the clinical research process and found that although
most demonstrate efficacy or its promise, few have convincingly demonstrated
treatment utility. Effectiveness
studies are needed to evaluate utility and are a priority of research funding
agencies.
Controversies surrounding EBPs have had many
consequences. One perhaps
unexpected consequence has been the degree to which this debate has informed
research on the role of specific and nonspecific factors in therapy. Innovative and integrative work is
being done in this area.
Controversy has also revealed problems with the potential misuse of EBPs
by purchasers and the legal system.
Most critics argue that evidence supportive of EBPs may show efficacy
but is limited regarding utility and as such is insufficient to warrant their
exclusive use, exclusive funding, and legal penalties for nonuse. Unfortunately, the practical and
financial motives of payers are not likely to be swayed simply by a risk that
efficacious EBPs may not generalize.
Options for addressing misuse of EBPs such as applications of outcome
assessment and the development treatment guidelines reflecting empirical and
clinical input were presented. In
the end, demonstration of the strengths and limitations of EBPs will determine
the future of the movement, and results of those efforts are what the next
decade is likely to witness.
Addis M. E., & Krasnow A. D.
(2000). A national survey of practicing psychologistsÕ attitudes toward
psychotherapy treatment manuals. Journal of Consulting and Clinical
Psychology, 68, 331–39.
Agras, W. S., & Berkowitz, R. (1980). Clinical
research in behavior therapy: Halfway there? Behavior Therapy, 11,
472-487.
Agras, W. S., Kazdin, A. E. & Wilson, G. T.
(1979). Behavior therapy: Toward an applied clinical science. San
Francisco: W. H. Freeman.
American Psychological Association. (2000). Criteria
for evaluating treatment guidelines. Washington, DC: American Psychological
Association.
Barlow D. H. (2002). Anxiety and its disorders: The
nature and treatment of anxiety and panic. New York: The Guilford Press.
Barlow, D. H. & Craske, M.
G. (1994). Mastery of Your Anxiety and Panic (Vol. II). Albany, NY:
Graywind.
Barlow, D. H., Hayes, S. C., & Nelson, R. O.
(1984). The scientist practitioner: Research and accountability in clinical
and educational settings. New York: Pergamon Press.
Beutler, L. E. & Castonguay, L. G. (Eds.) (In
press). Empirically Supported Principles of
Therapy Change. New York: Oxford University Press.
Beutler, L. E., Malik, M., Alimohamed, S., Harwood, T.
M., Talebi, H., Noble, S., & Wong, E. (2003). Therapist variables. In M. J.
Lambert (Ed.), Handbook of psychotherapy and behavior change (5th ed.)(pp.
227-306). New York: John Wiley and Sons.
Bohart, A. D., OÕHara, M., &
Leitner, L. M. (1998). Empirically violated treatments: disenfranchisement of
humanistic and other psychotherapies. Psychotherapy Research, 8,
141–57.
Borkovec, T. D., Abel, J. L., & Newman, H. (1995).
Effects of psychotherapy on comorbid conditions in generalized anxiety
disorder. Journal of Consulting & Clinical Psychology, 63 (3),
479-483.
Borkovec, T. D.,
Echemendia, R. J., Ragusea, S. A., and Ruiz, M. (2001). The
Pennsylvania Practice Research Network and
possibilities for clinically meaningful and scientifically rigorous
psychotherapy effectiveness research. Clinical Psychology: Science and
Practice, 8, 155-167.
Brown, T. A., Antony, M. M., & Barlow, D. H.
(1995). Diagnostic comorbidity in panic disorder: effect on treatment outcome
and course of comorbid diagnoses following treatment. Journal of Consulting
& Clinical Psychology, 63 (3), 408-418.
Bruce, T.J., & Sanderson,
W.C. (1998). Specific phobias: Clinical applications of evidence-based
psychotherapies. New Jersey: Jason Aronson.
on psychological interventions. The Clinical
Psychologist, 49(2), 5-15.
Chambless, D.L, Baker,
M.J., Baucom, D., Beutler, L.E., Calhoun, K.S., Crits-Christoph, P., Daiuto,
A., DeRubeis, R., Detweiler, J., Haaga, D.A.F., Johnson, S.B., McCurry, S.,
Mueser, K.T., Pope, K.S., Sanderson, W.C., Shoham, V., Stickle, T., Williams,
D.A., Woody, S.R. (1998). Update on Empirically Validated Therapies: II. The
Clinical Psychologist, 51(1), 3-16.
Chambless, D. L., & Ollendick, T. H. (2001).
Empirically supported psychological interventions: Controversies and evidence. Annual
Review of Psychology, 52, 685-716.
Chambless, D.L., Sanderson, W.C., Shoham, V., Johnson,
S.B., Pope, K., Crits-Christoph, P., Baker, M., Johnson, B., Woody, S.R., Sue,
S., Beutler, L., Williams, D., & McCurry, S. (1996). Empirically validated
therapies: A project of the division of clinical psychology, american
psychological association, task force Crits-Christoph,
P., Frank, E., Chambless, D. L., Brody, C., Karp, J. F. (1995). Training in
empirically-validated treatments: What are clinical psychology students
learning? Professional Psychology: Research and Practice, 26,
514–522
Empirically supported
psychosocial interventions for children. [Special issue]. (1998). Journal of
Clinical Child Psychology, 27, 138–226.
Fishman, D. B. (1999). The case for pragmatic psychology.
New York: New York University Press.
Frese,
F. J., Stanley, J., Kress, K., & Vogel-Scibilia, S. (2001). Integrating evidence-based practices
and the recovery model. Psychiatric Services, 52 (11), 1462-1468.
Gatz, M., Fiske, A., Fox, L. S.,
Kaskie, B., Kasl-Godley, J. E., et al. (1998). Empirically validated
psychological treatments for older adults. Journal of Mental Health and
Aging, 41, 9-46.
Goisman, R.M., Warshaw, M.G., & Keller, M.B. (1999). Psychosocial treatment
prescriptions for generalized anxiety disorder, panic disorder, and social
phobia, 1991-1996. American Journal of Psychiatry, 156, 1819-1821.
Henry, W. P. (1998). Science,
politics, and the politics of science: the use and misuse of empirically
validated treatments. Psychotherapy Research, 8, 126–40.
Hoagwood, K., Hibbs, E., Brent,
D., & Jensen, P. (1995). Introduction to the special section: efficacy and
effectiveness in studies of child and adolescent psychotherapy. Journal of
Consulting and Clinical Psychology, 63, 683–687
Horvarth, A. O., & Symonds,
D. B. (1991). Relationship between working alliance and outcome in
psychotherapy: A meta-analysis. Journal of Counseling Psychology, 38,
139–149.
Hunsley, J., Dobson, K. S.,
Johnston, C., & Mikail, S. F. (1999). Empirically supported treatments in
psychology: Implications for Canadian professional psychology. Canadian
Psychologist, 40, 289–302.
Kazdin, A. E. (2003). Research
design in clinical psychology (4th ed.). Needham Heights, MA: Allyn &
Bacon.
Kendall, P. C., & Chambless,
D. L. (Eds.). (1998). Empirically supported psychological therapies [Special
issue]. Journal of Consulting and Clinical Psychology, 66 (3)
–167.
Kovacs, A. (1995). We
have met the enemy and he is us! The Independent Practitioner, 15(3),
135-137.
Lambert, M. J. (1998).
Manual-based treatment and clinical practice: Hangman of life or
promising development? Clinical Psychology: Science
and Practice, 5, 391–395.
Lambert, M. J., &
Barley, D. E. (2001). Research summary on the therapeutic
relationship and psychotherapy outcome. Psychotherapy:
Theory/Research/Practice/Training, 38, 357-361.
Levant, R. F. (in press). The empirically-validated
treatments movement: A practitioner/educator perspective. Clinical
Psychology: Science and Practice.
Lipsey, M. W., & Wilson, D. B. (1993). The
efficacy of psychological, educational, and behavioral treatment: Confirmation
from meta-analyses. American Psychologist, 48, 1181-1209.
McLean, P. D., Woody, S., Taylor, S., & Koch W. J.
Comorbid panic disorder and major depression: implications for
cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology,
66 (2), 240-247.
Miklowitz, D. J., & Clarkin,
J. F. (1999). Balancing internal and external validity. Prevention and
Treatment, 2, Article 0020004c. Available on the World Wide Web: http://jurnals.apa.org/prevention/volume2/pre002004c.html
Moras, K. (1998). Internal and
external validity of intervention studies. In A. S. Bellack, & M. Hersen
(Eds.), Comprehensive Clinical Psychology (pp. 201–24), Oxford,
UK: Elsevier.
Nathan, P. E., & Gorman, J.
M. (Eds.). (1998). A Guide to Treatments That Work. New York: Oxford
University Press.
Nathan, P. E., & Gorman, J.
M. (Eds.). (2002). A Guide to Treatments That Work (Vol. II). New York:
Oxford University Press.
Norcross, J. C. (2001).
Purposes, processes, and products of the task force on
empirically supported therapy relationships. Psychotherapy:
Theory/Research/ Practice/Training, 38, 345-356.
Persons, J. B., Bostrom, A.,
& Bertagnolli, A. (1999). Results of randomized controlled trials of
cognitive therapy for depression generalize to private practice. Cognitive
Therapy and Research, 23, 535– 548
Plante, T.G., Andersen, E.N., Boccaccini, M.T.
(1999). Empirically supported
treatments and related contemporary changes in psychotherapy practice: What do
clinical ABPPs think. The
Clinical Psychologist, 52, 23-31.
Roth, A. D, & Fonagy, P.
(1996). What works for whom? A critical review of psychotherapy research.
New York: Guilford.
Sackett, D. L., Richardson, W.
S., Rosenberg, W., & Haynes, R. B. (1997). Evidence-Based Medicine. New
York: Churchill Livingstone.
Sanderson, W.C., Hiatt, D., & Schwartz, J. (2001).
Psychological treatment of panic disorder by managed care providers. Manuscript
submitted for publication.
Sanderson, W. C., Raue, P. J.,
& Wetzler, S. (1998). The generalizability of cognitive behavior therapy
for panic disorder. Journal of Cognitive Psychotherapy, 12,
323–330.
Sanderson, W. C., & Woody,
S. R. (1995). Manuals for empirically validated treatments: A project of the
task force on psychological interventions. Clinical Psychologist, 48
(4), 7–11.
Seligman, M. E. P. (1995). The
effectiveness of psychotherapy: the consumer reports study. American
Psychologist, 50, 965–974.
Seligman, M. E. P.,
& Levant, R. (1998). Managed care policies rely on inadequate
science. Professional Psychology: Research
and Practice, 29, 211-212.
Shadish, W. R., Matt, G. E., Navarro, A. M., &
Phillips, G. (2000). The effects of psychological therapies under clinically
representative conditions: A meta-analysis. Psychological Bulletin, 126,
512-529.
Shapiro, D. A, & Shapiro, D.
(1982). Meta-analysis of comparative therapy outcome studies: a replication and
refinement. Psychological Bulletin, 92, 581-604.
Silverman, W. H. (1996).
Cookbooks, manuals, and paint-by-numbers: Psychotherapy in the 90Õs. Psychotherapy,
33, 207–215.
Spirito, A. (Ed.). (1999).
Empirically supported treatments in pediatric psychology (Special issue). Journal
of Pediatric Psychology, 24, 87-174.
Stevens, S.E., Hynan, M.T., & Allen, M. (2000). A
meta-analysis of common factor and specific treatment effects across the
outcome domains of the phase model of psychotherapy. Clinical Psychology: Science & Practice, 7
(3), 273-290.
Task Force on Promotion
and Dissemination of Psychological Procedures. (1995). Training in and
dissemination of empirically-validated psychological treatments. The
Clinical Psychologist, 48, 3-23.
Taylor, C.B., King, R., Margraf, J. (1989). Use of medication and in vivo exposure in volunteers for
panic disorder research. American Journal of Psychiatry, 146, 1423-1426.
Tsao, J. C., Lewin, M. R., & Craske, M.G. (1998).
The effects of cognitive-behavior therapy for panic disorder on comorbid
conditions. Journal of Anxiety Disorders, 12 (4), 357-371.
Wade, W. A., Treat, T. A,
Stuart, G.L. (1998). Transporting an empirically supported treatment for panic
disorder to a service clinic setting: a benchmarking strategy. Journal of
Consulting and Clinical Psychology. 66, 231-239.
Weissman, M. .M., &
Sanderson, W. C. (2002). Problems and promises in modern
psychotherapy: The need for increased training
in evidence based
treatments. In B. Hamburg (Ed.), Modern Psychiatry: Challenges in
Educating Health Professionals to Meet New
Needs, New York City: Josiah
Macy
Foundation.
Weisz, J. R.,Weiss, B., Han, S.
S., Granger, D. A., & Morton, T. (1995). Effects of psychotherapy with
children and adolescents revisited: A meta-analysis of treatment outcome
studies. Psychological Bulletin, 117, 450-468.
Westen, D. and Morrison, K. ( 2001). A multidimensional meta-analysis of
treatments for depression, panic, and generalized anxiety disorder: An
empirical examination of the status of empirically supported therapies. Journal
of Consulting and Clinical Psychology, 60, 875-899.
Wilson, G. T. (1998).
Manual-based treatment and clinical practice. Clinical Psychology: Science
and Practice, 5, 363-375.
Wilson, J. J., & Gil, K. M.
(1996). The efficacy of psychological and pharmacological interventions for the
treatment of chronic disease-related and non-disease-related pain. Clinical
Psychology Review, 16, 573-597.
Wolpe, J. (1958). Psychotherapy by reciprocal
inhibition. Stanford, CA: Stanford University Press.
Woody, S. R., & Sanderson,
W. C. (1998). Manuals for empirically supported treatments: 1998 update. Clinical
Psychologist, 51 (1), 17-21.
Table 1
Workgroup Criteria for the Identification of Empirically-Supported Therapies.
(Adapted and reprinted with permission from Chambless, D. L., & Ollendick, T. H. (2001).
Empirically supported psychological interventions: Controversies and evidence.
Annual Reviews of Psychology,
52, 685-716.)
________________________________________________________________________
Division
12 Task Force criteria (Chambless et al 1998)
I. At least two good between-group design experiments must demonstrate efficacy one or more of the following ways:
A. Superiority to pill or psychotherapy placebo, or to other treatment.
B. Equivalence to already established treatment with adequate sample sizes.
OR
II. A large series of single-case design experiments must demonstrate efficacy with
A. Use of good experimental design and
B. Comparison of intervention to another treatment