on Psychotherapy Outcome
What is the history and status of psychotherapy outcome research?
Nearly five decades of psychotherapy outcome research has clarified the factors that account for successful outcome in psychotherapy. These factors are those those which are the basis of person-centered therapy; that is, the client/therapist relationship and the internal and external resources of the client.
The basis of most therapies are related to the question: What might be the most effective treatment for a particular client dysfunction? This perceptual belief dominates the party line of the profession of psychotherapy to the extent that it ignores the results of five decades of research.
A special issue of the American Psychologist on Outcome Assessment of Psychotherapy in 1997 illustrates the rigidity of the perceptual belief in the fundamental assumption of treatment by method for types of dysfunction. In ten quasi-scholarly articles and five "comments" revolving around Seligman's (1995) assertions concerning the usefulness of The Consumer Reports survey on mental health in a previous issue of the American Psychologist. Only one author referred to the importance of the relationship in therapeutic "encounters" (and this only a tangential reference). All articles abound with the terminology of "interventions" for treatment. The clear focus is upon the therapist EXPERTISE and METHOD OF TREATMENT for the PARTICULAR DYSFUNCTION) The factors which have most consistently been related to positive outcome over decades of research; i. e.; the client-therapist relationship and the self resources of the client, are virtually ignored. Furthermore, research reviews on the relationship in counseling are likewise embedded in this "specificity myth". Sextons and Wiston"s (1994) reasonably thorough coverage of research on the relationship in counseling is indicative of the bias even though they seek and encourage a different paradigmatic view predicated upon a "social constructivist" view. These authors point out that " . . . it is only the counseling relationship that has consistently been found to contribute to the success of the therapeutic process (Luborsky et al, 1988; Orlinsky & Howard, 1986)" (p. 7). Sexton & Wiston (1992) summarize:
The research has confirmed what was widely recognized: The success of any therapeutic endeavor depends on the participants establishing an open, trusting, collaborative relationship or alliance (Frank & Gunderson, 1990). In addition, research has shown that failure to form such an alliance is strongly associated with client noncompliance with treatment plans (Eisenthal, Emery, Lazare, & Udin, 1979); premature termination (Saltzman, Luetgert, Roth, Creaser, & Howard, 1976; Tracy, 1977); as well as poor outcome (Alexander & Luborsky, 1986). (p. 7)
What is the direction of the research concerning the relationship?
The review of the literature on research concerning the relationship in psychotherapy undergoes a subtle shift of focus. This is reflected by the above statement that " . . . failure to form such an alliance is strongly associated with client noncompliance with treatment plans . . . " The relationship is converted to, "alliance" and defined by "client noncompliance" when not formed. In part, this shift is due to the authors use of Gelsos and Carter's (1985) multidimensional model of the relationship. This model is founded upon psychoanalytic premises resulting in identification of the relationship as real, unreal and as a working alliance. The unreal aspect of the model is based upon the concept of transference. The working alliance is an extrapolation to all therapies, focusing on agreement of goals and tasks by and the emotional bond of the client and therapist, but still associated with the psychoanalytic approach. As such, the entire model is presented from the stance of "therapist expertise and method to dysfunction treatment" ideas even when the relationship is the focus of discussion. This basic assumption seems apparent by the authors' periodic references to such terms as "client compliance", "interventions", and measuring the alliance as the "client's collaboration" in therapy. Even when considering the relationship, the fundamental assumption of our treatment system is embedded in the paradigm of the specificity myth.
What does the research on psychotherapy demonstrate?
The following overview of several major research reviews on psychotherapy outcome demonstrates both the fallacy of "specificity myth" and provides support for the validity of Rogers basic ideas.
Method of Inquiry:
I examined a number of reviews of psychotherapy outcome research studies as well as re-examined my own inquiry into effective psychotherapy over the past three decades. The most prominent conclusions from research reviews are the following:
1. Effective psychotherapy is primarily predicated upon (1) the relationship between the therapist and the client and (2) the inner and external resources of the client.
2. The type of therapy and technique is largely irrelevant in terms of successful outcome.
3. Training, credentials and experience of therapists are irrelevant to successful therapy.
4. Clients who receive psychotherapy improve more than clients who do not receive psychotherapy.
5. There is little evidence to support the position that there are specific treatments for particular disabilities.
6. The most consistent of the relationship variables related to effectiveness are the conditions of empathy, genuineness and unconditional positive regard.
The Integrative Statement of Carl R. Rogers:
person who has, in my view, come closest to identifying the critical elements
of psychotherapeutic effectiveness is Carl Rogers (1957). His statement
concerning the necessary and sufficient conditions of therapeutic personality
change is an integrative statement for psychotherapy and helping relationships
that is separate from his statement (Rogers, 1959) concerning the conditions
as part of client-centered therapy (Stubbs & Bozarth, 1996; see earlier
chapter on The Integrative Statement). As I have noted several times earlier,
this is an important point in that Rogers' efforts were mostly directed
toward this core of "necessary and sufficient" attitudes for helping relationships
rather than directed towards the development of "Client-Centered Therapy"
as many have assumed. From this perspective, it is speculated in a previous
chapter that therapists must achieve their own congruence including the
use of their own "technique system" in order to maximize their capacity
for experiencing empathic understanding and unconditional positive regard
for the client. In short, I assert that techniques and theoretical formulations
are for the therapist rather than for particular clients and this, essentially,
allows therapists freedom to enter the person to person relationship with
their clients in the best possible way. Although Client-Centered Therapy
most likely maximizes the probability of such a relationship if the therapist
holds the principles of the approach, the conclusions of the psychotherapy
research suggest, as Rogers proposed, that the conditions can be embedded
in other forms of therapy and helping situations. Focus on the relationship
and the client's inner and outer resources can occur to some extent in
all therapeutic endeavors. It is, however, Rogers theory of therapy that
focuses upon thes success factors as the sine qua non of effective therapy.
are some of the particular reviews identifying data that supports the conclusion
of the specificity myth?
The Data Base:
The major reviews that substantiate my conclusions are briefly summarized as follows:
Strupp, H. H., Fox, R. E. and Lessler, K. (1969). Patients view their psychotherapy. Baltimore: The John Hopkins Press.
This study was an early survey of samples from a psychiatric outpatient clinic wherein patients gave their accounts of their treatments. The "success" patients in this study were those who had high internal motivation, initiative, and viewed discomfort and determination as crucial to their success. The composite "patient's" view of the "good" therapist was " . . . that of a keenly attentive, interested, benign, and concerned listener-a friend who is warm and natural, is not averse to giving direct advice, who speaks one's language, makes sense and rarely arouses intense anger" (p.117).
In addition, they found that inexperienced therapists did as well as highly experienced therapists from the clients' view: "There were no appreciable differences in outcome or quality of the therapeutic relationship, length of therapy or frequency of sessions had no measurable bearing on outcome, and differences in therapeutic competence, as judged by supervisors, were also inconclusive" (p. 119).
The authors found that their conclusions were "strikingly similar to the conclusions . . . " which Berenson and Carkhuff (1967) had drawn from their review of counseling and psychotherapy research.
Nearly three decades later, the research reviews of psychotherapy outcome research reflect the major notions observed by Berenson and Carkhuff's (1967) review and the Strupp et. al. (1969) study of the importance of the relationship; namely, the importance of the clients' involvement in their own treatment and the minuscule influence of "interventive" techniques.
Stubbs, J. P. & Bozarth, J. D. (1994). The dodo bird revisited: A qualitative study of psychotherapy efficacy research. Applied & Preventive Psychology 3: 109-120.
At the Third International Forum of the Person-Centered Approach in 1987, I (Bozarth, 1993) first reported that the prominent conclusion of the time concerning Rogers' hypothesis of the necessary and sufficient conditions for therapeutic personality change was that the conditions were necessary but not sufficient. One was led by the literature to think that this position was well substantiated by research studies. However, my review did not find a single study which supported this position. Dr. Stubbs examined the literature several years later in an unpublished paper and also found the results to be confusing. This led us to the qualitative study of "The dodo bird revisited . . . ". This qualitative study of psychotherapy research effectiveness reports that Rogers' hypothesis is the most stable major thread running through the effectiveness of psychotherapy throughout, at least, the last four decades. Of five emergent temporal categories of focus, the abiding relationships to outcome that emerged in some form are those that Rogers (1957) hypothesized in his classic integrative statement as core ingredients for therapeutic personality change (i.e. congruence, unconditional positive regard and empathic understanding). Stubbs and Bozarth also specifically note that the predominant temporal category that is the forerunner to the specificity question is the category that the conditions are necessary but not sufficient has virtually no research support. We did not find one direct study which supported this assertion. Conclusions were, at best, extrapolations of flawed logic. That is: The logic that support for Rogers' hypothesis is weak; hence, something more must be needed, and that thing is some form of interventive technique.
The major implications of this study in relation to effective psychotherapy are that (1) the major thread running through the nearly five decades of efficacy research is the relationship of the therapist and client and that a strong part of that data refers to Rogers attitudinal conditions of the therapist; and (2) the precursor of the "specificity" assumption is the unsupported theme of Rogers' conditions being necessary but NOT sufficient; and (3) the research foundation for the "specificity question" has abysmal research support..
Duncan, B. L., & Moynihan, D. W. (1994). Intentional utilization of the clients frame of reference. Psychotherapy, 31, 294-301.
Concomitant to the publication of the "Dodo Bird" qualitative study of psychotherapy research, Duncan and Moynihan summarized reviews of quantitative research studies (e. g., Lambert, Shapiro & Bergin, 1986; Lambert, 1992) to propose the application of the outcome research to practice. Duncan and Moynihan's (1994) argument is actually from a very person-centered perspective. They propose a model predicated on recent conclusions concerning the research on psychotherapy outcome. They point out that the reviews of outcome research (Lambert, 1992; Lambert, Shapiro & Bergin, 1986) suggest that 30% of the outcome variance is accounted for by the common factor of the client-counselor relationship across therapies, techniques account for 15% of the variance as does placebo effect and 40% of the variance is accounted for by extratherapeutic change variables (factors unique to the client and his/her environment). Such research findings suggest to them the utility of intentionally utilizing the client's frame of reference.This article is important to consider here because it summarizes the outcome research in a way that focuses on the critical variables in successful outcome. As well, it lays out the basis for a therapy model that is predicated upon scientific method hypothesis testing research. Although their model is very aligned to the model of Client-Centered Therapy in its purist form, the emphasis upon the influence of the extratherapeutic variables of the client as the greatest contributor to outcome may suggest that more integral consideration of therapist action and reaction within the empathic context may enhance effectiveness.
Consumer Reports. (1995, November). Mental health: Does therapy help? pp. 734-739.
This survey questionnaire to readers of the Consumer Reports concerning the effectiveness of psychotherapy again buttresses the findings from other reports; e. g., refuting the assertion " . . . of the usefulness of specific techniques for specific disorders . . . ". As a survey, the study is subject to the usual methodological critiques of the scientific method paradigm; such as, lack of random assignment, lack of systematic treatment and other factors threatening internal validity. It is noteworthy, however, that Seligman (1995), who is a major champion of the "efficacy" study (scientific method, hypothesis paradigm studies), changed his view of the way to study psychotherapy effectiveness. He comments that within this framework, the studies of specific treatment for specific dysfunction (which he terms, "efficacy" studies) do not consider the realities of therapy. He elaborates that the efficacy designs seldom go beyond the internal validity of the studies. His conclusion is well stated: "The efficacy study is the wrong method for empirically validating psychotherapy as it is actually done, because it omits too many crucial elements of what is done in the field." In other words, the nature of the methodological designs and their inherent constraints (e.g., control studies, exclusive and manualized treatments, random assignment, limited treatment times, single diagnoses), ignore many crucial elements of the actual practice of psychotherapy in the field.
The Consumer Reports' survey and Seligman's assessment is important in that the conclusions of this report are consistent with conclusions of reviews of studies with more "rigorous" research designs (as, for example, summarized by the Duncan & Moynihan report and the Stubbs & Bozarth study) and the viability of such surveys are acknowledged even though outside of the "efficacy study" paradigm.
Bohart, A. C. and Tallman, K. (1996, Summer). The active client: Therapy as self-help. Journal of Humanistic Psychology, 36,3, 7-30.
This review of psychotherapy outcome studies concludes that it is the active client who is ultimately the therapist. What makes psychotherapy work? is answered in this review as, "the active client." The authors' interpretations of the research buttress the conclusion that the factors clients see as helpful are not usually technique factors but rather more general processes like providing support. It is further concluded that therapists must not only use the client's frame of reference and rely more on the client but " . . . we must truly understand that it is the whole person of the client who generates the processes and solutions that create change'" (p. 26). The therapist provides a safe working place for client dialogue, experiencing and exploration; provides a set of procedures that can be used by clients to create new self experiences as a way to develop new perspectives and solutions; and provide therapist interactive experience and feedback.
Silverman, W. (1996). (1996, Summer). Cookbooks, manuals, and paint-by-numbers: Psychotherapy in the 90's. Journal of Humanistic Psychology, 33,2, 207-346.
This article is a reaction to recommendations of the Division 12 Task Force Report on Promotion and Dissemination of Psychological Procedures (1993, October). The author points out that the Task Force had, at that time, cited eighteen specific treatments interventions from a total of thirty three studies worthy of "empirically-validated interventions" and actually ignored the conclusions that emerge from several decades of thorough reviews of psychotherapy outcome literature. Silverman's analogy to the Task Forces procedures is worth quoting since their thrust is an acceleration of the fiction of the specificity hypothesis. Silverman states:
Let us try a group hypnotic induction. Please imagine that you are back in graduate school taking a seminar in Psychotherapy Research. The professor asks you to write a paper about effective psychotherapies. You will share your scholarship with the rest of the students in the class so that they can also become knowledgeable on the subject. After delivering your paper to the professor, you tell her that while you did not do a comprehensive review of the literature, you asked several of your other professors what they believed to be effective therapies and then you documented these impressions. Furthermore, in defining effectiveness you would only consider those studies that manualized the treatment process. Notice the look on her face as you explain this to her.
Your paper cites approximately thirty papers as evidence of effectiveness out of the thousands of papers that have been published in the last twenty-five years. From these thirty papers, you list eighteen treatments that are valid, fifteen of which are forms of behavioral modification or cognitive behavioral therapy. As you hand in your paper to the professor, examine her hand for signs of tension. You also inform the professor that regardless of the grade you may receive, you will be making specific recommendations to the Director of Training about what sorts of therapy Clinical Supervisors are to accept. You will also ask the Academic Dean to make specific changes in the graduate curriculum to accommodate your findings. What autonomic reactions do you notice displayed by your professor from the generalizations you make? Now I want you to imagine what grade you will receive. Do you find this hypnotic induction too unrealistic? Well, it time to wake up and face reality. (p. 207)
Silverman's analogy is not only a clear statement about the thrust of the particular Task Force but represents the essence of the undermining of effective mental health care by the myth of specific treatment for particular dysfunction hypothesis.
Bozarth, J. D. (1997). Psychotherapy outcome research designs. Preliminary Report.
In 1996, I decided to do another perusal of the research designs of psychotherapy outcome research. It is clear that the profession of psychology holds the "gold standard" (the "efficacy" or true design study) of psychotherapy research as the sine qua non for the determination of psychotherapeutic efficacy (Division 12 Task Force Report on Promotion and Dissemination of Psychological Procedures, 1996; Dawes, 1996; Seligman, 1996) even though there seems little evidence that such research even exists. Such inquiry is further complicated in that when (if) it exists that such " . . . studies are unable to demonstrate either clear-cut efficacy or adequate descriptions of the effective therapist or effective techniques" (Kisch, 1980).
The perusal of the "efficacy" studies covered a span of two decades from 1970 through 1989. Efficacy in this review, however, is defined as "the power to produce an effect" (Webster's Ninth New Collegiate Dictionary, 1991), with "effect" being "the power to bring about a result." Seligman (1996) uses the term "efficacy" to identify scientific method hypothesis testing studies (i.e., the true experimental study). It is unclear to me that the terms "efficacy" and "effectiveness" have been so differentiated to any significant degree in previous writings. The studies searched, thus, had a more general meaning than that of being only those studies identified as "true experimental designs".
My intent was to identify the general types of studies of psychotherapy, first, as to being quantitative, qualitative or other reports. Second, the quantitative studies were identified by type of design; i. e., pre-experimental, true experimental, quasi-experimental, correlational-causal and Ex Post Facto designs (Leedy, 1993). The "gold standard" is the true experimental design which has adequate random sampling and a control group. By Seligman's definition, all of the studies would be true experimental designs but the more general definition used for this review allowed a few more studies to be examined.
The initial computer searches (Galin, Eric & PsycINFO) resulted in nearly 1000 articles found. However, this was rapidly decreased to sixty-four articles due to non-related titles and content (883), non-research papers (21), not having an efficacy issue (23), and exceptionally poor methodology (6). There remained 26 quantitative studies, 2 qualitative studies, and 33 general reviews of the research studies. Three additional articles were meta-analysis. It must be noted that this search did not include some of the studies of "effectiveness" reviewed in previous articles (e. g., Duncan & Moynihan, 1994; Stubbs & Bozarth, 1994) but, as noted above, they were also not limited to studies of "efficacy" as defined by Seligman.
Of the quantitative studies (It is in this ball park that we would find the "gold standard" studies), 10 were pre-experimental designs, 5 were quasi-experimental, 4 were correlational-causal and only six were true experimental designs. The fact that there were only six studies that met the criterion of the "gold standard" is, of course, revealing in and of itself. When we examine the studies in slightly greater detail, it is even more revealing. The samples represent varying populations that include stutterers, the elderly, Russians, depressed women, and individuals in crisis. The treatments are also variable ranging from unidentified general psychotherapy in two of the studies, "supportive psychotherapy" in one of the studies, focused intervention, assertiveness training and Rational-Emotive Therapy compared to Systematic Desensitization. In short, these true experimental design studies have little common ground regarding the question of efficacy and there is not a hint of replication. Although some studies not recorded in the particular computer searches were no doubt missed in this search, this inquiry is consistent with other reviews. There are few true experimental designs and as summarized previously by Stubbs and Bozarth (1994): "The research concerning specificity of treatment, dysfunction, therapist variables, and client variables is characterized by fragmentation, few replications and lack of generalizability" (p.116).
What are the primary conclusion of psychotherapy outcome research?
The Data Base Conclusion
The conclusion is clear: There is not a research foundation for the underlying assumption of specific treatments for specific dysfunctions. The specificity myth is replete. I repeat Stubbs and my previous comment that the direction of the research continues to prove "' . . . 'significantly insignificant to help' and often obscures what is most significantly helpful" (Stubbs & Bozarth, 1994, p.117). The most clear research evidence is that effective psychotherapy results from the resources of the client (extratherapeutic variables) and from the person to person relationship of the therapist and client. The specificity and systematizing of these variables remain somewhat murky although they do include Rogers' hypothesized variables of the attitudinal qualities. The research on relationship reviewed by Sexton and Whiston supports the conclusion " . . . that there are significant individual differences among and within clients over time and that these individual differences account for the majority of the variance in counseling outcome (Martin, 1990)" (p. 58). The data increasingly points to "the active client" and the individuality of the client as the core of successful therapy.
As O'Hara (1995) aptly concludes:
It isn't the technique, it isn't the therapist, it isn't the level of training, it isn't the new wonder drug, it isn't the diagnosis. It is our clients' own inborn capacities for self-healing, and it is the meeting--the relationship in which two or more sovereign and sacred "I's" meet as a "we" to engage with significant questions of existence" (p. 19, 30-31).
efficient practice model should be based on the "true" findings of the
last five decades of research on psychotherapy effectiveness. At the core,
Rogers seminal contributions are a foundation for effective treatment.
This paper was originally presented as Playing the probabilities of psychotherapy at The Association for the Development of the Person-Centered Approach, Las Vegas, NV. (1997, May).
Adapted with permission from:
Bozarth, J.D. (1998). Playing the probabilities of psychotherapy. Person-Centred Practice 6, (1}, pp. 33-37.
in Bozarth, J. D. (1999). Person-Centered Therapy: A Revolutionary Paradigm.
Ross-On-Wye, England: PCCS BOOKS.