R.I.P. Prescriptive Clinical Prediction Rules

A lot of very good ideas have had a positive influence on clinical practice. Simple concepts such as washing one’s hands, restricting unnecessary care from those who don’t need it, and crowdsourcing in research have helped revolutionize clinical care and healthcare research. Some simple ideas, while well intended, fail to lead to a positive change in patient outcomes. One such idea is prescriptive clinical prediction rules (pCPRs).

Prescriptive CPRs use baseline criteria called treatment effect modifiers, which are gathered from a physical examination to inform the type of treatment that a patient should preferentially receive. Their purpose is to better match patients to treatments, based on their predicted responsiveness to that treatment, independent of a diagnosis. As we stated in a recent BJSM editorial[1], pCPRs are a theoretical form of ‘personalized medicine’ that help translate research findings into clinical decisions at the level of the individual patient. Yet, despite the fact that multiple systematic reviews have noted concern about the value of these rules and the violations of these tools, pCPRs have been built into treatment based classifications[2] and have been incorporated into monodisciplinary clinical practice guidelines[3].

As the self-appointed medical examiner of this blog, I’m taking this opportunity to pronounce that pCPRs are dead. It’s no secret I’ve complained about these for some time[4], but let’s discuss the evidence and how I’ve come to this conclusion.

I’ll start with the obvious and well documented. The simple, derivative modeling methods used by numerous individuals in previous studies captured prognostic factors, rather than prescriptive factors[5]. In other words, the rules identified patients who were going to improve anyway, regardless of the treatment they received. In well documented pCPRs, there are assumptions that selected signs and symptoms such as acute pain or non-radicular symptoms, will respond more specifically to a designated form of treatment. In reality, the natural history associated with those signs and symptoms is very favorable, meaning that the improvement will not be associated with the care received, but with time. In fact, the natural history of many musculoskeletal conditions-such as acute low back-is so favorable, that patients require interventions that have very high treatment effects to provide any noticeable between-groups difference. Most physiotherapy interventions fail to provide very high treatment effects.

Conventional pCPRs require specific predictors, a detailed outcome measure (dichotomized), and an assumption that the intervention applied provides distinguishing mechanisms. To maximize the potential of the model, the predictors should be mediating factors. Non-modifiable variables such as age, gender, or duration of symptoms cannot be changed through treatment. Modifiable predictors such as fear, catastrophizing, strength loss, or flexibility, all can be targeted by interventions, leading to further influence on the outcome measure.

We have shown that altering one’s threshold of “success” can lead to different CPRs[1]. Simply put, different pCPRs will be created among the same group of patients receiving the same treatment if the outcome measure threshold is changed. We found that when the threshold on an ODI was a 30%-change versus a 50%-change, the CPRs were notably different; despite that both thresholds-and other percentages-have been advocated within the literature. This suggests that the modeling is fragile and will differ across studies, populations, and definitions of success/non-success in outcome measures. Further, when different outcomes measures are used, different CPRs are also created: within the same population[1].

An assumption of pCPRs is that there are differences in interventions and that targeting a patient’s needs with that intervention should improve overall outcomes. At present, this assumption is built more on theory than fact. In the psychological literature, the shared mechanisms theory suggests that there is similarity in outcomes across presumably different treatment approaches. There are many examples in the psychological literature that demonstrate, comparatively, there are no differences among the many forms of interventions for major depression[6], pain[7], and for panic disorder and obsessive-compulsive disorder[8]. Whether or not manual therapy, exercise, or other varieties of physiotherapy intervention influence patients with selected dysfunctions (e.g., spine) differently is still debatable.

It’s no secret, that in the mid 2000’s, journals clambered to publish pCPRs, despite questionable quality, small sample sizes, and inappropriate modeling methods [9]. The profession of physiotherapy was inundated with dubious pCPRs that only served to heighten the passion in using these tools. Publication created a deification of the tools and subsequently, these are firmly embedded in clinical practice and the clinician psyche. Removing ineffective care options that are shrouded in emotion from the clinical practice priorities of clinicians is a huge challenge but it is my hope that we can distance ourselves from pCPRs. Prescriptive CPRs lived a wild, short and controversial life, and wish them well on the other side. But we won’t miss them.

Acknowledgements

Dr Cook would like to thank Dr Frank Keefe for the information regarding the shared mechanisms theory.

About Chad Cook

Dr Chad Cook Duke UniversityDr Chad Cook is a Professor and Program Director of the Division of Physical Therapy in the Department of Orthopaedics and a Clinical Researcher in the Duke Clinical Research Institute, at Duke University. Chad is actively involved in big data research with a focus on orthopedic care processes and is a special topics editor for JOSPT and an associate editor with BJSM.

Email: Chad.cook@duke.edu Twitter: @chadcookPT 

References

[1] Haskins R, Cook C. Enthusiasm for prescriptive clinical prediction rules (eg, back pain and more): a quick word of caution. Br J Sports Med. 2016;50(16):960-1.

[2] Fritz JM, Cleland JA, Childs JD. Low back pain. Subgrouping patients with low back pain: evolution of a classification approach to physical therapy. J Orthop Sports Phys Ther. 2007;37(6):290-302.

[3] Delitto A, George SZ, Van Dillen LR, Whitman JM, Sowa G, Shekelle P, Denninger TR, Godges JJ; Orthopaedic Section of the American Physical Therapy Association. J Orthop Sports Phys Ther. 2012;42(4):A1-57.

[4] Cook CE. Potential pitfalls of clinical prediction rules. J Man Manip Ther. 2008;16(2):69-71.

[5] Hancock M, Herbert RD, Maher CG. A guide to interpretation of studies investigating subgroups of responders to physical therapy interventions. Phys Ther. 2009;89(7):698-704.

[6] Cuijpers P, Brännmark JG, van Straten A. (2008). Psychological treatment of postpartum depression: A meta-analysis. J Clin Psych. 2008;64(1):103–118.

[7] Wampold BE, Mondin GW, Moody M, Stich F, Benson K, Ahn H. (1997). A meta-analysis of outcome studies comparing bona fide psychotherapies: Empirically, “all must have prizes.” Psychological Bulletin. 1997;122(3):203-215.

[8] Ougrin D, Latif S. Specific psychological treatment versus treatment as usual in adolescents with self-harm: systematic review and meta-analysis. Crisis. 2011;32(2):74-80.

[9] Cook C, Shah A, Pietrobon R. Lumbopelvic manipulation for treatment of patients with patellofemoral pain syndrome: development of a clinical predication rule. J Orthop Sports Phys Ther. 2008;38(11):722.

Commissioning Editor: Neil O’Connell

Comments

  1. Dr. Cook, thank you, insightful as always. The crux of your post in my mind is “comparative effectiveness.” Is “manipulation” or “encouragement to move” or “dry needling” or “nothing” going to help with the stiff low back post injury? I find it uncomfortable to think that these researchers didn’t consider natural history and response to standard treatment on the whole in their research designs. Is that what you are implying? If one day we get better at sub-grouping patients into non-responders to natural history, and then can somehow break the non-responders into treatment groups and study them, will you conjur up your magic ju ju and raise CPRs from the dead?

    Chad cook Reply:

    I love that you still retain faith. I do as well. When the time is right I will conjure my magic ju ju. I will be the first to ju ju!!

  2. Kathleen Sluka says:

    Love this post. I think the problem with clinical prediction rules, at least as I have seen them applied in physical therapy, is that they are not examining what are the underlying mechanisms for the pain. Does the patient have a peripheral injury with peripheral sensitization, does the patient have neuropathic signs and symptoms, does the patient have central sensitization, does the patient have psychological barriers like catastrophizing, fear, or depression? Once we understand these, we might be able to make better choices. Pharmacological treatments are certainly geared this way: gabapentin is often given for neuropathic pain, local anesthetics and NSAIDs are greater for peripherally driven pain, and inflammation, reuptake inhibitor are great for those with central sensitization or loss of central inhibition. Our problem in physical therapy is that we are still learning the mechanisms by which our treatments reduce pain. If we could get a better handle on these, then we might be able to make more appropriate treatment choices based on matching the underlying mechanism of the pain with the underlying mechanism for how the treatment works to reduce pain. So for the person who talked about sub-grouping, yes that is a good start and a way to begin to look for underlying confounders and contributors to the pain outcomes, but we still need to be thinking about the mechanisms of the pain. It turns out that most people have several factors going on at once, which of course makes it difficult for the clinician. The up side: exercise can improve healing after nerve injury, reduces pro-inflammatory cytokines (peripherally, systemically, and centrally) while increasing anti-inflammatory cytokines, reduces central sensitization and excitability, and increases central inhibition. So you could just get everyone on an exercise program-but you need them to adhere to the program which is where other factors come into play!

  3. Agree on all points EG. I think “lack of responsibility” is a big one!

  4. Isn’t it time the term “multi-disciplinary pain clinic” disappeared too? Not just the term, but the whole concept?

    Consider from the patient’s point of view what it means to land in a MDPC.

    – “I must be in a bad place, because I’ve been told I need a whole team of people to help me”.
    – “I must be a hard case, because no single person is willing to take responsibility for helping me. By doing everything as a group, all these practitioners avoid personal responsibility for creating change”.
    – “If I need 4 different professions to help me, and I’m told progress is likely to be gradual, that means that each profession on its own is weak in effect. Why am I paying huge fees for treatments that barely work?”.

    kal fried Reply:

    I SOOOOOOO AGREEEEE!!!!
    Interventional pain mgt even worse.
    Wrote a blog on this – http://kalfried.com.au/wordpress/?p=73

  5. Let go JP.
    Picture a pain boxing match featuring ‘context’ versus ‘pathology’. Context wins by a knockout in the first round. And in every rematch. Observing a success does not make it a direct result of an intervention. It is time we all got off that bus.
    I suggest the science of the powerful psychobiological effects of placebo / nocebo be explored (Benedetti, Finnis and others).
    Definitely agree with EG et al that fundamental and broad paradigm change is required and the sooner the better. Science is questioning and re-questioning, not accepting. Our personal and professional sensibilities are only important to ourselves; they don’t benefit the considerable numbers of patients with non-adaptive pain who have paradoxically grown in number while treatment options and technologies have exploded.
    The body of research you refer to is indeed not dead at all. It simply won’t die because of the entrenched ‘belief’ that it should live, which is a serious ongoing problem.
    PS: great article Dr Cook.

  6. Jessie Podolak says:

    I appreciated this post from Dr. Cook, as it made me reflect on pCPRs, which I have used with very good success in my practice for the past 8 years. (The first 10 years of my career were spent chasing the proverbial left on right sacral torsion with very mixed results). While I think he makes some compelling points about prognostic factors versus predictive, and regarding the shared mechanisms theory, I find the overall tone of the post discouraging.

    I agree with many of the replies which credit the success of our various interventions to therapeutic alliance and all of the variables that are extremely difficult to measure. Perhaps it was the kindness, the confidence, the optimism, and/or the warm and skilled therapeutic touch I delivered that did the trick with my hundreds of manipulated patients who only needed 1-2 visits to turn around their episode of acute LBP. Perhaps most of them would have turned around on their own. However, they sought treatment, and I had something reasonable, safe, efficient, relatively cost-effective and helpful to offer. In truth, given the bio-psycho-social complexity of the human being, I doubt that we will ever be able to explain the exact mechanisms by which ANY intervention works.

    However, I do not see the pCPRs as an explanation for, nor an end-all be-all justification for, any one specific treatment. Do the researchers assert that they are intended to be absolutes? I believe the hard-working professionals who have dedicated much time, energy and talent to examining these “rules” (yes, we need a better term for than that) would be first to say they are intended to be a starting point in the decision making process. That is why they are so great for novice clinicians, but they are merely a starting point and not a cook book. As I see it, clinical prediction rules have aided in the delivery of quality care, and have propelled our profession to much higher levels in a relatively short amount of time, regardless of surrounding attitudes.

    We all know that evidence-based practice is the sweet spot where the literature, clinician experience, and patient values intersect. The “wild, short and controversial life” of the pCPRs has given some semblance of structure in an often contradictory and confusing sea of literature. But by no means are they dogma. We still are mandated to use our clinical decision making skills, and the originators of the rules will be the first to assert that. In my interactions with faculty at various PT programs, the rules are presented in the appropriate framework: tools/aides in a bigger decision-making process that considers more than just the literature.

    The discouraging aspect of the original post for me is that it highlights the discord within our profession. I am all for professional banter, challenging one another, and allowing “iron to sharpen iron” as we dialogue like this. But to declare a body of research DEAD and to “wish it well on the other side” seems disrespectful. I think we can play nicer in the sandbox together, can’t we? I can respect Dr. Cook’s opinion as anyone else’s, but had his tone been a bit more well-mannered toward the intent of the research in physical therapy, I would have taken his opinion with a little more gravity.

    EG Reply:

    “I doubt that we will ever be able to explain the exact mechanisms by which ANY intervention works”.

    Rogers proved with his extensive body of research that it’s entirely possible to nail down exact psychological mechanisms leading to success. Gendlin, his successor, also used research to add a crucial level of understanding. The advantage is obvious – useless processes are let go, and useful processes are honed. Speed, efficiency, efficacy, power.

    Fixing acute localized LBP in 1-2 treatments is standard. Even a student should be able to do that much. If we’re going to reach a point where it’s standard practice to fix severe, chronic LBP in 5 minutes, we need to know exactly what’s going on.

    It’s possible, but there needs to be a huge paradigm shift. All pain is caused by resistance to painful thoughts; all healing is caused by release of this resistance.

  7. Hi I had difficulty but found this paper.Interestingly I just read variables in the therapist are more important to outcomes than variables in the patient.

    http://sophia.stkate.edu/cgi/viewcontent.cgi?article=1057&context=msw_papers

    David Fitzgerald Reply:

    Thanks Graham- there’s a bit of reading in that!!

  8. Although I do not disagree with Dr. Cook, to offer some push back, let me suggest that for the novice, entry level clinician having a CPR as another tool to identify patients that ‘should’ improve might be beneficial. As with most, if not all things we use, it is part of a picture of the patient that must be interpreted through clinical reasoning.

  9. Do I read this correctly, and does this mean all the work on sub-typing low back pain presentations and then determining what should happen next is either redundant, or inaccurate? I’m inclined to agree but nevertheless there is an extraordinary amount of energy spent by various researchers and policy makers to embed “cookie cutter” approaches to treatment in our health system in NZ at least…

    Chad Cook Reply:

    Bronnie, I see the work toward classifying care mechanisms for LBP as a process that does not have to have pCPRs to be effective. In essence, a classification can consist of a number of different routing mechanisms that allow flexibility in filtering a patient toward of form of care. The challenge with pCPRs is that these are highly specific (usually) and are less flexible. Just my two cents. Chad

    Stephen McDavitt Reply:

    I’m with you Chad. Steve McD.

  10. I would tend to agree with Graham in that I now find myself more concerned with achieving mutually agreeable outcomes (whenever possible) rather than engaging in speculative mechanisms of therapeutic affect. As a trained manual therapist with a keen interest in pain science I must acknowledge that the efficacy story does change over time (and that I may, in fact, be pedalling mistruths) but If the therapeutic outcome is satisfactory I can live with this Machiavellian approach.
    If the alternative is to pontificate from a deluded position of academic superiority and deny patient’s treatment (which unfortunately is an almost daily occurrence in my jurisdiction) then indeed it is a sad future for the profession as John alluded to.
    Is it really so hard to know if we are helping a patient???

    John Barbis Reply:

    Good comments. I agree with most of them. The difficulty, however, comes in justifying our care for payment. Money always is the kicker. As someone who has spent a significant amount of my career on insurance expert panels trying to justify payment for physical therapy, not having good clinically based rationales, either based on potential physiological models or clinical outcome models, makes it difficult to argue that a patient needs highly trained professional care.

    I no longer am concerned about reimbursement issues, but I think most physical therapists are. Given the pressure to cut health costs and given the cost of PT in the US, we could be some of the low hanging fruit that is ripe for picking. I know the battles I have been involved in in the past. Those battles will become more difficult in the future with these outcome studies and how we verbally frame what PT will be will be critical to PT’s financial survival.

    David Fitzgerald Reply:

    Fair point John. For what it’s worth I’ve ran a cash based practice for close on 30 years. As such the final arbiter of reimbursement is the payer…the patient!
    I’ve found that this most “overt” transactional arrangement to be an excellent mediator of the value of a service. In fact, one of the biggest challenges I have had over the years is honing new Therapists skills to cope with the pressure of patient expectation and delivering a service of value (perceived or physical – if I dare to separate on a forum like this!!). This may appear crude (and un-deniably very non-scientific) but it does seem to reflect reality.
    I take your point regarding policy frameworks etc. and I have no answer to that dilemma (other than delight that I don’t have to engage!!). I wonder if such evaluation criteria were applied to all other medical /Allied health practice what the determination would be?

    Grahams point about therapeutic relationship is the “Elephant in the room” which we all have know about for a few decades now but the practical implications are so profound I don’t think clinician’s, researchers or policy makers what to really embrace this issue.
    No self-respecting clinician wants to think of themselves as a “snake-oil salesman” (no matter how well intentioned) but we may well be deluding ourselves by not addressing it?

    EG Reply:

    “Elephant…therapeutic relationship…but the practical implications are so profound”.

    Yes, they are extremely profound. Entire industries are set to be toppled, or transformed at least. It will be as massive as the technological/digital revolution.

    What I expect to see in the future:

    – Physios trained in apprenticeships
    – Heavy reliance on biofeedback (video, EEG, HRV, etc). and real time personal feedback
    – Skill-based learning > knowledge accumulation
    – A focus on *actually healing pain*, rather than improving functional capacity, fitness, core strength, attending psych sessions and paddling in a hydro pool.
    – A focus on speed and immediacy of effect
    – A focus on mental development – meditation, primarily, but also hypnosis and suggestion.

  11. What psychotherapy literature shows is that 60% of outcomes are governed by the therapeutic alliance, 30% by whether the practitioner believes in the treatment and 10% by the chosen treatment type(eg Gestalt,CBT etc).
    I don’t think physical therapies are any different.It’s about two people in a room working collaboratively,listening ,with empathy, compassion,caring and understanding of goals.
    Practitioner language (including body) can create positive or negative primes for the patient.
    Whether you rub ,stretch or click is immaterial.

    Nicole Andrews Reply:

    Hi Graham,
    Great point! Do you have a reference for the stats?

  12. John Barbis says:

    Well done. A recent article in the JOSPT also supports this idea. The study looks at the predictive value of MDT (Mckenzie Method). Here is the link for the abstract: http://www.jospt.org/doi/abs/10.2519/jospt.2016.6266.

    As an elder PT, all I can say is that I am glad that I am retired. As I look back on all of the concepts, theories, techniques and practice patterns that I accepted as either fact or best practice and had to discard in the past 40 years, I found that I could modify my practice style to accommodate the new information. With time I could make what would seem to be a radical change in how I practiced by incremental change. Recently, however, the evidence for change has come so quickly, that the core beliefs and practices can no longer be changed incrementally. The dissonance and professional discomfort produced by having to discard large chunks of most of what I practiced, studied, and worked at so hard over the years to master would be, in many ways, professionally earth shattering. Caught between what I felt comfortable and competent with and what I now know to be most likely ineffective would be a horrible place to be as I walked in to treat that next patient. Luckily I do not have to do that tomorrow or in the days ahead.
    Physical Therapy will need to look carefully at how it will be forced to evolve for it to survive. What scares me most is looking at the current future for the profession in the US where PT schools are graduating every larger numbers of graduates with ever higher levels of educational debt. Those numbers were based on past practice patterns. What does that portend for the future?