Jim Hinojosa (2013) wrote an interesting essay in the recent AJOT entitled "The evidence based paradox" which was published in "The Issue Is..." section. I believe that this is an important article to read and discuss.
I was a little concerned when I was reading the article because although he took some rather specific steps to tread cautiously and he did not go so far as an outright rejection of an evidence-based model there is still danger in propelling our thinking backward when you express the kinds of concerns that he expressed.
Hinojosa outlines barriers to EBP including possible inapplicability of the established levels of evidence given the important qualitative and individualized nature of OT practice. He correctly references the revised model that Tomlin and Borgetto (2011) presented that has already addressed this concern. Tomlin and Borgetto (2011) proposed a model of a 'Research Pyramid' that includes and expresses a value for qualitative inquiry, so I am not sure why this is still identified as a barrier by Hinojosa.
Hinojosa also criticizes systematic reviews for the potential of individualized bias and inclusion criteria errors but that is kind of like shooting fish in a barrel. Of course there will be the potential for bias in nearly every endeavor; the potential should not preclude adoption of methods that are otherwise sound as long as we maintain the ability to study and examine and even criticize the reviews! The point here is that even though there can be potential for 'problems' we should not take that as an invitation or opportunity to reject the method.
Evidence based practice is a standard and an objective - not an absolute. The reason why we know this is precisely related to the creation of the standards which includes reference to 'levels of evidence.' We can engage in academic conversations about whether or not the standards apply to the way that we practice but I think a more important method for framing the question is 'What happens when we step too far away from evidence-based models?'
Since I am a street level practitioner I would like to outline some things that happen when we step away from evidence. I see these things on a daily basis in my practice. Consider this:
When we step away from evidence based practice we are free to ignore good research and the efforts of many scientists who are attempting to better inform our practice strategies. That leaves us free to reject the fact that some interventions are simply not supported. What is the practical application of this? Well, if we reject evidence we are liberated to use treatment strategies that are not effective and we return to a lower point of theory driven decision making. Look no further than the state of pediatric practice to see this in action: we had a theory about a 'brushing' protocol that was sold like snake oil for over twenty-five years to thousands upon thousands of practitioners who paid money to learn the methods. Those thousands upon thousands of practitioners went out into the world and brushed an uncounted number of children - and justified their intervention based on what they learned from experts who were applying their 'theories.' They also brushed those uncounted children with the promised notion that 'research is coming!!!!' which was the mantra chanted in those continuing education courses - which I remember specifically because I attended one of the first ones back in 1989. Well fast forward to 2013 and in a retrospective analysis there has been an ENTIRE INDUSTRY developed around this intervention. Millions of dollars have been spent on continuing education and buying the 'right' brushes. It is impossible to calculate the dollar figure impact of how many sessions have been billed for this treatment. Now, what do we have to show for this treatment that is based on theory? We have absolutely NO EVIDENCE that it is even effective. In fact, it is not even referenced in our most basic entry level pediatric text books. I didn't edit those textbooks so I don't know what the decisions were related to why this extraordinarily mythical intervention was not included, but I suspect it had something to do with the fact that there is absolutely NO EVIDENCE that this is effective, and in fact those studies that have been done essentially show that there are no consistent effects that can cause us to have confidence in brushing with a 'particular method' and using a 'particular brush.'
Dr. Hinojosa is an academic and I am sure he must have contact with students who go out into the field and practice occupational therapy. What is an academic response to a student who looks in the basic pediatric textbooks and can't even find a reference to a brushing intervention that is so popularized by clinicians? Do we tell students that it is 'OK' to brush children based on our theories, even though our theories have not been supported by research - YET? EVER? MAYBE SOMEDAY???
HOW LONG DO WE WAIT??? Is 25 years long enough to wait before a clinician 'gives up' and assumes that this is snake oil and that research is actually NOT coming???
Here is another way to conduct the analysis in step with concerns that our practice is directed at an individual level and is therefore difficult to measure quantitatively. What I really want Dr. Hinojosa to reconsider is the impact of this on a family - a topic that he has studied (Hinojosa, 1990) in an article that I rather enjoyed. I evaluated a three year old child this week who has multiple developmental delays related to prematurity. Among many other problems, this child can not hold a toy because any time any object or person touches her hands she pulls them away. The parent WANTS the child to hold a toy and WANTS the child to play and not have this difficulty. At that moment I was confronted with a practice dilemma: do I practice by evidence or do I practice by mythology? What do I recommend??? I know the theory and am no clinical slouch. I understand Melczak and Wall (1965), I understand the animal research on deep pressure and calming (Grandin, 1992), and I also understand the theory of behavioral conditioning or placebo if that is what is in play, and I understand our mythic interventions that include brushing but that have no direct research support after 25 years.
So what are we supposed to do here, continue with mythic-based practice and subject the public to treatment strategies that any science-oriented thinker has long ago rejected? More specifically, with this child lying in the bean bag chair in front of me while the parent expresses her sadness that the child can't even hold onto a toy and play, what am I supposed to do?
Well at that moment, even though I can't find the reasons in my head to justify the use of brushing, I almost wanted to say the words to the family: 'Well you can TRY this. We don't really have any supporting evidence but some people find that it is helpful.' I thought of the study done on weighted vests that showed even though there was no change in the child at least it made the parents feel good (Stephenson & Carter, 2009).
Then the parent brought me back to reality. Even before I could say anything she said, "Please don't ask us to try brushing. Every therapist we have ever seen wants us to brush this child and I can't tell you how many times we have tried it. Some of them even blamed us for it not working because they said we weren't following the protocol closely enough."
If I am using ANY kind of evidence, even the qualitative kind, shouldn't I at least remember that parents 'know' more than therapists when it comes to this topic? Shouldn't I remember that when considering adherence to a brushing home program that a qualitative study indicated that parents are concerned with efficacy and response to the intervention while therapists were more concerned with whether or not the family had the time and was actually following the protocol (Segal & Beyer, 2006)?
I don't believe that there is any paradox or even dilemma. In consideration of quantitative and qualitative evidence, if that even matters, is that the evidence seems to be useful for guiding our practice. Or informing practice, at least, which Hinojosa seems to support.
I just want him to consider that publication of an opinion piece like this can lead to some practitioners having some sense of being justified in resorting to a 'theory-based' methodology and rejecting calls for scientific thinking. Theory-based methodologies have their time and place - but we need to be careful that we are not feeding pseudoscientific models just because someone can dream up a proposed theoretical justification for something. That is happening a lot, particularly in pediatric occupational therapy practice, and there is not enough discussion about the problem.
Questioning the value of evidence based practice has the potential to be a giant step backward, and in total, I think the Tomlin and Borgetto model (2011) already addressed every concern that Hinojosa raised in his essay.
Grandin, T. (1992). Calming effects of deep touch pressure in patients with autistic disorder, college students, and animals. Journal of Child and Adolescent Psychopharmacology, 2(1), 63–72.
Hinojosa, J. (1990). How mothers of preschool children with cerebral palsy perceive
occupational and physical therapists and their influence on family life. Occupational Therapy
Journal of Research, 10, 144–162.
Hinojosa, J. (2013). The issue is... The evidence-based paradox. American Journal of Occupational Therapy, 67, e18-e23.
Melzack, R., & Wall, P.D. (1965). Pain mechanisms: a new theory. Science, 150, 971–979 .
Segal, R., & Beyer, C. (2006). Integration and application of a home treatment program: A study of parents and occupational therapists. American Journal of Occupational Therapy, 60, 500–
Stephenson, J., & Carter, M. (2009). The use of weighted vests with
children with autism spectrum disorders and other disabilities. Journal
Of Autism And Developmental Disorders, 39(1), 105-114.
Tomlin, G. & Borgetto, B. (2011). Research pyramid: A new evidence-based practice
model for occupational therapy. American Journal of Occupational Therapy, 65(2), 189-196.