Jim Hinojosa (2013) wrote an interesting essay in the recent AJOT entitled "The evidence based paradox" which was published in "The Issue Is..." section. I believe that this is an important article to read and discuss.
I was a little concerned when I was reading the article because although he took some rather specific steps to tread cautiously and he did not go so far as an outright rejection of an evidence-based model there is still danger in propelling our thinking backward when you express the kinds of concerns that he expressed.
Hinojosa outlines barriers to EBP including possible inapplicability of the established levels of evidence given the important qualitative and individualized nature of OT practice. He correctly references the revised model that Tomlin and Borgetto (2011) presented that has already addressed this concern. Tomlin and Borgetto (2011) proposed a model of a 'Research Pyramid' that includes and expresses a value for qualitative inquiry, so I am not sure why this is still identified as a barrier by Hinojosa.
Hinojosa also criticizes systematic reviews for the potential of individualized bias and inclusion criteria errors but that is kind of like shooting fish in a barrel. Of course there will be the potential for bias in nearly every endeavor; the potential should not preclude adoption of methods that are otherwise sound as long as we maintain the ability to study and examine and even criticize the reviews! The point here is that even though there can be potential for 'problems' we should not take that as an invitation or opportunity to reject the method.
Evidence based practice is a standard and an objective - not an absolute. The reason why we know this is precisely related to the creation of the standards which includes reference to 'levels of evidence.' We can engage in academic conversations about whether or not the standards apply to the way that we practice but I think a more important method for framing the question is 'What happens when we step too far away from evidence-based models?'
Since I am a street level practitioner I would like to outline some things that happen when we step away from evidence. I see these things on a daily basis in my practice. Consider this:
When we step away from evidence based practice we are free to ignore good research and the efforts of many scientists who are attempting to better inform our practice strategies. That leaves us free to reject the fact that some interventions are simply not supported. What is the practical application of this? Well, if we reject evidence we are liberated to use treatment strategies that are not effective and we return to a lower point of theory driven decision making. Look no further than the state of pediatric practice to see this in action: we had a theory about a 'brushing' protocol that was sold like snake oil for over twenty-five years to thousands upon thousands of practitioners who paid money to learn the methods. Those thousands upon thousands of practitioners went out into the world and brushed an uncounted number of children - and justified their intervention based on what they learned from experts who were applying their 'theories.' They also brushed those uncounted children with the promised notion that 'research is coming!!!!' which was the mantra chanted in those continuing education courses - which I remember specifically because I attended one of the first ones back in 1989. Well fast forward to 2013 and in a retrospective analysis there has been an ENTIRE INDUSTRY developed around this intervention. Millions of dollars have been spent on continuing education and buying the 'right' brushes. It is impossible to calculate the dollar figure impact of how many sessions have been billed for this treatment. Now, what do we have to show for this treatment that is based on theory? We have absolutely NO EVIDENCE that it is even effective. In fact, it is not even referenced in our most basic entry level pediatric text books. I didn't edit those textbooks so I don't know what the decisions were related to why this extraordinarily mythical intervention was not included, but I suspect it had something to do with the fact that there is absolutely NO EVIDENCE that this is effective, and in fact those studies that have been done essentially show that there are no consistent effects that can cause us to have confidence in brushing with a 'particular method' and using a 'particular brush.'
Dr. Hinojosa is an academic and I am sure he must have contact with students who go out into the field and practice occupational therapy. What is an academic response to a student who looks in the basic pediatric textbooks and can't even find a reference to a brushing intervention that is so popularized by clinicians? Do we tell students that it is 'OK' to brush children based on our theories, even though our theories have not been supported by research - YET? EVER? MAYBE SOMEDAY???
HOW LONG DO WE WAIT??? Is 25 years long enough to wait before a clinician 'gives up' and assumes that this is snake oil and that research is actually NOT coming???
Here is another way to conduct the analysis in step with concerns that our practice is directed at an individual level and is therefore difficult to measure quantitatively. What I really want Dr. Hinojosa to reconsider is the impact of this on a family - a topic that he has studied (Hinojosa, 1990) in an article that I rather enjoyed. I evaluated a three year old child this week who has multiple developmental delays related to prematurity. Among many other problems, this child can not hold a toy because any time any object or person touches her hands she pulls them away. The parent WANTS the child to hold a toy and WANTS the child to play and not have this difficulty. At that moment I was confronted with a practice dilemma: do I practice by evidence or do I practice by mythology? What do I recommend??? I know the theory and am no clinical slouch. I understand Melczak and Wall (1965), I understand the animal research on deep pressure and calming (Grandin, 1992), and I also understand the theory of behavioral conditioning or placebo if that is what is in play, and I understand our mythic interventions that include brushing but that have no direct research support after 25 years.
So what are we supposed to do here, continue with mythic-based practice and subject the public to treatment strategies that any science-oriented thinker has long ago rejected? More specifically, with this child lying in the bean bag chair in front of me while the parent expresses her sadness that the child can't even hold onto a toy and play, what am I supposed to do?
Well at that moment, even though I can't find the reasons in my head to justify the use of brushing, I almost wanted to say the words to the family: 'Well you can TRY this. We don't really have any supporting evidence but some people find that it is helpful.' I thought of the study done on weighted vests that showed even though there was no change in the child at least it made the parents feel good (Stephenson & Carter, 2009).
Then the parent brought me back to reality. Even before I could say anything she said, "Please don't ask us to try brushing. Every therapist we have ever seen wants us to brush this child and I can't tell you how many times we have tried it. Some of them even blamed us for it not working because they said we weren't following the protocol closely enough."
If I am using ANY kind of evidence, even the qualitative kind, shouldn't I at least remember that parents 'know' more than therapists when it comes to this topic? Shouldn't I remember that when considering adherence to a brushing home program that a qualitative study indicated that parents are concerned with efficacy and response to the intervention while therapists were more concerned with whether or not the family had the time and was actually following the protocol (Segal & Beyer, 2006)?
I don't believe that there is any paradox or even dilemma. In consideration of quantitative and qualitative evidence, if that even matters, is that the evidence seems to be useful for guiding our practice. Or informing practice, at least, which Hinojosa seems to support.
I just want him to consider that publication of an opinion piece like this can lead to some practitioners having some sense of being justified in resorting to a 'theory-based' methodology and rejecting calls for scientific thinking. Theory-based methodologies have their time and place - but we need to be careful that we are not feeding pseudoscientific models just because someone can dream up a proposed theoretical justification for something. That is happening a lot, particularly in pediatric occupational therapy practice, and there is not enough discussion about the problem.
Questioning the value of evidence based practice has the potential to be a giant step backward, and in total, I think the Tomlin and Borgetto model (2011) already addressed every concern that Hinojosa raised in his essay.
Grandin, T. (1992). Calming effects of deep touch pressure in patients with autistic disorder, college students, and animals. Journal of Child and Adolescent Psychopharmacology, 2(1), 63–72.
Hinojosa, J. (1990). How mothers of preschool children with cerebral palsy perceive
occupational and physical therapists and their influence on family life. Occupational Therapy
Journal of Research, 10, 144–162.
Hinojosa, J. (2013). The issue is... The evidence-based paradox. American Journal of Occupational Therapy, 67, e18-e23.
Melzack, R., & Wall, P.D. (1965). Pain mechanisms: a new theory. Science, 150, 971–979 .
Segal, R., & Beyer, C. (2006). Integration and application of a home treatment program: A study of parents and occupational therapists. American Journal of Occupational Therapy, 60, 500–
Stephenson, J., & Carter, M. (2009). The use of weighted vests with
children with autism spectrum disorders and other disabilities. Journal
Of Autism And Developmental Disorders, 39(1), 105-114.
Tomlin, G. & Borgetto, B. (2011). Research pyramid: A new evidence-based practice
model for occupational therapy. American Journal of Occupational Therapy, 65(2), 189-196.
Saturday, February 23, 2013
Thursday, February 14, 2013
It is important for families to understand how dysfunctional our municipal entities are in administering healthcare systems. The current problems with the municipal early intervention program underscores ABC Therapeutics' commitment to providing private services for families through their own health insurance plans. Still, there is a need for a municipal system that serves as a social safety net - I am just not sure if there will be any providers willing to work in those systems if things continue down this road. -CJA
Brad Hutton, Director
New York State Department of Health
Bureau of Early Intervention
Empire State Plaza
Corning Tower, Room 287
Albany, NY 12237-0660
Bureau of Early Intervention
Empire State Plaza
Corning Tower, Room 287
Albany, NY 12237-0660
February 14, 2013
Dear Mr. Hutton:
I am in receipt of the recent email announcement regarding cut off dates for transfer of responsibility for early intervention billing from the Counties to the State that will be effective 4/1/13.
There is a concerning lack of information in this email given that there will be significant changes to the billing systems in the very near future. In fact, I have not even received any kind of contract from NY State that will outline the new relationship that will begin in just 45 days.
Most importantly, the email states that information about billing methodology will be sent to providers in mid-March. That will leave only two weeks' notice to interpret the guidelines and to develop systems for billing.
I am an agency provider and I employ several therapists who work in the Early Intervention Program. Payment delays are already routine and planned for given the known reimbursement cycle from the Counties. The unknown billing system that will be implemented in 45 days by the State and unknown reimbursement cycle is a severe threat to the employment of many people who work for the Early Intervention Program.
I suspect that larger agency providers will be able to absorb the uncertainty of these times, but as you are aware, a very large number of small agency and individual providers are under threat. No one can operate a small business under a situation where the billing methodology is unknown and the reimbursement cycle is unknown.
This lack of functional communication to the providers who serve the Early Intervention Program can place the system in jeopardy. Even if large agencies are able to absorb payment delays it is unlikely that they will hire up all of the smaller providers who have to leave the system. Shortages of providers is a real potential problem.
Early intervention is just a fractional portion of my private practice, but I am concerned that it may soon be an even smaller portion because quite frankly I don't know how I am supposed to provide services to a program that seems to be taking steps to purposefully implode. This has all the appearances of a planned demolition.
If it is not the government's intention to purposefully implode this program, I suggest delaying implementation of a new billing system and providing at least a 2 month window so providers can plan for and implement the new billing processes and plan for a possibly new reimbursement cycle. I am not fighting the changes, sir; I am only interested in having a reasonable notice period so I can actually review the pending contract and figure out how the new relationship will pragmatically work. I want to continue employing people who will work under your program. Will NY State allow this?
The provider community for the Early Intervention Program has been patiently waiting for guidance. Two weeks' notice is unreasonable.
Christopher J. Alterio, Dr.OT, OTR
Owner, ABC Therapeutics
Tuesday, February 12, 2013
There is a preliminary injunction regarding the Early Intervention arms length (conflict of interest) regulations. The new regulations that prohibit agencies from both evaluating and providing services will apparently not be in effect until a final decision in the case is granted. Municipalities are now waiting for actual BEI direction on how to proceed.
1. The NYS Developmental Disabilities Planning Council has a good compilation of links describing the adopted changes HERE
2. My blog post supporting the regulations and additional justification for that support can be found HERE
3. You can view the ACTS press release HERE. I have a copy of the order granting the injunction but am having some technical difficulties getting it uploaded onto my server - so I will provide a link as soon as I am able.
The order is very interesting and the Court essentially found that there was merit to the argument that the Governor overstepped his authority and bypassed the legislature because these changes were initially presented to the legislature and voted down.
It seems that the Court rejected statistics related to frequency of self-referral and it will be interesting to see if that information is re-presented because the data is in fact compelling. I am wondering why the Court did not see it that way.
The most important consideration for providers is that a lawsuit claiming technical errors in Executive process may temporarily keep the political barbarians from the clinical gate of the early intervention program. However, the barbarians were using battering rams the first time around. Just wait until they break out the trebuchets - because you can be certain that this is no longer just a pragmatic concern of the Counties related to their Medicaid costs - now it is also a slap across the face of Executive power.
I believe in checks and balances too - but this is a time when we should be contributing to policy creation by negotiation and participation. Attempting to solve this through the Courts might not lead to an end that people really want.
Thursday, February 07, 2013
For the entire day I have been on cute overload. Two ridiculously cute infants and then six ridiculously cute toddlers and preschoolers all came through my doors today for OT. Another therapist saw several ridiculously cute grade school students later in the afternoon. They were all so cute that by the end of the day I was practically giddy.
Then at the very end of the day I saw an older child who we all absolutely love. He is also ridiculously cute. He has a severe disability and is nonverbal and for whatever reason he had a full blown crisis and put his foot right through my wall. That never happened before with him - but of course we have had lots of experiences over the years when kids come in and have behavioral difficulties and become really upset. Parents sometimes feel embarrassed or sometimes mad or sometimes frustrated and sometimes they feel all those things at the same time.
As my friend settled and the parent readied him to leave I said, "We will see you next week." The parent replied, "If you are willing to sustain more damage!"
I tried to explain that of course I wanted him to come back and that I could care less about the wall and actually I was quietly praying and very thankful that he didn't direct his anger toward a window where he could have injured himself - but I don't know if the parent heard me.
So I wanted to write it down as a proclamation today. I love cute kids and I had plenty of that today. But I also love kids even when it might seem that it is impossible to see them as cute - or when their behaviors are really difficult to manage - because actually the cute factor does not drive my understanding of how kids of all temperaments and skills might need help.
I can fix a wall. That's easy to do. I know it is harder to try and fix behavioral difficulties for children who are nonverbal and who have severe disabilities. I also know that parents can be at their wit's end because these problems can seem so intractable. But I will still try and I will not give up even in the face of the meltdowns that are sure to happen over time.
I'll still try even if a child kicks a hundred holes in my wall.
Actually, those meltdowns motivate me to try to be even more helpful. And I don't get intimidated or upset in any way. Even when a child's behavior is not 'cute.'
And that is as honest as I know how to be about it.