Mixing up chickens and eggs: A response to Fleming-Castaldy and Gillen
In the recent AJOT Fleming-Castaldy and Gillen wrote an interesting article entitled "The issue is: Ensuring that education, certification, and practice are evidence based." In the article, they examine why outdated neurophysiologic techniques are promoted in the profession of occupational therapy as opposed to more current and evidence based motor learning and motor control theories. They argue that outdated models based on Bobath and Brunnstrom have no place in contemporary practice.
In spirit I agree wholeheartedly. There are many techniques that are still present in contemporary practice and we hang onto those techniques even in the face of mounting evidence that other models are more appropriate. I have spent considerable time in this blog discussing this issue as it relates to certain sensory-based approaches and the very limited evidence that exists for their continued promotion and use.
Toward the end of the article they suggest that educators need to critically appraise their curricula, that textbook authors need to thoughtfully consider what they are including in texts, and that NBCOT needs to consider their exam development methodology.
They do not mention any role for the professional association (AOTA) outside of a general reference to ACOTE standards and I think this is a serious error. This weekend at the AOTA conference I attended a special interest section workshop that promoted three pediatric sensory based interventions that have virtually no supporting evidence. One of those interventions was a very controversial listening/auditory method that has been labeled experimental by other important professional associations. I believe that our professional association has a direct responsibility to promote higher quality presentations at the national conference that reflect consideration of evidence - particularly when those presentations are coming out of the special interest sections of the association.
I have experience with certification examinations so I wanted to address several incorrect statements about the role of a certifying body. These are my own opinions and I am not speaking for NBCOT as I am no longer on their board.
Fleming-Castaldy and Gillen state that outdated material regarding neurophysiological approaches appears on the certification examination and in the reference texts that are used to create the examination. They further state that educators feel compelled to teach that material because they want their students to pass the examination. These statements are begging the question based on the incorrect assumption that the certification examination drives educational content. This is a serious logical fallacy.
In actuality, NBCOT generates its list of references based on a survey of EDUCATIONAL PROGRAMS and TEXTBOOKS THAT EDUCATORS ARE USING IN THEIR CURRICULA. The list of textbooks is not created out of whole cloth and the point of origin is not NBCOT.
Evidence takes on many forms - including what is written in textbooks and what is anecdotal to practice. I fully agree that this is not the best evidence but a certification examination has the responsibility to reflect actual practice - even when actual practice is ignoring higher level evidence like research studies. It is a conundrum to be sure.
Items appear in the practice analysis and are ultimately reflected in an examination blueprint based on the criticality of responses from entry level practitioners. In other words, entry level practitioners are using those techniques. If they are using those techniques it is not because they appear on an NBCOT reference book list. They are using those techniques because that is what they were taught and that is what they see in the field.
Therefore, the correct origination of content is the academic program and the field. NBCOT uses references based on feedback from programs about the texts they are using and by responses of entry level practitioners.
Furthermore, Fleming-Castaldy and Gillen state that NBCOT practice analysis survey respondents may not have updated their practice and may not be using evidence. This is a factually incorrect assertion because the 2008 practice analysis was given to practitioners who worked only from 0-36 months. If practitioners who completed the survey don't use evidence based practice it is not because they haven't UPDATED their methods; it is because they were never educated in those methods OR if they were educated in those methods then they chose to abandon them when they entered practice. That has nothing to do with practice analysis methodology - that is a problem in the field.
It is important to raise this question about why there is a hesitancy in our profession to abandon methods that lack evidence and I congratulate the authors for raising an important issue. However, I believe it is important to point out some errors in their analysis as well as to assign greater responsibility to the professional association. The certification examination is a lagging indicator that is downstream of many other problems including broad professional tolerance for outdated methods, teaching outdated methods in occupational therapy curricula, lack of AOTA guidance about abandoning outdated methods, and poor textbook writing.
There would be an appropriate hue and cry if a certifying body suddenly took it upon itself to unilaterally dictate the standards of what the profession was supposed to teach and practice. Certification examination development is a mirror of the profession - and nothing more. The best that a certification body can do is HOPE to indirectly influence practice by promoting evidence (e.g. offering free database access to all certificants). They can't directly drive that process - that is the responsibility of the profession itself, and that includes its educators and its practitioners.
References:
Castaldy-Fleming, R. and Gillen, G. (2013). The issue is: Ensuring that education, certification, and practice are evidence based. American Journal of Occupational Therapy, 67, 364-369.
National Board for Certification in Occupational Therapy (2008). 2007 NBCOT Practice Analysis. NBCOT: National Board for Certification in Occupational Therapy
In spirit I agree wholeheartedly. There are many techniques that are still present in contemporary practice and we hang onto those techniques even in the face of mounting evidence that other models are more appropriate. I have spent considerable time in this blog discussing this issue as it relates to certain sensory-based approaches and the very limited evidence that exists for their continued promotion and use.
Toward the end of the article they suggest that educators need to critically appraise their curricula, that textbook authors need to thoughtfully consider what they are including in texts, and that NBCOT needs to consider their exam development methodology.
They do not mention any role for the professional association (AOTA) outside of a general reference to ACOTE standards and I think this is a serious error. This weekend at the AOTA conference I attended a special interest section workshop that promoted three pediatric sensory based interventions that have virtually no supporting evidence. One of those interventions was a very controversial listening/auditory method that has been labeled experimental by other important professional associations. I believe that our professional association has a direct responsibility to promote higher quality presentations at the national conference that reflect consideration of evidence - particularly when those presentations are coming out of the special interest sections of the association.
I have experience with certification examinations so I wanted to address several incorrect statements about the role of a certifying body. These are my own opinions and I am not speaking for NBCOT as I am no longer on their board.
Fleming-Castaldy and Gillen state that outdated material regarding neurophysiological approaches appears on the certification examination and in the reference texts that are used to create the examination. They further state that educators feel compelled to teach that material because they want their students to pass the examination. These statements are begging the question based on the incorrect assumption that the certification examination drives educational content. This is a serious logical fallacy.
In actuality, NBCOT generates its list of references based on a survey of EDUCATIONAL PROGRAMS and TEXTBOOKS THAT EDUCATORS ARE USING IN THEIR CURRICULA. The list of textbooks is not created out of whole cloth and the point of origin is not NBCOT.
Evidence takes on many forms - including what is written in textbooks and what is anecdotal to practice. I fully agree that this is not the best evidence but a certification examination has the responsibility to reflect actual practice - even when actual practice is ignoring higher level evidence like research studies. It is a conundrum to be sure.
Items appear in the practice analysis and are ultimately reflected in an examination blueprint based on the criticality of responses from entry level practitioners. In other words, entry level practitioners are using those techniques. If they are using those techniques it is not because they appear on an NBCOT reference book list. They are using those techniques because that is what they were taught and that is what they see in the field.
Therefore, the correct origination of content is the academic program and the field. NBCOT uses references based on feedback from programs about the texts they are using and by responses of entry level practitioners.
Furthermore, Fleming-Castaldy and Gillen state that NBCOT practice analysis survey respondents may not have updated their practice and may not be using evidence. This is a factually incorrect assertion because the 2008 practice analysis was given to practitioners who worked only from 0-36 months. If practitioners who completed the survey don't use evidence based practice it is not because they haven't UPDATED their methods; it is because they were never educated in those methods OR if they were educated in those methods then they chose to abandon them when they entered practice. That has nothing to do with practice analysis methodology - that is a problem in the field.
It is important to raise this question about why there is a hesitancy in our profession to abandon methods that lack evidence and I congratulate the authors for raising an important issue. However, I believe it is important to point out some errors in their analysis as well as to assign greater responsibility to the professional association. The certification examination is a lagging indicator that is downstream of many other problems including broad professional tolerance for outdated methods, teaching outdated methods in occupational therapy curricula, lack of AOTA guidance about abandoning outdated methods, and poor textbook writing.
There would be an appropriate hue and cry if a certifying body suddenly took it upon itself to unilaterally dictate the standards of what the profession was supposed to teach and practice. Certification examination development is a mirror of the profession - and nothing more. The best that a certification body can do is HOPE to indirectly influence practice by promoting evidence (e.g. offering free database access to all certificants). They can't directly drive that process - that is the responsibility of the profession itself, and that includes its educators and its practitioners.
References:
Castaldy-Fleming, R. and Gillen, G. (2013). The issue is: Ensuring that education, certification, and practice are evidence based. American Journal of Occupational Therapy, 67, 364-369.
National Board for Certification in Occupational Therapy (2008). 2007 NBCOT Practice Analysis. NBCOT: National Board for Certification in Occupational Therapy
Comments