Twenty years of SIPT - where do we go next?
Standardized tests are periodically discarded or updated because the normative group that the test was developed around may have changed characteristics. I am unaware of any 'industry standards' regarding the life expectancy of standardized tests; rather, professionals in a field tend to come to consensus about the relative usefulness of tests on their own. I would be interested in knowing what other therapists think about the Sensory Integration and Praxis Tests - which were published in 1989.
I became certified in the administration of these tests seventeen years ago. At first I found them highly useful but the more I gave the tests the more I understood the limitations. Now it has gotten to a point where I do my best to discourage people when they ask for this test - not just because of the inherent weaknesses of the test construction but also because of how old the norms are.
There have been some legitimate concerns with the tests including extremely poor test-retest reliability for several of the subtests - most notably included postrotary nystagmus, kinesthesia, location of tactile stimulation, and figure-ground perception. The statistical data for these tests is published in the test manual (Ayres, 1989).
Additionally, a complex set of cluster and factor analysis studies include this data that has unacceptable reliability - and 'diagnostic groupings' were developed on a inconsistent data set that included these measures. Mulligan (1998) conducted a very interesting confirmatory factor analysis on the original groupings and found that there was not good fit between Ayres' original groupings and the data of over 10,000 cases in the WPS database. She completed a new exploratory factor analysis and proposed a new model of diagnostic groupings but I am uncertain if anything ever came of her suggestions.
Research is best when it builds upon previous research. Asher, Parham, & Knox (2008) completed a study ten years after Mulligan's analysis. They looked at validity of Ayres original diagnostic groupings as measured on two test cases. When I read this last year I remember thinking that the LACK of reference to Mulligan's work was really odd.
One interesting comment made in the Asher, Parham, & Knox article was that they used a convenience sample of therapists in the Los Angeles area. The significance of this could be lost on people who don't understand the geographic distribution of the SI research knowledge base - which is largely concentrated in a handful of areas around the country. The study made me wonder - would the results have been different if you asked therapists OUTSIDE of Los Angeles area? If anyone asked a certain therapist in Western New York to participate in a validity study on SIPT diagnostic groupings there would have been a loud question about the Mulligan study. It is a pertinent issue - you can't talk about broad validity when you only include people who are all drinking the same Kool Aid, so to speak.
All of this is lost on parents, who read things on the Internet and want their child tested using the SIPT. Sometimes parents come to me for therapy after someone else has tested their child on the SIPT. Many of those parents are confused when I suggest re-assessment using current tools - they already paid hundreds or thousands of dollars for some assessment that their insurance company didn't reimburse - and then I tell them that they need updated assessment that their insurance company WILL reimburse. It is hard to have a good conversation about test reliability and validity with parents who just invested a lot of money out of pocket - so I lose some of those patients to be sure and the parents go off looking for someone who will validate their SIPT investment. That is kind of unfortunate, and I think we are doing parents a disservice by setting them up for this kind of conflict.
Occupational therapists have a lot to offer children regarding assessment of practic, visual perceptual, and sensory motor functions that were measured in the original SIPT. We just need an updated test. This matters especially because there are statements about the SIPT being the gold standard of tests for sensory integration - and if we are relying on a twenty year old test with questionable reliability that failed a confirmatory factor analysis then we are in trouble.
This is all also related to the push for SPD to be included in the new DSM. I think it is inadequate to rely on parent or teacher reports of processing problems as the primary form of diagnostics. The information from these parent or teacher reports is informative but it is not diagnostic. We need an updated or reworked SIPT - and in my opinion we need to shelve this current test. We should all agree that it was an amazing first step - but the second step is long overdue.
References:
Asher, A.V., Parham, L.D., & Knox, S. (2008). Interrater reliability of Sensory Integration and Praxis Tests (SIPT) score interpretation. American Journal of Occupational Therapy, 62, 308-319.
Ayres, A. J. (1989). Sensory Integration and Praxis Tests. Los Angeles: Western Psychological Services.
Mulligan, S. (1998). Patterns of sensory dusfunction: A confirmatory factor analysis. American Journal of Occupational Therapy, 52, 819-828.
I became certified in the administration of these tests seventeen years ago. At first I found them highly useful but the more I gave the tests the more I understood the limitations. Now it has gotten to a point where I do my best to discourage people when they ask for this test - not just because of the inherent weaknesses of the test construction but also because of how old the norms are.
There have been some legitimate concerns with the tests including extremely poor test-retest reliability for several of the subtests - most notably included postrotary nystagmus, kinesthesia, location of tactile stimulation, and figure-ground perception. The statistical data for these tests is published in the test manual (Ayres, 1989).
Additionally, a complex set of cluster and factor analysis studies include this data that has unacceptable reliability - and 'diagnostic groupings' were developed on a inconsistent data set that included these measures. Mulligan (1998) conducted a very interesting confirmatory factor analysis on the original groupings and found that there was not good fit between Ayres' original groupings and the data of over 10,000 cases in the WPS database. She completed a new exploratory factor analysis and proposed a new model of diagnostic groupings but I am uncertain if anything ever came of her suggestions.
Research is best when it builds upon previous research. Asher, Parham, & Knox (2008) completed a study ten years after Mulligan's analysis. They looked at validity of Ayres original diagnostic groupings as measured on two test cases. When I read this last year I remember thinking that the LACK of reference to Mulligan's work was really odd.
One interesting comment made in the Asher, Parham, & Knox article was that they used a convenience sample of therapists in the Los Angeles area. The significance of this could be lost on people who don't understand the geographic distribution of the SI research knowledge base - which is largely concentrated in a handful of areas around the country. The study made me wonder - would the results have been different if you asked therapists OUTSIDE of Los Angeles area? If anyone asked a certain therapist in Western New York to participate in a validity study on SIPT diagnostic groupings there would have been a loud question about the Mulligan study. It is a pertinent issue - you can't talk about broad validity when you only include people who are all drinking the same Kool Aid, so to speak.
All of this is lost on parents, who read things on the Internet and want their child tested using the SIPT. Sometimes parents come to me for therapy after someone else has tested their child on the SIPT. Many of those parents are confused when I suggest re-assessment using current tools - they already paid hundreds or thousands of dollars for some assessment that their insurance company didn't reimburse - and then I tell them that they need updated assessment that their insurance company WILL reimburse. It is hard to have a good conversation about test reliability and validity with parents who just invested a lot of money out of pocket - so I lose some of those patients to be sure and the parents go off looking for someone who will validate their SIPT investment. That is kind of unfortunate, and I think we are doing parents a disservice by setting them up for this kind of conflict.
Occupational therapists have a lot to offer children regarding assessment of practic, visual perceptual, and sensory motor functions that were measured in the original SIPT. We just need an updated test. This matters especially because there are statements about the SIPT being the gold standard of tests for sensory integration - and if we are relying on a twenty year old test with questionable reliability that failed a confirmatory factor analysis then we are in trouble.
This is all also related to the push for SPD to be included in the new DSM. I think it is inadequate to rely on parent or teacher reports of processing problems as the primary form of diagnostics. The information from these parent or teacher reports is informative but it is not diagnostic. We need an updated or reworked SIPT - and in my opinion we need to shelve this current test. We should all agree that it was an amazing first step - but the second step is long overdue.
References:
Asher, A.V., Parham, L.D., & Knox, S. (2008). Interrater reliability of Sensory Integration and Praxis Tests (SIPT) score interpretation. American Journal of Occupational Therapy, 62, 308-319.
Ayres, A. J. (1989). Sensory Integration and Praxis Tests. Los Angeles: Western Psychological Services.
Mulligan, S. (1998). Patterns of sensory dusfunction: A confirmatory factor analysis. American Journal of Occupational Therapy, 52, 819-828.
Comments
However, I feel like just a drop in an ocean of overwhelming acceptance. I always feel that OTs are too much like sheep, eager to accept any technique on face value, regardless if they've been untested and unproven. Reading your blog was refreshing b/c it made me realize that I'm not alone in not being blindly accepting of popular OT "tools".
I think that as a part of an overall look at praxis the OPr test offered some interesting information, specifically as it related to motor planning WITHOUT the benefit of visual feedback.
Now I do not know if Oral Praxis REALLY relates to processing of proprioceptive information, although the concept certainly has reasonable face validity. I am not certain how much it matters because in reality there is so much parallel and heterarchical processing between all sensory channels that it might not even be correct to try to evaluate them distinctly.
I leave most oral practic considerations to speech therapists anyway who probably have a much deeper appreciation for such things. I suppose there are some potential areas of interest for OT, particularly if the OT is focusing on tongue lateralization and feeding issues - but I don't know if we need a standardized test to quantify such things. That brings me to the 'why?' question.
Despite this the point of the blog entry is that the test and norms are over 20 years old. Time to move on.