This chapter provides an introduction to the form of, advantages of, and software that implements the Multi-Faceted Rasch Model (MFRM)—an extension of the family of Rasch models that allows users to account for and correct for multiple sources of systematic measurement error. Specifically, the MFRM depicts measurement situations in which sampling occurs on more than one contextual variable relating to the indictors of the latent trait being measured. The three examples provided in the chapter—analyses of componential features of items, differential item functioning, and ratings—illustrate its application to three measurement contexts for which it would be difficult to obtain both diagnostic and corrective information using other measurement models.
Link to FACETS website (Note free demo versions of WINSTEPS and FACETS are available if you want to play with the software. I highly recommend it.)
Link to ACER ConQuest
Abstract (Summary)
This study examined whether fidelity of implementation to reform-based instructional strategies embedded in a middle school physical science curriculum unit developed by the Harvard-Smithsonian Center for Astrophysics moderated the causal relationship between curriculum condition and classroom mean achievement in a quasi-experiment testing the effectiveness of the unit. The study sample included 48 6 th grade science classrooms selected randomly from 8 Montgomery County Public Schools middle schools, assigned randomly to either the treatment or comparison condition in the Scaling up Curriculum for Achievement, Learning, and Equity Project (SCALE-0) quasi-experiment of The George Washington University.
This dissertation was a secondary analysis of SCALE-uP's 2005-2006 fidelity of implementation data collected with the Instructional Strategies Classroom Observation Protocol (ISCOP), which captured whether the Project 2061 instructional strategies rated Satisfactory or Excellent in the ARIES: Exploring Motion and Forces (M&F) treatment unit were present during implementation in treatment and comparison classrooms. ISCOP Likert-like scores for each classroom were subjected to Rasch analysis; rating scale diagnostics, category collapsing, and fit statistics were used to develop a reliable continuous fidelity of implementation measure for each classroom.
Results from hierarchical multiple regression analysis performed on the fidelity of implementation measures indicated that when controlling for prior knowledge, fidelity of implementation to the Project 2061 instructional strategies rated Satisfactory or Excellent in M&F moderated the causal relationship between science curriculum condition and classroom mean achievement. Follow-up post hoc analyses at two select fidelity measures indicated that treatment classrooms with High Fidelity were predicted to have higher classroom mean achievement than comparison classrooms with High Fidelity to the same set of instructional strategies, and this difference was statistically significant ( p <.05); however, there was no statistically significant difference in classroom mean achievement between treatment and comparison classrooms with Low Fidelity. Although reform-based instructional practices were present in both treatment and comparison classrooms, these practices were related positively to outcomes only in classrooms supported by the treatment unit. This dissertation showed that effects on student achievement are enhanced when teachers use reform-based science curriculum materials and have high fidelity of implementation to the instructional strategies embedded in these materials.
Here is a link to Dr. O'Connell's WINSTEPS control file if you want to replicate some of her analyses... Thanks to her for sharing!!!