<h1>Differentiating Among Gifted Learners: A Comparison of Classical Test Theory and Item Response Theory on Above-Level Testing</h1> <h2>Brandon LeBeau, Susan G. Assouline, Duhita Mahatmya, Ann Lupkowski-Shoplik</h2> <h3>Belin-Blank Center, University of Iowa</h3> # Above Level Testing - Participate in a Talent Search (for example, a university-based talent search) - Above-level testing in school using available instruments or through a university program. For example, www.i-excel.org # Above-level testing helps us discover talented students and tailor educational options for them ![](/figs/bell-curve.jpg) # BESTS In School Testing - Belin-Blank Exceptional Student Talent Search (BESTS) - 4th – 6th graders: Take I-Excel in **school**, not on the traditional open Saturday testing offered throughout the U.S. - The goal: More students have access to above-level testing + Available to any talented student in the school + Not just for those whose parents show initiative to register for testing # A new, online test, I-Excel… - **…is an online test for very capable 4th – 6th graders ** + uses an online platform developed by the Belin-Blank Center for use as an above-level assessment - **…licenses content developed by ACT ** + that was designed to measure academic progress of junior high students. + From that content, Belin-Blank has been identifying the academic talents of bright 4th- 6th grade students for over 20 years. - **…provides the power of above-level testing** # I-Excel - Science (28 items) - Math (30 items) - Reading (30 items) - English (40 items) - Contains 8th grade level items - Used as an above-level test for 4th-6th graders # Challenges for use - Reliability and validity are for an 8th grade normative sample + Is not a high-achieving sample + Would like the score to represent aptitude - Same score range for all students + Ignores grade student currently is in + Ignores how far away student is from 8th grade content exposure # Research Questions 1. To what extent does the information obtained through CTT compare with information obtained from IRT in the above-level testing environment? 2. How does the item level analysis with IRT impact aptitude estimates? 3. To what extent do CTT and IRT methods provide similar classifications of high-aptitude for students initially identified as high-achieving? # Who this research benefits - Researchers - Practioners, teachers, parents could use the IRT aptitude scores # Sample - 1,893 students took I-Excel between March 2016 through July 2017 + A single form of I-Excel - Grades: + 4th: 13.7% + 5th: 28.1% + 6th: 58.2% - Sex: + Female: 49% + Male: 51% - Students took all four subareas (English, Math, Reading, and Science) # Classical Test Theory (CTT) - Number of questions answers correctly - Percentile Ranks - Item-level statistics + Proportion correct + Biserial correlations - Statistics are sample dependent - Scores based on 8th grade normative sample # Item Response Theory (IRT) - Mathematical model - Similar to logistic regression, but with a latent variable (i.e. aptitude) - Individual response strings are used to estimate... + item parameters (i.e. discrimination, difficulty) + person parameters (i.e. aptitude) - Accounted for grade through multigroup IRT framework # IRT Uses Student Response Strings ![](/figs/theta_overtime_color.png) # Additional IRT Benefits - Create developmental scale + Can measure growth - Could move to computer adaptive testing - Can use IRT linking methods # CTT Violin Plots ![](/figs/NPR_violin.png) # Differentiating students ![](/figs/rawscore_math_english_aptitude_color.png) # Average Aptitude Estimates ![](/figs/average_numcorrent.png) # Identify students ready for additional challenge ![](/figs/classification.png) # Enhancing the 4Ds of talent development through IRT scale - Improve reliability and validity of estimates of high-achieving students' aptitude - Broaden identification of high-achieving students ready for advanced curriculum - Better align assessments to interventions (e.g., enrichment, acceleration) - Refine evaluation of high-achieving students' growth in domains # Proportion Correct ![](/figs/prop_correct.png) # Explore Aptitude for Identification ![](/figs/math_accuracy.png) # Thank You - Questions? - https://brandonlebeau.org/slides/nagc2019/