<h1>Differentiating Among Gifted Learners: A Comparison of Classical Test Theory and Item Response Theory on Above-Level Testing</h1>
<h2>Brandon LeBeau, Susan G. Assouline, Duhita Mahatmya, Ann Lupkowski-Shoplik</h2>
<h3>Belin-Blank Center, University of Iowa</h3>
# Above Level Testing
- Participate in a Talent Search (for example, a university-based talent search)
- Above-level testing in school using available instruments or through a university program. For example, www.i-excel.org
# Above-level testing helps us discover talented students and tailor educational options for them
# BESTS In School Testing
- Belin-Blank Exceptional Student Talent Search (BESTS)
- 4th – 6th graders: Take I-Excel in **school**, not on the traditional open Saturday testing offered throughout the U.S.
- The goal: More students have access to above-level testing
+ Available to any talented student in the school
+ Not just for those whose parents show initiative to register for testing
# A new, online test, I-Excel…
- **…is an online test for very capable 4th – 6th graders **
+ uses an online platform developed by the Belin-Blank Center for use as an above-level assessment
- **…licenses content developed by ACT **
+ that was designed to measure academic progress of junior high students.
+ From that content, Belin-Blank has been identifying the academic talents of bright 4th- 6th grade students for over 20 years.
- **…provides the power of above-level testing**
# I-Excel
- Science (28 items)
- Math (30 items)
- Reading (30 items)
- English (40 items)
- Contains 8th grade level items
- Used as an above-level test for 4th-6th graders
# Challenges for use
- Reliability and validity are for an 8th grade normative sample
+ Is not a high-achieving sample
+ Would like the score to represent aptitude
- Same score range for all students
+ Ignores grade student currently is in
+ Ignores how far away student is from 8th grade content exposure
# Research Questions
1. To what extent does the information obtained through CTT compare with information obtained from IRT in the above-level testing environment?
2. How does the item level analysis with IRT impact aptitude estimates?
3. To what extent do CTT and IRT methods provide similar classifications of high-aptitude for students initially identified as high-achieving?
# Who this research benefits
- Researchers
- Practioners, teachers, parents could use the IRT aptitude scores
# Sample
- 1,893 students took I-Excel between March 2016 through July 2017
+ A single form of I-Excel
- Grades:
+ 4th: 13.7%
+ 5th: 28.1%
+ 6th: 58.2%
- Sex:
+ Female: 49%
+ Male: 51%
- Students took all four subareas (English, Math, Reading, and Science)
# Classical Test Theory (CTT)
- Number of questions answers correctly
- Percentile Ranks
- Item-level statistics
+ Proportion correct
+ Biserial correlations
- Statistics are sample dependent
- Scores based on 8th grade normative sample
# Item Response Theory (IRT)
- Mathematical model
- Similar to logistic regression, but with a latent variable (i.e. aptitude)
- Individual response strings are used to estimate...
+ item parameters (i.e. discrimination, difficulty)
+ person parameters (i.e. aptitude)
- Accounted for grade through multigroup IRT framework
# IRT Uses Student Response Strings
# Additional IRT Benefits
- Create developmental scale
+ Can measure growth
- Could move to computer adaptive testing
- Can use IRT linking methods
# CTT Violin Plots
# Differentiating students
# Average Aptitude Estimates
# Identify students ready for additional challenge
# Enhancing the 4Ds of talent development through IRT scale
- Improve reliability and validity of estimates of high-achieving students' aptitude
- Broaden identification of high-achieving students ready for advanced curriculum
- Better align assessments to interventions (e.g., enrichment, acceleration)
- Refine evaluation of high-achieving students' growth in domains
# Proportion Correct
# Explore Aptitude for Identification
# Thank You
- Questions?
- https://brandonlebeau.org/slides/nagc2019/