Differentiating Among Gifted Learners: A Comparison of Classical Test Theory and Item Response Theory on Above-Level Testing

Brandon LeBeau

2019-11-08

Above Level Testing

• Participate in a Talent Search (for example, a university-based talent search)
• Above-level testing in school using available instruments or through a university program. For example, www.i-excel.org

Above-level testing helps us discover talented students and tailor educational options for them

BESTS In School Testing

• Belin-Blank Exceptional Student Talent Search (BESTS)
• 4th – 6th graders: Take I-Excel in school, not on the traditional open Saturday testing offered throughout the U.S.
• The goal: More students have access to above-level testing
  • Available to any talented student in the school
  • Not just for those whose parents show initiative to register for testing

A new, online test, I-Excel…

• …is an online test for very capable 4th – 6th graders
  • uses an online platform developed by the Belin-Blank Center for use as an above-level assessment
• …licenses content developed by ACT
  • that was designed to measure academic progress of junior high students.
  • From that content, Belin-Blank has been identifying the academic talents of bright 4th- 6th grade students for over 20 years.
• …provides the power of above-level testing

I-Excel

• Science (28 items)
• Math (30 items)
• Reading (30 items)
• English (40 items)
• Contains 8th grade level items
• Used as an above-level test for 4th-6th graders

Challenges for use

• Reliability and validity are for an 8th grade normative sample
  • Is not a high-achieving sample
  • Would like the score to represent aptitude
• Same score range for all students
  • Ignores grade student currently is in
  • Ignores how far away student is from 8th grade content exposure

Research Questions

1. To what extent does the information obtained through CTT compare with information obtained from IRT in the above-level testing environment?
   
2. How does the item level analysis with IRT impact aptitude estimates?
   
3. To what extent do CTT and IRT methods provide similar classifications of high-aptitude for students initially identified as high-achieving?

Who this research benefits

• Researchers
• Practioners, teachers, parents could use the IRT aptitude scores

Sample

• 1,893 students took I-Excel between March 2016 through July 2017
  • A single form of I-Excel
• Grades:
  • 4th: 13.7%
  • 5th: 28.1%
  • 6th: 58.2%
• Sex:
  • Female: 49%
  • Male: 51%
• Students took all four subareas (English, Math, Reading, and Science)

Classical Test Theory (CTT)

• Number of questions answers correctly
• Percentile Ranks
• Item-level statistics
  • Proportion correct
  • Biserial correlations
• Statistics are sample dependent
• Scores based on 8th grade normative sample

Item Response Theory (IRT)

• Mathematical model
• Similar to logistic regression, but with a latent variable (i.e. aptitude)
• Individual response strings are used to estimate…
  • item parameters (i.e. discrimination, difficulty)
  • person parameters (i.e. aptitude)
• Accounted for grade through multigroup IRT framework

IRT Uses Student Response Strings

Additional IRT Benefits

• Create developmental scale
  • Can measure growth
• Could move to computer adaptive testing
• Can use IRT linking methods

CTT Violin Plots

Differentiating students

Average Aptitude Estimates

Identify students ready for additional challenge

Enhancing the 4Ds of talent development through IRT scale

• Improve reliability and validity of estimates of high-achieving students’ aptitude
• Broaden identification of high-achieving students ready for advanced curriculum
• Better align assessments to interventions (e.g., enrichment, acceleration)
• Refine evaluation of high-achieving students’ growth in domains

Proportion Correct

Explore Aptitude for Identification

Thank You

• Questions?
• https://brandonlebeau.org/slides/nagc2019/