Guidelines for Ensuring the Technical Quality of Assessments Affecting English Language Learners and Students with Disabilities:
Development and Implementation of Regulations

Overview and Background

These guidelines, prepared by the Special Populations Strand of the Assessment and Accountability Comprehensive Center (AACC), focus on the technical quality of assessments for English language learners (ELLs) and students with disabilities (SWDs). This document is an evolving document that will periodically be updated to incorporate new information. This document is intended to provide information to Regional Comprehensive Centers (RCCs) and states as they work to comply with the regulations of No Child Left Behind (NCLB) affecting their special student populations (i.e., SWDs, ELLs). These guidelines also are intended to help RCCs and states:

• gauge where a state is with regard to meeting federal requirements relevant to the assessment and accountability of special student populations;
• focus attention on priority issues related to implementing practices and systems that are in compliance with federal regulations; and
• select implementation strategies that have evidence of effectiveness, given the particular needs and conditions of the state.

As mentioned previously, information presented in these guidelines will be updated as new and relevant research, guidance, and strategies become available for consideration and evaluation by the AACC. Additionally, these guidelines will be updated to meet the evolving needs of RCCs and states.

Regions reported that "helping raise the achievement of at-risk, special needs, and ELL students" is a key priority with significant implications for the development of accountability and assessment systems.

States are at varying stages of implementing federal regulations (NCLB Title I, Title III) affecting the assessment and accountability of their special student populations. According to our initial analysis, which involved an examination of the needs and priorities related to assessment and accountability that were identified in the U.S. Department of Education–organized Regional Advisory Committee (RAC) Reports, nine of the ten regions reported that “helping raise the achievement of at-risk, special needs, and ELL students” is a key priority with significant implications for the development of accountability and assessment systems (see Table 1 for an overview of needs across regions).

Table 1. Needs and Priorities Related to Assessment and Accountability Identified in the Regional Advisory Committee Reports¹

Priorities
Help raising achievement of at-risk, special needs, and ELL students	X	X		X	X	X	X	X	X	X
Appropriate assessments that are valid and reliable for special and diverse populations (e.g., ELLs, special education, low SES, ethnic minority)	X	X				X	X	X	X	X
Training for teachers in use of assessment data	X			X	X			X		X
Formative and summative assessments	X	X			X		X	X		X
Resources to address needs identified by assessment data	X			X	X		X
Alignment of standards, instruction, and assessment	X			X	X			X
Training for administrators in use of assessment data	X				X		X			X
User-friendly and timely dissemination of assessment data	X			X				X
Locally developed assessments (linguistically and culturally appropriate)			X			X	X
Dissemination of best practices		X				X
Assessment-related technology training for teachers								X
Consistency in benchmarking assessments from LEA to LEA										X
Assessment development training								X

Confirmatory evidence of the most pressing assessment and accountability needs identified by the RAC reports was obtained from a review and analysis of NCLB reports, evaluations, and critiques across the research and political spectrum that ranged from the highly technical (Linn, Baker, & Betebenner, 2002; Gong, 2005; Fast & Hebbler, 2004; NCES, 2003, 2004; Rabinowitz & Ananda, 2002; Rabinowitz, 2004) to the more general (Center on Education Policy, 2005; Uzzell, 2005; Education Week, 2004, 2005). Therefore, a strand of the AACC’s work is dedicated to the assessment and accountability of ELLs and SWDs.

Federal Peer Review comments to states have identified areas where states need assistance vis-à-vis key review criteria and with regard to their special student populations. Such results, along with recent publications and surveys, indicate that states and districts need help with the development and implementation of technically adequate assessment systems for special student populations (Abedi, 2004; Herman & Dietel, 2005; American Diploma Project, 2004), and special attention to the technical quality of these assessments is needed to ensure they are valid and accessible for these students.

States and districts need help with the development and implementation of technically adequate assessment systems for special student populations, and special attention is needed to ensure these systems are valid and accessible for these students.

Various strategies and systems for assessment and accountability of ELLs and SWDs exist; however, they are not aggregated in any methodical fashion so that there is no complete understanding of the (a) quality² of these strategies/systems and the (b) context or conditions³ under which these strategies/systems are being implemented. Additionally, no framework exists to meaningfully organize the information that is available to RCCs and states.

Therefore, this document focuses on the issue of technical quality and presents:

• critical elements from the Federal Peer Review technical quality criteria (Title I) and Title III Office of English Language Acquisition, Language Enhancement, and Academic Achievement for Limited English Proficient Students (OELA) Monitoring Reports, with available examples of acceptable and incomplete evidence;
• a comparison of Federal Peer Review (Title I) critical elements with validated criteria for ensuring the technical adequacy of assessments for special student populations;
• a summary of research and resources relevant to key issues, including: accommodations, standard setting, and Annual Measurable Achievement Objectives (AMAOs) for ELLs; and
• a comparison of Title I and Title III requirements for assessing ELLs that are applicable to the technical quality of those assessments.

Note: At the request of the U.S. Department of Education, the AACC is leading the development of an initial draft framework for English language proficiency standards and assessments. The AACC will provide updates on this framework as they become available.

¹ Based on available reports as of July 2005. Recent conversations with Regional Comprehensive Centers (RCCs) and states have confirmed that the assessment and accountability of English language learners (ELLs) and students with disabilities (SWDs) continue to be areas of need.
² Quality refers to the degree to which the strategies/systems comply with NCLB regulations (Title I, Title III).
³ Context and conditions include: financial, political, historical, and demographic.

Please cite as: Sato, E., Rabinowitz, S., Worth, P., Gallagher, C., Lagunoff, R., & McKeag, H. (2007). Guidelines for Ensuring the Technical Quality of Assessments Affecting English Language Learners and Students with Disabilities: Development and Implementation of Regulations. (Assessment and Accountability Comprehensive Center report). San Francisco: WestEd.

© 2007 WestEd. All rights reserved.

Home | About Us | NCLB | Resources | Events | Contact Us| Login