Can Your State Be a Pilot Site for Federal Growth Model Flexibility?

A Self-Evaluation Tool

New: Federal Peer Review Guidance - January 25, 2006

On November 21, 2005, U. S. Education Secretary Margaret Spellings issued a policy letter inviting State educational agencies (SEAs) to submit proposals that incorporate growth models into State plans for school and district accountability.¹ According to the letter, the flexibility to allow growth models flows from the Department’s “commonsense approach to implementing the No Child Left Behind Act of 2001 (NCLB).” The approach establishes that States that showed results and followed the tenets of NCLB would be eligible for flexibility to help them meet the law's goal of getting every child to grade level proficiency by 2013-14. Growth models are one such example of flexibility. The deadline for submitting proposals for growth model flexibility is February 17, 2006.

In her letter, the Secretary identified seven Core Principles that States must meet to be granted flexibility under this pilot program. In addition to the letter, the Department also disseminated information about the program through a fact sheet and a press release. The Council of Chief State School Officers (CCSSO) has compiled a website with other resources on this program. The CCSSO will sponsor a daylong seminar on growth models and this pilot program on January 25, 2006. In this document, we step through the core principles to help you evaluate whether your state’s system meets them. For each principle, we provide a description of readiness indicators, including examples, organized by the following key:

	Red light. The accountability system does not meet the principle.
	Yellow light. The accountability system incompletely meets the principle or has the potential within 12 months to meet the principle. A promising feature or approach that is likely to provide useful information for improving data, assessment, or accountability systems might also appear in this category, if there are concerns with some facet of the approach.
	Green light. The accountability system completely meets the principle.

The U. S. Department of Education (ED) intends to approve in 2006 no more than ten proposals for growth model flexibility, and ED has stated that it will approve only those proposals that can meet each of the core principles. A State with something short of seven “green lights” on this tool may be able, however, to address gaps in the accountability system to secure federal approval. It should be emphasized that this tool is intended as a pre-proposal guide and does not substitute for the technical peer review of proposals for this pilot program.

Background on Growth and Accountability Models

It may be useful to distinguish growth models in general from accountability models appropriate for NCLB. Successful proposals will need to clearly explain both. The growth model describes how changes in student performance will be measured, including the set of mathematical steps or procedures that the State will follow to measure growth. The accountability model, by contrast, is the larger framework that describes how the State will use this information (and perhaps other information) to make a decision about adequate yearly progress (AYP).

There are three main types of accountability models: status, improvement, and growth (Gong, 2004; Goldschmidt et al., 2005; Haertel, 2005).² The status model looks at a snapshot of proficiency at a point in time. The improvement model compares the proficiency of the current year's students with the proficiency of earlier students, usually from the prior year. The growth model looks at changes in the proficiency of students for whom test scores at multiple points in time are available (usually the prior and current years). The accountability model specifies how the proficiency information will be used to make a determination concerning AYP. This federal pilot program will consider State proposals to use growth models as the basis for AYP decisions.

Core Principle 1. The accountability model must ensure that all students are proficient by 2013-14 and set annual goals to ensure that the achievement gap is closing for all groups of students.

Core Principle 1 relates directly to three ideas that are central to the original NCLB legislation: that all students reach proficiency by 2013-14, that the States have annual proficiency goals, and that States monitor the performance of all groups of students.

The growth targets for students who are scoring below grade level are the same or lower than those for students who are scoring at or above grade level. Such targets will perpetuate or increase the existing achievement gap. The accountability model defers the required growth so that students need to grow substantially more in later years than in earlier years. A model with this property does not reflect realistic, continuously improving expectations for growth.

Students who are scoring below grade level have comparable growth targets for one year (or two) as students who are scoring at or above grade level. The State would need to present strong evidence to justify such a provision, since the achievement gap would not be closing while comparable targets are in effect. The State presents an incomplete plan to compare how the growth model works with how the current AYP model works. States receiving approval will be expected to provide data comparing the two models.

Growth targets are larger for students who are farther below proficient. The targets are systematic and lead to proficiency for all students by 2014. Appropriate growth targets for all students serve to narrow the achievement gap and would close the gap by 2014. The State presents a comprehensive plan to compare how the growth model works with how the current AYP model works. Differences between the models are clear, including how measures of status and safe harbor operate within the growth model.

Core Principle 2. The accountability model must establish high expectations for low-achieving students, while not setting expectations for annual achievement based upon student demographic characteristics or school characteristics.

Many States have argued that allowing growth models would be fundamentally fairer to schools, particularly to schools that have had lower performance but have exhibited high growth. Core Principle 2 ensures that student or school characteristics are not the basis for achievement targets. States may use past performance, as it relates to a trajectory of growth to proficiency, as part of their proposals.

The State proposes a model that controls for student demographics or school characteristics. If the State proposes a regression or multivariate/multi-level model, the independent variables may not include race/ethnicity, socio-economic status, school AYP status, or any other academic covariate.

Though growth patterns inform the targets, the targets are determined so that all students reach proficiency. ED guidance has stated that models must measure growth relative to standards, not relative to typical growth patterns. However, ED has been clear and consistent that moving all students to proficiency by 2014 is the overarching goal.

The proposal clearly defines growth trajectories for reading/language arts and mathematics that lead to proficiency by 2014. Performance expectations are coherent and lead to proficiency relative to rigorous grade level achievement standards.

Core Principle 3. The accountability model must produce separate accountability decisions about student achievement in reading/language arts and in mathematics.

Another of the central features of the NCLB legislation is separate accountability measures and decisions in reading/language arts and mathematics. Core Principle 3 reinforces this NCLB tenet.

The accountability model combines reading/language arts and mathematics performance into a single composite score used for accountability. With a single composite score determining adequate yearly progress, information about where to target resources and interventions is not available. Index models may be used if the index can be calculated separately for reading and mathematics and for each relevant student subgroup. The index model would also need to meet other conditions specified in Part IV of the attachment to the November 21, 2005 letter from Secretary Spellings. Other subject areas compensate for low achievement in reading/language arts or mathematics. While States are free to report student achievement in other subject areas, high achievement in reading/language arts and mathematics is the only way to meet AYP.

The model produces separate accountability decisions about achievement in the two subjects, but there are compensatory elements in the model. For example, at a school, sufficiently high growth in mathematics for one grade's students might overcome lower growth in mathematics at another grade. The State needs to demonstrate that the model produces separate accountability decisions for reading and mathematics.

The model produces separate accountability decisions about achievement in reading/language arts and in mathematics. AYP is met when there is sufficient achievement in both reading/language arts and mathematics. The State's proposal must include reporting that separates performance by subject area. Index methods, if used, are done so within a subject area, not across subject areas.

Core Principle 4. The accountability model must ensure that all students in the tested grades are included in the assessment and accountability system. Schools and districts must be held accountable for the performance of student subgroups. The accountability model, applied statewide, must include all schools and districts.

NCLB calls for testing and accountability for all students in the tested grades. In addition, the State must disaggregate and report scores for all numerically significant student subgroups. Core Principle 4 ensures that this requirement is preserved under the growth model flexibility.

One or more student groups are not fully represented in the assessment and accountability system. All students need to be represented in the assessment and accountability system. The data system in place does not allow for the disaggregations called for by NCLB. The growth model proposal will need to include how the State plans to disaggregate and report performance data for all of the subgroups required by NCLB.

The State's data collection activities during the 2004-05 school year did not yield the required data, but those of 2005-06 do. The proposal needs to demonstrate that the data system can support all of the accountability requirements of NCLB.

All students are included in the assessment and accountability system, and the system includes all required subgroups. The proposal demonstrates that all students are included in the assessment and accountability system. The State has designed robust provisions for schools and districts that are small or subject to high mobility. The State has considered the challenges to validity and reliability of these cases and has developed procedures to promote technical quality.

Core Principle 5. The State's NCLB assessment system, the basis for the accountability model, must include annual assessments in each of grades three through eight and high school in both reading/language arts and mathematics, must have been operational for more than one year, and must receive approval through the NCLB peer review process for the 2005-06 school year. The assessment system must also produce comparable results from grade to grade and year to year.

In its assessment provisions, NCLB defines the technical requirements of a Statewide assessment system. Core Principle 5 restates technical features that a successful growth model proposal must have.

The assessment program or the accountability model does not include all of the required grades for more than one year. The accountability model is based on results from tests only at benchmark grades. Assessment results are reported on a scale that is not comparable from year to year or does not allow the computation of growth. States will need to demonstrate comparability of scores from year to year. The assessment system did not receive peer review approval for the 2005-06 school year. Some required piece of the assessment system is missing or insufficient to garner peer review approval.

The assessment system received full approval with recommendations in the peer review process. A State must have either full approval or full approval with recommendations in order to be approved for the growth model pilot. A large field test data set is available to provide information on the performance of one or two missing grades. The assessment system will cover all required grades next year. The State's proposal must address how the State will determine growth at all grades.

The proposal clearly explains how students will be assessed with tests that produce comparable results from grade to grade and year to year. Technical information that establishes comparability of scores is present in the proposal.

Core Principle 6. The accountability model and related State data system must track student progress.

Using students' progress as the basis for AYP is perhaps the biggest change that the growth model flexibility will afford successful SEA proposals. To make that happen, States must meet Core Principle 6.

The State's proposal is based on a model that does not track individual student progress. The model must track the assessment and accountability-related data of individual students over time.

The databases that contain assessment data over multiple years are not fully linked through a unique student identifier, but the State can demonstrate that it can track students and make a connection between each student's score from one grade to the next. The State would need to demonstrate how it plans to connect students and their data across multiple years.

The State provides evidence of the capacity of its data system to implement a growth model, including linking student scores through such means as a unique student identifier. The proposal contains enough technical detail about the data system so that it is clear that the system can sustain a growth model.

Core Principle 7. The accountability model must include student participation rates in the State's assessment system and student achievement on an additional academic indicator.

Core Principle 7 ensures that States maintain two features of NCLB's accountability provisions under the growth model flexibility.

The accountability model omits either participation rates or an additional academic indicator. Proposals must include these elements.

The accountability model includes participation rates and an additional academic indicator, but the State needs to estimate certain information for certain schools and districts. The proposal should explain the rationale for and method of any estimation or imputation of information.

The accountability model includes participation rates as well as an additional academic indicator. The proposal includes, for all required student groups, the participation rates and the performance on an additional academic indicator.

^1. Secretary Spellings' letter and other helpful resources are listed in the Resources section at the end of this document.

^2. These sources are available online and provide more detailed description of the models. Note that Haertel (2005) uses the terms Uniform Target, Successive Cohort, and Individual Growth to refer to the status, improvement, and growth models, respectively.

Resources

Choi, K., Goldschmidt, P., and Yamashiro, K. 2005. Exploring models of school performance: from theory to practice. In Yearbook of the National Society for the Study of Education vol. 104, J. Herman and E. Haertel Eds. Blackwell Publishing.

Council of Chief State School Officers. 2005. CCSSO assistance to SEA leaders on growth models. http://www.ccsso.org/projects/Accountability_Systems/Growth_Models/.

Goldschmidt, P., Roschewski, P., Choi, K., Auty, W., Hebbler, S., Blank, R., & Williams, A. 2005. Policymakers' guide to growth models for school accountability: how do accountability models differ? Washington, DC: Council of Chief State School Officers. Available http://www.ccsso.org/publications/details.cfm?PublicationID=287.

Gong, B. 2004, November 15. Models for using student growth measures in school accountability (Paper presented at the Council of Chief State School Officers "Brain Trust" on value-added models). Retrieved December 5, 2005 from http://www.nciea.org/publications/GongGrowthModels111504.pdf.

Gong, B., Blank, R. K., & Manise, J. G. 2002. Designing school accountability systems: Towards a framework and process. Washington, DC: Council of Chief State School Officers. Available http://www.ccsso.org/content/pdfs/designing_school_acct_syst.pdf.

Haertel, E. H. 2005. Using a longitudinal student tracking system to improve the design for public school accountability in California. Stanford, CA: Stanford University. Retrieved December 26, 2005 from http://ed.stanford.edu/suse/faculty/haertel/Haertel-Value-Added.pdf.

Spellings, M. 2005. Address to the Council of Chief State School Officers' (CCSSO) Annual Policy Forum, Richmond, VA, November 18, 2005. Retrieved December 5, 2005 from http://www.ed.gov/news/pressreleases/2005/11/11182005.html#skipnav2.

Spellings, M. 2005, November 21. Key policy letters signed by the education secretary or deputy secretary [regarding additional flexibility related to growth models]. Washington, DC: U.S. Department of Education. Retrieved December 5, 2005 from http://www.ed.gov/policy/elsec/guid/secletter/051121.html.

U. S. Department of Education. Growth Models: Flexibility and Accountability [fact sheet]. Washington, DC: U.S. Department of Education. Retrieved December 5, 2005 from http://www.ed.gov/admins/lead/account/growthmodel/factsheet.html.

Home | About Us | NCLB | Resources | Events | Contact Us| Login