New York Formative Assessment Technical Assistance Project

Lessons Learned

The collaborative effort among the Assessment and Accountability Comprehensive Center (AACC), the New York Comprehensive Center (NYCC), the New York State Education Department (NYSED), and Syracuse City School District (SCSD) has focused on increasing the use of formative assessment practices in SCSD pilot schools and classrooms. This project was initiated in August 2007, and has engaged staff from the four organizations in ongoing professional development opportunities designed to increase the knowledge, skills and practice of the utilization of formative assessment to inform teaching and learning in SCSD teachers and NYSED staff.

The following "lessons learned" provides a reflective list of findings relative to both the challenges and successes experienced by the collaborating partners focused on the development of formative assessment practice within the Syracuse City School District. These "lessons learned" are a result of reflective discussions by partner members, and will be continually updated as the effort and project progresses.

1. Assessing readiness factors for all participating organizations of a Formative Assessment (FA) initiative or study is essential to determine if sufficient resources, (personnel, programmatic, and material) are present to sustain both initial and long-term success.

For both State Education Department and Regional Comprehensive Center staff, it was critical to determine the level and extent of the engagement required over multiple years to work with a local district on formative assessment practice. Partners clarified various roles and responsibilities at the district and state level through initial and ongoing discussions. Over time and iteratively, commitments and roles become clearer through the use of a variety of tools and research resources including: Program Readiness Factors (see link below), the NYCC Capacity Building Model, and various articles related to implementation research. Identifying multiple personnel with content, professional development and assessment expertise that could commit to a sustained multiyear effort was essential. In addition, it is important to review criteria that focus on a district's level of preparedness prior to engaging in any formative assessment pilot initiative.

See: Readiness Factors: Criteria for Evaluating a District's Level of Preparedness for a Formative Assessment Pilot Study (PDF)

2) It is important to have, identify, and support district leaders responsible for coaching and supervising implementation of new skills and practice within the classroom who have been trained in communication, coaching, and systems change skills.

NYCC professional developers have worked regularly with district leaders and instructional support teachers (ISTs) charged with and committed to the support and implementation of formative assessment practice within district classrooms. The district's commitment of financial and personnel resources in this manner is essential to the support for teacher growth at the classroom level. The understanding and ability to utilize "adult learning" strategies and skills will assist instructional support teachers responsible for the growth of formative assessment practice within district classrooms. The existence, or development of internal structures utilizing partners that can be leveraged to provide support, increases an opportunity for success beyond what district personnel alone might be able to accomplish.

3) There is a need to build an adequate amount of time with state, district, and school leadership participants prior to implementation to develop a clear vision of program goals and implementation activities.

NYCC consultants used the term "period of incubation and planning" to describe the importance, before implementation begins, to clarify vision, purpose, process and commitment. Defining the roles of all technical assistance providers and district leaders is critical. This was accomplished through a series of six planning meetings between March and October of 2007. These meeting involved all partners and focused on developing specific goals and aligned activities to support implementation. This process can often take longer than expected when working to align project goals with existing district initiatives. In Syracuse, as with many districts, a sense of 'urgency for improvement' has resulted in the initiation of many separate initiatives and the resulting challenges to develop and follow a cohesive unified plan for realizing intended outcomes and improvement targets.

4) An agreed upon "plan of action" needs to be realistic in both expectations for engagements and a time frame for implementation.

Often, competing programs within a district stretch the capacity for deep staff learning and engagement. During the initial planning stage, technical assistance providers and district personnel need to design training and technical assistance engagements that are supported by opportunities for district staff to learn new skills and content. Examples of training agendas used in Syracuse can be accessed through the links at the end of this paragraph. Opportunities to practice and reinforce the implementation of new strategies within classrooms are essential. During the first year of work NYCC staff worked with Syracuse mathematics coaches to build their knowledge and skills in working with classroom teachers on formative assessment practices such as: Understanding the progression of learning, and developing high quality questions and strategies to provide student feedback.
See: Learning Progressions, Margaret Heritage and
Syracuse Formative Assessment Training Agendas (DOC).

5) A clearly articulated definition of formative assessment needs to be understood by, and communicated to, all participants. Access to high-quality research and expertise is necessary to guide program development and inform practice.

Support from an expert plays an invaluable role in clarifying the definition of formative assessment, as well as shaping and focusing a district study or initiative so that it is fully aligned and supported by the most current research. For the Syracuse Formative Assessment Project, Dr. Margaret Heritage of the AACC initially framed the "big picture" and definition of formative assessment, and currently provides regular feedback on the design of professional development and district support for teachers. This consultation is important in guiding professional developers and district leaders in promoting change within classrooms aligned to the most current research on formative assessment practice.

PowerPoints used by Dr. Heritage with NYSED, NYCC, and Syracuse district leaders and staff during an initial August 21st, 2007 training session can be accessed through the following links:
Assessment: Quality Tools and Processes (PPT) and
Formative Assessment (PPT).

The Council of Chief State School Officers (CCSSO), State Collaborative on Assessment and Student Standards (SCASS), and Formative Assessment for Teachers and Students Project (FAST), currently post publications providing valuable information relative to the definition and attributes of formative assessment.
See: Attributes of Effective Formative Assessment, Sarah McManus and
Learning Progressions, Margaret Heritage.

6) Having a firm foundation in 'data literacy,' understanding different types of data and purposes for data use helps district staff become more effective in implementing formative assessment practice.

The ability to discriminate among different types and purposes for data leads to greater fluency and 'ease of use' in collaborative engagements related to formative assessment practice. For example, Syracuse schools and teachers that have previously engaged with programs that promote and require regular data use, (e.g. Reading First) demonstrated greater 'ease' in discussions and planning for classroom instruction informed by formative assessment data. The opportunity to receive training in the different types of data being utilized, as well as data 'terms' and 'use,' should be considered for staff with little previous experience in this area. An important document developed with district staff outlined the district's assessment approach: Identifying assessments, assessment tools, where assessments fit in the 'assessment cycle,' and expectations for data use to help articulate the aligned system of assessment for all involved in the project.
See: Syracuse City School District Assessment Approach and Assessment Cycle.

7) Teacher content knowledge is important for the successful implementation of formative assessment practice. Content knowledge can be developed simultaneously with the implementation of formative assessment if there is external support.

The development of formative assessment practice within a classroom requires that teachers have the necessary content knowledge to assess student misconceptions, gaps in learning, and respond during the course of instruction to the learning needs. Providing students with appropriate interventions to clarify misconceptions, gaps in knowledge, skills and understanding is best supported by teachers who have deep content knowledge with the ability to design additional learning opportunities informed by evidence from formative assessment.

8) Professional development engagements need to communicate the research, theoretical knowledge, and identify images or "pictures of practice" that exemplify the use of formative assessment strategies in the classroom.

NYCC professional developers found the sharing of the theoretical knowledge behind formative assessment practice to be fairly straightforward. However, communicating the look of a classroom that engages in regular formative assessment practice is a more significant challenge. Providing video images (see links below) and committing to a cyclical approach of theory informing practice, and a continual review of changing practice informed by theory has been important in the development of new classroom strategies. It is important to have good models and exemplars for teachers beginning or expanding their use of formative assessment practice. One vehicle that the NYCC staff used to show models of FA practice was the use of classroom videos.
See: Teachers Video 1: Primary Assessment - Formative Assessment 1 and
Teachers Video 2: Primary Assessment - Formative Assessment 2

In addition, NYCC staff continually worked with Syracuse district leaders and mathematics coaches to reinforce the idea that formative assessment practice includes a range of attributes that all need to be utilized.
See: Attributes of Effective Formative Assessment, Sarah McHanus