Guide to Quality Instructional Materials

Content Areas
Alabama is currently vetting science materials for all grade levels.

Team Selection
The ALEX Team recruits Resource Reviewers based upon their educational expertise as well as experience with the ALEX Program. Once recruited, the Team considers two variables when ranking reviewer effectiveness:

• Timeliness – How long does a lesson review take after assigned?
• Review Quality – Is the resource ready for Tier 2 approval without having to be rejected? Are there grammatical or spelling issues? Do objectives, procedures, and content standards align?

By using these two variables, the Team ranks reviewers on a scale from 1 – 5. Reviewers with a higher rank are the ones who are utilized first when resources are submitted for review. Typically, reviewers can expect to spend an average of 30 minutes per lesson review. You can find the ALEX Resource Reviewer Application by clicking here.

Process and Training
ALEX provides the ALEX Lesson Plan Self-Check for reviewers as well as the ALEX Lesson Plan Criteria Checklist. ALEX utilizes an online interface for vetting lesson plans and resources. ALEX assigns lesson plans/resources to reviewers. The online interface allows the reviewer to see what the lesson plan will look like when published. It also allows the reviewer to check links and make minor changes like grammatical, spelling, etc. After reviewing the resource and determining that it meets the criteria, the reviewer approves the resource. If a resource does not meet the criteria, the reviewer adds comments to the author. Comments should include information on how the lesson can be improved and why it was rejected. Otherwise, the author will have no guidance on how to improve chances for the lesson being approved. The reviewer The reviewer rejects the resource based on Level1. This action sends an email with comments to the plan author AND the ALEX Team, which also includes the review.

In addition to the online review interface, ALEX also maintains a spreadsheet to keep track of where the lesson plans are in the review process. If you reject a lesson, the reviewer should type “No” in the Tier1 Approved column of the spreadsheet. If the resource author resubmits, it will automatically come back to your evaluator’s area. Review the plan again to see if appropriate changes were made. If not, reject the plan again and make a note in the Notes column. You are responsible for reevaluating and rejecting/approving a plan if it is resubmitted, even after you have been paid for the review. If the resource passes your review, click Approve Level 1, then change the Tier1Approved “No” to a “Yes” and add to the Notes column.

Tip
If, for example, there are problems with the Title, Overview, and Procedures, etc., list those lesson plan parts in a Word or Google Doc to make notes. You might consider recording notes for every reviewed plan in one document. Notes can then be copied and pasted in the Comments box if the plan is rejected, and also archived in case there are questions later.

Tools
Reviewers use the ALEX Lesson Plan Criteria Checklist for lesson plan review. The checklist includes five criteria:

• Criteria 1: Alignment to CCRS/CoS
• Criteria 2: Focus/Shifts of the CCRS/CoS
• Criteria 3: Instructional Supports
• Criteria 4: Assessment
• Criteria 5: Format and Technical Logistics

A lesson plan that Meets Criteria is one in which the writer provides compelling evidence of a clearly-written, well-structured Lesson Plan that engenders student engagement, deep learning, and preparation for living/working successfully a digital world.

Compensation/Benefits
ALEX pays reviewers $20.00 for each review for lesson plans. ALEX assigns lessons to reviewers based upon their stated area(s) of expertise, and expect the reviewers to be “subject-matter experts.” Reviewers are paid after completing 10 reviews.

Platform
The Alabama Learning Exchange (ALEX) is a free website designed to curate and share educational materials and resources. Lesson plans and resources are searchable by subject area and grade, as well as author, title, school, or ID number. Educators can create a lesson plan or resource by creating an ALEX Personal Workspace account. Personal Workspace enables educators to:

• Access Alabama’s CCRS Insight Tool.
• Save and Share Resources
• Submit Resources for other teachers to use.

In Idaho, the Curricular Materials adoption process provides review and evaluation of new curricular materials, according to Idaho’s six-year adoption cycle. Materials are screened for content, organization, presentation and quality. Instructional materials are approved by the State Board of Education and the choice of the instructional materials in subject areas is maintained by the districts.

Indiana’s Office of eLearning supports the #GoOpen movement by leveraging the expertise of classroom teachers. The Rockstars of Curation are vetting quality digital content and curating digital lessons and learning activities to support students and teachers across the state.

Content Areas
Indiana reviews English language arts, science, math and social studies instructional materials for K-12.

Team Selection
The Rockstars of Curation are a cohort of educators made up of K-12 ELA, science, social studies, and math teachers along with K-12 media specialists. The Rockstars join the cohort by applying online to the Office of eLearning. The applicants are asked to describe their digital learning experience and implementation practices. Additionally, they are asked to share previously curated digital instructional materials and secure administrative support.

Compensation/Benefits
#INeLearn budgets $25,000 for cohort work and designates team members to lead the work. Along with professional development opportunities and virtual work days, the Rockstars are also given opportunities to present at workshop and conferences throughout Indiana during the school year and over the summer. Rockstars enjoy the opportunity to network across the state and to collaborate with educators outside their district. Because the Rockstars are a celebrated cohort of professionals they are able to share their expertise with those just starting down the digital learning path. This supports a culture of continuous growth across the state and elevates our teachers to receive the recognition they deserve!

Process and Training
Once selected the Rockstars meet throughout the school year where they receive professional development on various digital learning topics and spend time collaboratively vetting and curating digital materials. The Rockstars are also given virtual work days throughout the school year where they are able to dedicate time to vetting and curating digital content to use in their own classrooms and share out across the state.

Tools
The Rockstars are encouraged to use whatever tools best meet the needs of their students. Online platforms such as district adopted learning management systems, Symbaloo, Blendspace, and Google Classroom are used fulfill the “workflow” component of sharing digital instructional materials with students. Additionally, the Rockstars are encouraged to use free, digital tools that promote student inquiry and creation such as Actively Learn, Screencastify, Sketch Up, and Animoto. The Rockstars also promote and utilize digital formative assessments such as Google Forms, Quizziz, Kahoot, and Formative for quick student feedback.

Platform
In 2015, Indiana formed a partnership with Amazon Inspire to house their digital content repository. Amazon Inspire is still in the testing phase and when it is fully operational, it will offer all Indiana teachers a free place to go to find quality, vetted resources by keyword or Indiana standard.

The Louisiana Department of Education leads the reviews of instructional materials to determine the degree of alignment with state content standards. Local school systems determine whether these instructional materials are appropriate to meet the educational needs of their students. Districts can acquire instructional materials with or without the benefit of these state reviews. Local school systems have three options when adopting textbooks and other instructional materials:

• Select materials evaluated through the state review process, which involves educators and parents;
• Engage in a local review process of materials not reviewed by the state; or
• Adopt a combination of state-reviewed materials and locally reviewed materials

Content Areas
The ongoing online instructional materials review focuses on materials in:

• Early childhood education
• K-12 social studies full courses,
• K-12 English language arts full courses and benchmark assessments, and
• K-12 math full courses and benchmark assessments.

Team Selection and Vetting
Materials are evaluated by a committee of Louisiana educators selected through an annual application process. Any current Louisiana public school educator can apply to be a part of a review committee as a Teacher Leader Advisor. Teacher Leader Advisors play an essential role in ensuring that educators across the state have access to high-quality curricula, on-going assessments and professional development.

Process and Training
Louisiana has a review process specific to digital instructional materials, including OER or proprietary. Louisiana’s Online Instructional Materials Reviews website provides information and guidance for the review of instructional materials. Teacher Leader Advisors reviews materials for alignment to the state academic standards using a rubric. Reviews are tiered based on their quality or alignment to state academic standards:

• Tier 1 materials exemplify quality and meet all criteria on the review rubric.
• Tier 2 materials are approaching quality. They meet the non-negotiable criteria on the review rubric.
• Tier 3 materials do not represent quality. They do not meet the non-negotiable criteria on the review rubric.

Publishers can submit materials for review each year, and have access to the review process, rubrics, and guidance, which is posted on the Online Instructional Materials Reviews website. Louisiana also publishes a weekly report to provide districts with information about timelines –when reviews will be posted and what is scheduled to be reviewed. Local school systems can use this information to make decisions regarding their local timelines and what might need to be reviewed locally. Reviews are sent to the publisher for their comments, which are posted with each state review. All materials are also made available for review by the public.

For OER, Louisiana developed materials for PreK-12 English Language Arts, mathematics, and administrative guidebooks. The lessons and unit plans are posted on the website in the Teacher Support Toolbox and are free to use in most instances. However, Louisiana does not use Creative Commons licensing for these materials.

Tools
Louisiana’s Online Instructional Materials Reviews Resources includes Guidance for Textbooks and Instructional Materials Reviews, rubrics by subject area, assurance of accessibility standards checklist and other helpful resources.

Platform
Louisiana maintains the list of reviewed instructional materials and resources on their website – Curricular Resources and Annotated Reviews. The reviewed instructional materials are organized by content area in three tiers. Tier 1 exemplifies quality; Tier 2 approaching quality; and Tier 3 not representing quality.

Maryland Office of Instructional Technology oversees the evaluation processes and the database for approved credit bearing student courses. In 2012, three legal requirements were passed.

• COMAR 13A.04.15 Digital Learning: requires that all student online courses align to Web Content Accessibility Guidelines 2.0 Level AA
• Senate Bill 461: requires that a review of all online courses include an assessment regarding accessibility for individuals with disabilities
• Senate Bill 674: requires that all credit bearing student and professional development courses must be evaluated, approved, and listed in the approved Maryland State Department of Education (MSDE) database prior to being available for student use

Content Areas
Online courses

Team Selection
All reviewers must have a minimum of 4 years of experience teaching the content in a classroom setting. MSDE maintains a dynamic database of qualified content reviewers. Reviewers from the approved third party or local school systems must be certified in the related course content. If a district requests a review, MSDE provides training to reviewers, as well as instructional design and content based rubrics for online courses.

Review Process
Individual vendors may submit courses for review or districts may request a review. Each content review requires at least three content expert educators and one educator trained in the review process. MSDE provides training to all content specialists. There are three options for reviewing the content of credit-bearing courses (click to access more detailed information):

1. Review by the district
2. Review by the state
3. Approved third party reviewer

For vendor submitted courses, MSDE provides written content and accessibility feedback to the course provider for each course reviewed. If a submitted course does not meet content requirements, the publisher has a year to revise and re-submit it at no cost. If a submitted course is approved for content but does not meet accessibility requirements, the publisher must develop and submit a timeline (up-to-5 years) that indicates when the requirements will be met. Courses must be submitted on an annual basis to ensure that timeline modifications have been met. If the course is not re-submitted within one year or the modifications have not been met, the course is removed from the approved list.

Following the content review, MSDE includes approved courses in their database. As with all instructional materials, local school systems have the right to determine which of the approved online courses will be offered to students.

Compensation/Benefits

• MSDE reviewers are compensated $350.00 ($35.00/ hour for a maximum of 10 hours).
• If the review is conducted by a local system’s central office, the local school system determines compensation.
• Approved third party reviewing entity contracts with independent reviewers and determines compensation.

Tools
Content rubric based on iNACOL online course standards for students. Content rubric based on iNACOL online course standards and Learning Forward standards for educators. Accessibility reviews are based on the Web Content Accessibility Guidelines 2.0 Level AA.

Platform
Vendor/local school systems courses are delivered on their own platform. MSDE courses are delivered via Blackboard.

 Tip
Successfully facilitating an online course is different than teaching a face-to-face course. As a result, MSDE requires that all online facilitators meet the following requirements:

• participate in an online course,
• successfully complete an online facilitator professional learning course, and
• shadow an experienced facilitator for one semester.

The MI Open Book Project is a multi-year initiative funded as part of the Technology Readiness Infrastructure Grant (TRIG) which empowers master teachers to come together, collaborate, and develop open educational resources for use in classrooms around Michigan.

Content Areas
During the 2015-16 school year, Michigan developed the following books: 3rd Grade Michigan History; 7th Grade Ancient World History; 8th Grade United States History: Revolution through Reconstruction 9th Grade United States History:Reconstruction through Today; High School Civics. During the 2016-17 school year, Michigan will develop the following books will be developed: Kindergarten; 1st Grade Families and Schools; 2nd Grade Communities; and High School World History.

Team Selection
Michigan selects master teachers to collaborate and work together to develop the open textbooks. Teachers come from all other the state and include high school, upper elementary, and elementary school teachers.

Process and Training
Professionals from the field of Social Studies who include ISD consultants, university faculty, and district curriculum leaders edit and maintain the digital textbooks. All of these reviewers have extensive background knowledge in the content area. After the teacher teams complete the development of the textbook, the editors spend two to three months editing the books. In addition, the books undergo a content review by some of the state content organizations. During the summer, the books are assembled and distributed in mid-August.

Compensation
Teachers were paid a stipend of $3,500. When bringing teachers together we provided a hotel room the night before each session.

Platform
Michigan posts the digital books on the Michigan Open Textbook Project website. Michigan provides the source files (iBooks Author and InDesign formats) so that teachers can download the digital books and add their own contents. All books run on iOS, OSX, Android, Windows, and Chrome.

Quote
It is a true collaboration of k-12 and higher education.

The Instructional Materials Adoption Process in Utah maintains local autonomy in the choice of instructional materials by providing a general list of approved materials. The adoption process also establishes, by contracting with each publisher, a price for instructional materials to be maintained throughout the adoption period, thus providing equality in cost for each school district. This system provides valuable information to all education agencies about the best instructional materials at the lowest price for Utah’s students. While schools and districts are not required to select materials from this list, most, if not all, find it a great aid in finding worthwhile resources for instruction and learning.

Content Areas
Utah reviews instructional materials for core academic areas as well as applied technology areas. Review areas may include and are not limited to: bilingual education/ESL; business; educational technology; general financial literacy; fine arts; and special education.

Team Selection

The Utah State Instructional Materials Commission, in conjunction with the Instructional Materials Center review instructional materials and provide an approved list at Recommended Instructional Materials System (RIMS). The Utah State Board of Education appoints the members of the Utah State Instructional Materials Commission. Members serve a four-year term and include:

• State superintendent of public instruction, or the superintendent’s designee;
• Dean of the college of education of one of the state owned schools on a rotating basis;
• One school district superintendent;
• One secondary school principal;
• One secondary school teacher;
• One elementary school principal;
• One elementary school teacher;
• Five persons not employed in public education.

The Utah State Instructional Materials Commission responsibilities are:

• Review instructional materials and make recommendations to the Utah State Board of Education.
• Accept proposals (bids) from publishers for instructional materials for state adoption.
• Appoint Instructional Materials Advisory Committees to evaluate instructional materials.
• Recommend policies for instructional materials adoption procedures to the State Board of Education.
• Review requests for renegotiation of instructional materials contracts and requests for instructional materials to be removed from contract.

Process and Training
Utah officially announces upcoming instructional materials reviews in June and December. Publishers submit a formal bid to the Utah State Instructional Materials Commission. Samples and titles of instructional materials are sent to each commissioner and three copies of each title are also sent to the Instructional Materials Center. These materials are evaluated by the Instructional Materials Advisory Committees on the basis of their alignment to standards, accessibility to all students, and appropriate pedagogy for ideal teaching and learning. Publishers seeking the highest recommendations (recommended primary) are required by law to submit correlation documents from independent reviewers that assure alignment to standards. The Instructional Materials Advisory Committees submit recommendations to the Utah State Instructional Materials Commission during their semiannual adoption meetings. The recommendations are then sent to the State Board of Education for final approval. Once approval is granted, publishers are required to send verification that materials meet National Instructional Materials Accessibility Standard (NIMAS). Utah’s rating-recommendation criteria for instructional materials are:

• Recommended Primary
• Recommended Limited
• Recommended Teacher Resource
• Recommended Student Resource
• Reviewed, But Not Recommended
• Not Sampled
• Not Reviewed

Resource: A sample timeline for the instructional materials vetting process.

Tools
Curriculum specialists in the Utah State Office of Education, in cooperation with subject area specialists in districts and schools develop rubrics for the review of instructional materials. The major focus of the rubrics is the core content for the course, but the rubrics also address issues such as equity, material quality and construction, adherence to Utah law, and other issues. The rubrics are designed to help reviewers evaluate instructional materials and guide the narratives for RIMS. Visit the Instructional Materials Center website to access the rubrics

Platform
Utah hosts a searchable database for Recommended Instructional Materials System (RIMS). Educators can search by subject area, publishers, or ISBN number. The Instructional Materials Adoption Procedures provides a listing of the best available instructional materials and eliminates those that are undesirable or of inferior quality. All educational patrons are encouraged to use this list to make preliminary decisions about curriculum to be used in the schools. “Instructional materials” mean textbooks or materials used as, or in place of, textbooks and which may be used within the state curriculum framework for courses of study by students in public schools to include: textbooks, workbooks, computer software, online or Internet courses, cd’s and dvd’s, and multiple forms of communication media. These materials shall be designed for student use, may be accompanied by or contain teaching guides and study helps, and shall appear on the online, searchable database (RIMS) of state-recommended instructional materials. Instructional materials are considered for all curriculum content areas once a year.

Washington
In Washington, the procurement of instructional materials is primarily conducted at the district level and there are not state laws restricting the selection and purchase of digital instructional materials. However, state law requires school districts to have policies and procedures in place to guide the selection and adoption of instructional materials.  Washington periodically conducts formal instructional materials reviews in various subjects as a service to school districts. The results of these reviews are meant as resources for school districts undergoing their own local review and/or adoption processes.

Additionally, in 2012, the state Legislature passed a bill to develop an OER Library of high-quality, openly licensed K-12 courseware that is aligned with the Common Core State Standards (CCSS) and reviewed by the state.

Team Selection
In Washington, superintendents and district office administrators are responsible for curriculum development and the instructional materials selection process. In larger districts, the policies are often established by district administrators and principals after discussion with content-area specialists and teachers. In smaller districts, the administrator or principal may by assisted by a classroom teacher interested in the specific content area. Washington recommends appointing a facilitator to: determine selection process/criteria for review committee members; identify review committee members and define roles; plan and facilitate all meetings; and implement a communication plan informing district teachers, administrators, parents and school board. Depending upon the district demographics, the review committee may include some or all of the following members:

• Classroom teachers
• Representatives from all schools involved in the district
• District Office Administrators and/or principal
• Representatives from key support roles i.e., special education, Title I, ELL,
• Librarian and other staff
• Parents and Community Members
• Education Association Representative
• Students (high school level)
• Experienced and beginning teachers
• Gender and ethnic diversity

Process and Training
Washington does not have a formal review process for instructional materials, although the state provides guidance for conducting instructional materials review and resources for reviewing existing instructional materials. As part of the guidance, Washington identifies the responsibilities of the review committee:

• Study the research and best practices for the content area.
• Review student data including demographic and achievement data.
• Study the Washington State Instructional Materials Review from OSPI, both criteria for review and outcome of state review.
• Gather information from teachers and other staff members using interviews, surveys or other methods.
• Examine the EALRs and GLEs and district curriculum materials.
• Establish clear definitions of significant terms.
• Review time allocations for content area.
• Determine method for decision-making; a consensus model is recommended

Washington conducted reviews of OER in high school mathematics and English Language Arts (ELA). Though this review focused on OER, the actual review process and tools are not specific to OER. They are designed to be used with any instructional material under consideration by school districts. As of August 2015, the Office of Public Instruction (OPSI) has conducted three OER review cycles. To date, 24 full-course mathematics curricula and 60 English language arts units have been evaluated by educators across the state. OPSI reviewed these open resources using nationally recognized review instruments and the same methodology as you would review any traditional instructional materials. In the 2015 OER Review Summary, reviewers found a number of resources in both mathematics and ELA that were well-aligned to the CCSS, and worthy of consideration by districts choosing instructional materials. OPSI provides this information to districts, but it does not endorse or require the adoption or use of any specific instructional materials by districts or schools.

Tools
The state recommends several instructional materials tools including EQuIP, IMET, and the Achieve OER Rubrics for use with ALL instructional materials, regardless of delivery platform. Washington also uses a rubric for Social Studies review to measure the alignment and overall quality of lessons and units to the Washington State Social Studies Learning Standards and the Washington ELA and Literacy in History/Social Studies Standards. This rubric also evaluates lessons and units for integration with the College, Career, and Civic Life Framework. Washington also developed a Technology Rubric to assess whether the instructional materials work with existing devices, require internet access, comply with accessibility requirements for students with special needs, as well as other considerations.

Platform
Washington hosts the OER library on its website and provides school districts with a broader selection of materials that can be updated more frequently than traditional textbooks.

Content Area
English, science, OER materials

Team Selection
Reviewers are selected by department chairs and principals. Reviewers must have taught the subject for at least three years.

Compensation/Benefits for Reviewers
Each subject-area lead is paid a $6,000 stipend. Other committee members are paid $500 honorarium to attend meetings. Work outside the contract is paid at $35 an hour.

Review Process
1. Collect potential resources
2. Calibrate rubric
3. Evaluate resources in small groups with rubric
4. Discuss each resource as a group
5. Assemble resources

Tools
Grossmont uses rubrics that are based on the Achieve Rubric and differentiate between core content and supplemental materials. The reviewers review the rubrics, and as a group, calibrate the rubric to ensure common interpretation. General rubric and Science specific rubric.

Platform
Grossmont is still determining best platform. Currently, OER materials are at: tech.guhsd.net/oer

Content Areas
K – 5 ELA (English Language Arts)
6 – 12 ELA (English Language Arts)
K – 5 Mathematics
6 – 12 Mathematics
K – 12 Science
K – 12 Social Studies

Team Selection
Lawrence Public Schools selects a teacher to act as Curriculum Facilitator to provide leadership and coordination in ongoing development, implementation, and support of district curriculum and instruction.

Compensation/Benefits for Reviewers
The Curriculum Facilitator receives 20 days of additional pay for this work. Lawrence compensates other staff for writing course masters.

Review Process
The Curriculum Facilitator:

• Provides, seeks, and creates quality curriculum and instruction resources, including electronic format.
• Curates and organizes resources.
• Ensures that all programs and content areas under the direction of this position have access to evidence-based instructional practices and resources.
• Participates in curriculum budget discussions.
• Provides leadership in determining program effectiveness in specific content focus areas.
• Observes and models effective instruction with classroom teachers

Tools
Lawrence uses the Comprehensive Blended Learning Course Master Essential Components Rubric to evaluate instructional materials. Key components of the rubric include:

• Establish clear learning goals
• Plan intentionally for all learner variability
• Utilize flexible methods and materials

Platform
Lawrence uses Amazon Hosting and Blackboard for approved materials.