TheEducation Assessment and Accountability Review Subcommittee met on Friday, October 6, 2006, at 1:00 PM, in Room 169 of the Capitol Annex. Senator Jack Westwood, Chair, called the meeting to order, and the secretary called the roll.
Guests: Clyde Caudill, Jefferson County Schools, Kentucky Associations of School Administrators; and Data Recognition Corporation.
LRC Staff: Sandy Deaton, Audrey Carr, Jonathan Lowe, Janet Stevens, and Lisa Moore.
The meeting began at 1:10 p.m. Representative Moberly made the motion to approve the minutes, and Representative Marzian seconded the motion. The motion was approved by voice vote.
Senator Westwood introduced Mr. Kevin Noland, Deputy Commissioner and General Counsel, Kentucky Department of Education (KDE) who explained administrative regulation 703 KAR 5:020 - The formula for determining school accountability. Mr. Noland said 703 KAR 5:020 & E establishes a single assessment system with two accountability dimensions: one is addressing the requirements of KRS 158.56455 to determine school classifications, and a second addressing the conditions necessary to conform to federal assessment and accountability requirements of the "No Child Left Behind Act of 2001 (NCLB)." The administrative regulation is being amended to make Adequate Yearly Progress (AYP) determinations for 2006 by averaging two years of data from the existing Kentucky Core Content Test grades in reading and mathematics for all schools and districts and subpopulations of sufficient size for the two years combined. This amendment is necessary to comply with the required plan for the implementation of "No Child Left Behind Act of 2001."
Senator Westwood asked for a motion to accept administrative regulation 703 KAR 5:020. Representative Moberly made the motion to accept the administrative regulation, and Representation Marzian seconded the motion. The motion was approved by voice vote. Senator Kelly asked for clarification of the motion, and Mr. Noland explained.
Senator Westwood introduced Mr. Gene Wilhoit, Commissioner, KDE, to make a Power Point presentation about Kentucky's implementation of the NCLB. He discussed the history of the NCLB Act, which was the 2001 re-authorization of the Elementary and Secondary Act of 1965. Kentucky is five years into implementation and still in the roll-out stage. Accountability measures required by the Act are comparable to those of Kentucky's school accountability and testing system. He said both NCLB and the Commonwealth Accountability Testing System (CATS) require school and district accountability for student performance; require annual, standards-based testing; and both require action by KDE, schools, and districts to overcome low performance.
Commissioner Wilhoit said prior to NCLB, Kentucky had already implemented several of the law's requirements including: high expectations for students; rigorous annual assessments tied to subject content and state standards; school rewards and consequences; school improvement plans; scholastic audits; highly skilled educators assigned to low-performing schools; assessment results disaggregated by student populations; a unified data collection and reporting system; and the goal of reaching proficiency by 2014. He said Kentucky, like many states, has modified and/or supplemented its student assessments to comply with NCLB. Kentucky now uses assessment results in a two-dimensional system to make both federal and state accountability decisions.
Commissioner Wilhoit discussed Kentucky's 2006 NCLB results. He said 766 schools met 100 percent of the NCLB AYP goals, which is 65.8 percent of all schools in Kentucky. He noted data was reported for all 176 school districts and 1,164 schools.
Commissioner Wilhoit said 88.9 percent of elementary schools, 31.9 percent of middle schools, and 22.7 percent of high schools met all their NCLB goals. Three hundred ninety-eight schools, or 34.2 percent, did not make AYP, but 241 of these schools made 80 percent or more of their goals, and 129 of these schools met at least 90 percent or more of their goals. Overall, 1,007 schools in the state, 86.5 percent, met 80 percent or more of their goals.
Commissioner Wilhoit said 80 of 176 school districts, 45.5 percent, met all of their AYP goals. Ninety-six school districts, 54.6 percent, did not make AYP, but 64 of these districts made 80 percent or more of their goals. Overall, 141 of 176 school districts, 81.8 percent, in the state met 80 percent or more of their goals.
Commissioner Wilhoit said statewide, 19 of 25 target goals, 76 percent, were met. All student groups met the requirements for participation rate. The most commonly missed goal among schools and districts was the performance of students with disabilities in reading.
Commissioner Wilhoit identified 156 schools in 92 districts receiving consequences. He noted the KDE provides: guidance on transfers and supplemental services; guidance on corrective actions; technical assistance from highly skilled educators; monitoring school and district improvement efforts; and voluntary assistance options for districts.
Commissioner Wilhoit said the Wellstone amendment offers flexibility by the United States Department of Education that allows states to average two years' worth of reading and math data to provide AYP information. The KBE approved adopting the Wellstone amendment for the reporting of 2006 NCLB data. He said under Wellstone, AYP was determined by using the percent of proficient and above data and participation data from the last two years of KCCT data in reading (grades four, seven, and ten), and mathematics (grades five, eight, and eleven.)
Senator Kelly asked Commissioner Wilhoit to explain the difference between the highly skilled educator program and the voluntary assistance program. Commissioner Wilhoit said the highly skilled educator program was the first designed program to move into the schools that were in assistance classifications. He said those individuals are local personnel on loan to the KDE, who go through intensive training, and then are placed within the schools as an external agent for a limited amount of time to try and help the school get turned around.
Commissioner Wilhoit said the voluntary assistance program pertains to the issue of what happens to the schools after the highly skilled educator leaves. It also deals with the issue of local ownership of the reform movement. It involves helping the entire school district to deal with the problem, and not just working with the teachers and principals, and is a contractual agreement between the school district and the KDE. Senator Kelly said that sounded like the audit. Commissioner Wilhoit said it builds upon the audit program, but the voluntary assistance program puts people in place that will stay with the school district for a period of time. Senator Kelly asked what type of people are involved in the voluntary assistance program, and Commissioner Wilhoit said it is a local board person who is exemplary in practice, and retired administrators who have proven their ability to move districts ahead that partner with the superintendent during the transition phase. He also said that the program has only been in place a year, so he is hesitant to label it a success, but he has received wonderful feedback about the program, and six of seven school districts saw dramatic improvements after implementation of the program.
Representative Draud said he has heard from a number of superintendents that the voluntary assistance program has been very beneficial to school districts. He asked if the idea was a sustained effort as a follow-up to the audit. Commissioner Wilhoit said the program has been so popular that schools that need assistance are now given the choice between the traditional intervention strategies and the voluntary assistance program, and 23 districts have opted for the voluntary assistance program.
Representative Moberly asked if all the options labeled in the handout on page eight were part of the voluntary assistance program. Commissioner Wilhoit said no, that once a school moves into the tier categories of assistance, there are a set of programmed intervention strategies that schools must do, and they get more extreme as the tier levels increase. Representative Moberly asked at what level the interventions become mandatory, and Commissioner Wilhoit said even at Tier 1, there are a set of strategies that must be completed by the school.
Representative Moberly asked if school districts can use the voluntary assistance program in addition to other KDE mandated interventions. Commissioner Wilhoit said yes, and it is a better approach to implement the voluntary assistance program before getting into an extreme situation or falling further behind. Representative Moberly asked if scholastic audits were a part of this, and Commissioner Wilhoit said yes, audits and reviews are the first level of data gathering and diagnosis.
Representative Moberly asked what percentage of schools in assistance are also required to take some action under the Commonwealth Assessment Testing System (CATS) accountability system. Commissioner Wilhoit said these are overlapping schools in some situations, but some schools may be in trouble in the top two tiers that may not be in trouble with CATS, and the reason this is possible is because using the same information, it would be possible for the school to be progressing overall, but have one subpopulation of students that are not moving ahead, which would trigger the NCLB sanctions.
Representative Moberly said these schools would also be eligible for money from the Commonwealth School Improvement Fund, but only if CATS consequences are apparent as well as NCLB. He asked if Commissioner Wilhoit thought there should be better coordination among assistance available between the two systems. Commissioner Wilhoit said the KDE is trying to do that, and staff are looking at the consequence grids to make sure that similar messages are getting sent to school districts.
Representative Moberly asked if the definition used for a child with a disability in NCLB is the same definition used under CATS. Commissioner Wilhoit said yes, there is alignment between federal and state definitions. Representative Moberly said it is a problem for children with different levels of disabilities to be labeled under one definition. Commissioner Wilhoit said all the disability categories need to be analyzed, and each disability cannot be treated the same. Representative Moberly said this is an issue that needs to be addressed, and he has voiced this concern for several years, but no progress seems to have been made in this area.
Senator Westwood asked about the high number of schools that are placed in Tier 3. Commissioner Wilhoit said it is a great concern and part of it is the timing of NCLB, and the trend needs to be watched over the next two or three years. He said if these schools end up dropping down into Tiers 4 and 5, there will a tremendous pressure on the schools and the KDE in having adequate resources to fully support the schools, which was one reason for creating the voluntary assistance program to have them improve dramatically before falling further behind.
Commissioner Wilhoit discussed Kentucky's 2006 ACT results. The ACT composite scores for Kentucky's college-bound high school seniors rose slightly in 2006. The 2006 composite was 20.6, a gain of 0.2 from 2005, when the composite was 20.4. Nationally, the 2006 composite scores was 21.1, a gain of 0.2 from 2005.
Commissioner Wilhoit said more Kentucky students took the ACT in 2006, and more took rigorous coursework, both trends since 1990. He said the ACT defines rigorous coursework as four or more years in English; three or more years of mathematics (including Algebra 1, Algebra 2 and geometry); three or more years of social studies; and three or more years of natural sciences.
Commissioner Wilhoit said the ACT's College Readiness Benchmarks indicate the probability that students who reach the levels will earn "C" grades or higher in certain credit-bearing first-year college courses. They are: a score of 18 or higher on the ACT English test; a score of 22 or higher on the ACT math test; a score of 21 or higher on the ACT reading test; and a score of 24 or higher on the ACT science test. He said among students in Kentucky: 67 percent met or surpassed the English benchmark, compared to 69 percent nationally; 34 percent met or surpassed the math benchmark, compared to 42 percent nationally; 50 percent met or surpassed the reading benchmark, compared to 53 percent nationally; and 23 percent met or surpassed the science benchmark, compared to 27 percent nationally.
Senator Westwood commented that the Council on Postsecondary Education (CPE) discussed using 19 as the math benchmark in Kentucky. Commissioner Wilhoit said the 19 is for a credit-bearing course, and Senator Westwood said the goal is to move the student to a credit-bearing course away from a remediation course. Commissioner Wilhoit said it is possible to have a credit-bearing course that is not Algebra, and the KDE is not satisfied with a 19 criteria when the Algebra criteria is a 21 or 22.
Senator Westwood asked about the CPE's benchmarks for the other subject matters. Commissioner Wilhoit said it is 18 for English, and English and math are the only two benchmarks they are using. Senator Westwood would like to have a discussion with the CPE about the 19 benchmark for the math criteria because the goal is for Kentucky students to succeed in college and to be prepared for high-paying jobs after college. Commissioner Wilhoit asked what profession would not require Algebra competencies, and Senator Westwood agreed and would like the figure looked at.
Representative Moberly asked why Kentucky is not seeing more improvements in college readiness with 5,000 more students taking more rigorous coursework. Commissioner Wilhoit said he would not expect to see more improvement initially, but actually would expect to see a drop in the schools at first because it not nearly enough people to make a difference yet.
Commissioner Wilhoit said African-American students in Kentucky who took the college-bound curriculum performed similarly to African-American students at the national level. He said at both the national and state levels, the gap between the performance of African-American and white students persisted.
Representative Draud asked if the 20.6 composite score for all students was an improvement in the 2006 ACT results. Commissioner Wilhoit said it was a dramatic improvement.
Commissioner Wilhoit discussed Kentucky's 2006 CATS results. He said slightly more than half of Kentucky's public schools met or exceeded their goals for the 2005-2006 accountability cycle of the CATS, and fewer than 45 are in the assistance categories. He said 591 schools met their goal; 533 schools are progressing; 14 schools are in Assistance Level 1; 14 schools are in Assistance Level 2; and 13 schools are in Assistance Level 3.
Commissioner Wilhoit said student performance in all subject levels and grade levels continue to improve, and student performance is categorized with four levels: novice, apprentice, proficient, and distinguished. The lowest percentages of novices by subject area were: science at the elementary level (7.36 percent); reading at the middle school level (7.26 percent); and writing at the high school level (14.11 percent). The highest percentages of proficient and above scores by subject area were: reading at the elementary and middle levels (69.97 and 63.06 percent, respectively); and practical living/vocational studies at the high school level (55.5 percent).
Commissioner Wilhoit said Kentucky is making better progress at the elementary level than at the middle and high school levels. He said reading is the highest performing area and this is good news since Kentucky has focused on reading the past couple of years as a high priority.
Commissioner Wilhoit said the overall goal of CATS is for all schools to reach 100, which is proficiency, by 2014. During the accountability cycle, 44 schools reached or exceeded 100 on their 2005-2006 combined indices, and 24 of those schools reached 100 for the first time. He also said schools are honored for reaching recognition points when their accountability indices pass 55, 66, 77, 88 and 100. He said 474 schools passed recognition points in this cycle, with 77 as the most common point.
Commissioner Wilhoit said 63 schools were designated Pace Setters, with indices ranging from 97.9 to 116.1. Pace Setters are the highest-scoring five percent of all schools that have reached the fourth recognition point (88) and met the dropout rate and novice reduction requirements.
Commissioner Wilhoit said school districts also are held accountable through CATS, with three classifications: 1) Exemplary Growth District; 2) Audit Level 1 District; and 3) Audit Level 2 District. He said 48 districts received classifications during this accountability cycle. He gave detailed explanations of each classification level which is located in the meeting folder in the Legislative Research Commission library.
Commissioner Wilhoit said another component of CATS is the Comprehensive Test of Basic Skills (CTBS), a nationally-normed test administered to third, sixth, and ninth graders each year. The CTBS results for 2006 show that Kentucky students are making slow but steady progress in reading, language arts, and mathematics.
Commissioner Wilhoit gave an update on the accountability policy changes. He said the Kentucky Board of Education (KBE) has met twice since the last Education Assessment and Accountability Review Subcommittee (EAARS) meeting, and Kentucky is on track with the weighting of math and language arts in the new accountability system. There have been no changes. However, there is a change in the high school piece, and he wants to receive feedback from the members of EAARS about the change to report back to the board. He said certain identified groups want the ACT to be given with certain conditions, but not be included as part of the calculus for accountability.
Senator Kelly said he appreciated hearing the board's response to Senate Bill 130 (2006). He said Commissioner Wilhoit received a favorable response from the EAARS subcommittee at a previous meeting when he indicated the direction that Kentucky would be taking, and he feels the new recommendations are a reversal from the original direction. He said the intention of implementing the ACT test was to reduce the layering of tests and the amount of time that students are tested. He also said it is important to align what is happening in the high schools to higher education. The Council on Postsecondary Education's recent report indicated that 53 to 54 percent of students, need remediation in at least one course upon entering postsecondary education, the same level as previous studies identified. He said Senate Bill 130 is a major policy initiative, supported by both chambers, to encourage more rigorous coursework, and to measure student knowledge through the ACT test. He said he wants to keep the focus on how the students will perform in college rather than worrying about how the test will perform. He said technical difficulties and concerns should not keep Kentucky from moving forward with this major policy initiative, as Kentucky is following the lead of other states who have already moved in this direction. He said Colorado and Illinois are seeing great results from having all students take the ACT test.
Senator Westwood said he is puzzled, because the statute for the Commonwealth Assessment Testing System (CATS) says the assessment program shall include a customized norm-referenced test (NRT) to measure, to the extent possible, the core content for assessment. He stated that the new recommendations state that a NRT should not be included in the high school index.
Commissioner Wilhoit said the first statement on the handout has to do with the administration of the NRT, which will occur, the issue is just how much weight it will represent. This explains why Kentucky could still utilize the ACT for a NRT, as required by Senate Bill 130, and not have a weight in the accountability system. He said Senate Bill 130 said the ACT would substitute for the NRT already in place. He said the big issues being expressed from the field were: 1) the concern of teachers having to teach more broadly than in the past; and 2) the purposes of the ACT and wondering if they could be applied to Kentucky's accountability system. He said the big issue is whether to count the ACT as a part of the NRT calculation.
Senator Westwood said that when he voted for Senate Bill 130 he thought the intention of the bill was to count the ACT as part of the NRT calculation. He said maybe the language was not stated clearly enough in the bill.
Representative Moberly asked Commissioner Wilhoit to clearly re-state the issue. Commissioner Wilhoit said he previously told the members that Senate Bill 130 would be incorporated into the accountability system in three ways: 1) merging of items; 2) part of the transition formula; and 3) the NRT percentage (previously five percent for the CTBS-McGraw Hill administration) and continue to use it as a part of the calculation at the high school. Commissioner Wilhoit said these three pieces were before the KBE, who expressed concerns about the broadness of the agenda, the pool between ACT and core content, and some of the issues around access to ACT, and test-taking patterns. The KBE suggested a compromise, that is different from what was previously discussed. He said the KBE is suggesting administering the ACT, and notifying the parents of the results, but it would not be included in the calculation that will add up to the total school accountability.
Representative Moberly asked if the inclusion of the ACT results was based upon the correlation with the core content, and if so, had a correlation study been done. Commissioner Wilhoit said it is being done. Commissioner Wilhoit said the issue is whether to use the total of the ACT as the NRT component.
Senator Winters asked specifically who was voicing the concerns from the field, and what initiated the move from five percent to zero percent. Commissioner Wilhoit said there was a comprehensive list of groups with concerns listed in the handout. He said one group specifically was concerned with the norm comparison; for example, if 100 percent of Kentucky students take the ACT compared to 50 percent of students taking it in another state. He said another group was concerned that students not be overly driven towards ACT preparation, and to keep focus on the state standards rather than on the test. Commissioner Wilhoit said most all concerns were technical in nature, and would have to be worked out.
Senator Winters said there is always resistance to crossing new frontiers. He is very positive that more students will go to college as a result of taking the ACT, and he feels good that duplicative testing will be eliminated.
Commissioner Wilhoit said he had earlier reported that the high school level had fewer technical difficulties than the other school levels because the assessments are secure and use changing forms every year against similar items, and the problems in the other areas are not apparent at the high school level. He reiterated that he board wants to hear what the EAARS members have to say.
Representative Draud said he wants the members of the KBE to be told that the intent of the General Assembly was to have the ACT calculated as part of the accountability cycle, and not just part of the testing system. Commissioner Wilhoit said it will be a part of accountability because results from aligned ACT items will be merged in with core content results, and in the transition it will not be all or nothing. Representative Draud said he is confused as to what the Commissioner is asking.
Commissioner Wilhoit clarified by asking members if they wanted the ACT to be counted in the high school formula as the NRT part counting a certain percentage. Representative Draud said that was the intent all along.
Representative Moberly asked if ACT in lieu of the NRT could be viewed as an add-on, such as the dropout rate. Commissioner Wilhoit said yes, if everything moves ahead, there are three areas that make up accountability: the academic index; the non-academic index; and the NRT. This proposal moves from three of those to two, but the ACT score still gets folded into the calculation of the academic index and will be a part of transition. His concern was that the General Assembly said that the ACT would replace the NRT, and although it was not written that it could not be eliminated from the calculation, he assumed that they meant that there would still be a NRT weight in the high school calculation. He said if there is going to be a NRT calculation, it will have to be the ACT because Senate Bill 130 says it will substitute for the prior NRT.
Representative Moberly asked if that was conditioned upon the analysis of how it correlated with the core content. Commissioner Wilhoit said not necessarily, the academic index was more of a correlation on the core content. Representative Moberly asked if Commissioner Wilhoit was proposing to include the ACT in a non-academic index. Commissioner Wilhoit said he originally proposed to include it in the third sector, which is the NRT. Representative Moberly asked what NRT Kentucky is using now, and Commissioner Wilhoit said the CTBS. Representative Moberly asked if CTBS aligned with the core content, and Commissioner Wilhoit said there was only 30 percent alignment. Representative Moberly asked if the ACT aligns more than 30 percent with the core content, and Commissioner Wilhoit said they are going to figure that out, but he thinks so.
Senator Kelly said Senate Bill 130 was a policy response to the lack of preparation of the students for college. Senate Bill 130 was meant to respond to that, but it was never intended to layer testing, just to integrate it. He said students have more at stake in the ACT because a larger percentage of students are taking it. He emphasized the importance of a quick return of information for diagnostic purposes without layering, as well as the ACT being valid at the student level. He said Kentucky needs an accurate measure of its students, and students should be pushed into more rigorous coursework.
Representative Moberly said substituting the ACT for the NRT calculation was not doing any damage to the alignment problem because the current NRT does not align, and counting the ACT only promotes the goals of Senate Bill 130. Commissioner Wilhoit said he wanted the members to be aware of the concerns being raised, and said there are issues that need to be addressed. For example, it is important for students to be compared to an equal cohort. It is not fair for some students who have prior preparation and resources to have an advantage over students who do not.
Representative Moberly asked what Commissioner Wilhoit meant by an equal cohort. Commissioner Wilhoit said it could be approached in two ways: compare Kentucky students to the national average percentage or to compare to a sub-cohort of another state that has 90 percent or more students taking the test.
Representative Moberly asked Commissioner Wilhoit for his opinion. Commissioner Wilhoit said there are a lot of technical difficulties to be worked out, but the most significant is the student taking the ACT as a sophomore and going into a cohort with a student who has not had the exam before. Representative Moberly said that would work itself out over time.
Senator Winters asked how many times the CTBS test is given in a student's school career. Commissioner Wilhoit said three times, with the latest time being in the 9th grade. He said the ACT would not be competing with CTBS, but only adds a new involvement at the high school level.
Commissioner Wilhoit said it is also a strength that the students are taking the EXPLORE assessment in the eighth grade and the PLAN assessment in the tenth grade. He said those assessments should be a powerful tool for schools if they utilize the results correctly, and should also help to drive local decisions. He thinks it is a better tool overall, even though it has more components, it is better than what is in place now.
Representative Draud asked if the real issue was a conflict between what the field wants and what the General Assembly wants, and Commissioner Wilhoit said yes.
Senator Kelly said that not all the groups identified in the handout are 100 percent opposed to the whole idea of Senate Bill 130, and he reiterated that the business community represented by the statewide Chamber of Commerce is very supportive of Senate Bill 130, as well as the Kentucky School Boards Association, and the Council on Postsecondary Education. Commissioner Wilhoit said he would take the responses back to the KBE.
Senator Westwood asked if Commissioner Wilhoit received any indication from the board that they were aware of the American Diploma Project report that stated that Kentucky's current high school assessment was basically written at a middle school level. He said Kentucky's students need more rigorous coursework to be fully prepared when entering postsecondary education.
Commissioner Wilhoit said a part of it will be the ability to make sure people are not chasing certain tests, and to remain focused on what Kentucky has said students should be able to know and do, and provide multiple measures to get there.
Representative Moberly said a little over 50 percent of the schools meeting their CATS goals is not satisfactory, and Commissioner Wilhoit agreed. Representative Moberly asked what Kentucky's most important priorities should be in order to improve this performance.
Commissioner Wilhoit said it is translating the vision of the legislature and the state into implementation at the local level. He said it comes down to the ability of the teachers to teach what the state has asked them to teach. He said sometimes positive interventions work better than the negative ones. An example of positive interventions include: good pre-school programs that turn around those learning deficits; solid reading and math programs at the elementary level, and continuing through middle and high school; and a major redesign of the high school curriculum.
Commissioner Wilhoit said there will be a push in the next few years from local school districts to reduce the standards, and would urge the General Assembly not to do so. He said the field staff does not have the capacity to reach the sub-populations that the state is asking them to reach, and to the levels that the state is asking them to achieve.
Representative Moberly discussed the Assistance to Schools Subcommittee that was established during this interim that highlight schools that are successfully moving their sub-populations along utilizing various strategies, enthusiasm, and outstanding leadership. He said he also hopes Kentucky will see the fruits of its efforts with the interventions in the next couple of years, as well as schools developing new attitudes that they can succeed. He also said Kentucky needs more resources in its schools and a lot of teachers are overworked, but there are many successes in place now across the state with the resources currently available.
Senator Kelly told Commissioner Wilhoit that he made a difference in Kentucky with his leadership, and he hopes that the Commissioner will put forth a good faith effort to carry out the intention of the General Assembly as he meets with the school board on the accountability issue. He said Kentucky has made progress, but there is a long way to go.
Senator Westwood said that the Office of Education Accountability (OEA) has sent a list of study issues out to the subcommittee members, and he would like feedback from members on preferences of issues. He said to communicate to the co-chairs, or to Ms. Marcia Seiler, Director, OEA, of any pressing preferences on what the subcommittee would like for the OEA to investigate over the next six to nine months.
With no further business before the committee, the meeting adjourned at 2:50 p.m.