Standardized Ohio tests lack validity, YSU study shows

The Ohio Department of Education

disagrees with the education professor’s findings.

By HAROLD GWIN

VINDICATOR EDUCATION WRITER

YOUNGSTOWN — Studies by a Youngstown State University education professor show that Ohio’s achievement tests are a better measure of a child’s socioeconomic living conditions than what they’ve learned in school.

The “lived experiences” of the pupils are what really determine how they do on such tests, and the result is the tests “have no academic validity nor educational accountability validity whatsoever,” Dr. Randy L. Hoover wrote in his study.

Hoover, who teaches in the Department of Teacher Education at YSU, did an examination of Ohio Achievement Tests in 593 Ohio public schools in 2000 and did a follow-up study covering 609 of the 611 public districts in the state this year.

The results were the same, he said: Pupil performance on the tests was most significantly affected by the nonschool variables found in their socioeconomic living conditions, not in-school variables such as class size or per-pupil expenditures.

Hoover’s most recent study covered achievement tests given to third-, fourth-, fifth-, sixth-, seventh- and eighth-graders as well as the Ohio Graduation Test and found that results could best be predicted by looking at district median family income, percentage of economically disadvantaged children in a district and the percentage of single-parent wage earners.

He used state and federal data to secure that base information and then compared it to district test scores.

The correlation showed that test results “are vastly more indicative of the out-of-school, lived experience of the students rather than indicative of academics,” he said, noting that no in-school variables showed the same correlation.

His findings apply to the achievement tests and the OGT, he said.

Children from wealthier districts are inclined to have a much broader exposure to the world and more of those lived experiences — and therefore do better on the tests, according to Hoover.

“Ohio’s current accountability system perpetuates the political fiction that poor children can’t learn and teachers in schools with poor children can’t teach,” Hoover said.

When the tests are standardized, they are normed on particular language use, vocabulary, values, socioeconomic perspectives and life experiences, he said. Too often, these norms are more or less alien to population groups outside the upper-class socioeconomic group upon whom the tests are most commonly normed, he said.

Ohio’s system is based on the belief that all children are the same when they come to school, and that’s wrong, he said.

The lived experience is extremely varied and often very diverse across families, wealth, individual differences, lifestyles and enrichment, he said.

He suggests that the state impose a moratorium on testing until a more valid examination system can be devised, suggesting that multiple assessments are needed to make results valid, not a single test.

A better method might be a series of tests or requirements for pupils to demonstrate an ability to use what they’ve learned, he said, citing a driver’s examination as an example.

Just taking the written portion of the driver’s test doesn’t get you a license. You must demonstrate that you can actually drive a car, he said.

Department response

The Ohio Department of Education doesn’t support Hoover’s findings and stands by its tests, said Karla Warren, ODE press secretary.

“Our tests undergo a detailed review process. All test questions are reviewed by a committee comprised of Ohio parents, community members and teachers,” Warren said. “In addition to this review, a fairness and sensitivity committee examines test questions. Members of this committee are selected to represent the cultural diversity within the state,” she said.

Ohio has high-poverty schools that are also high-performing, Warren pointed out.

“We call these schools ‘Schools of Promise’ and have shared their best practices with schools around the state,” she said.

The analysis of the data shows the test performance results are equally and consistently invalid regardless of whether the districts are performing poorly or well, said Hoover, adding that he isn’t surprised by ODE’s reaction.

He got a response from ODE on the 2000 study that he said was “blatant animosity,” noting that ODE brought in statisticians in an unsuccessful effort to refute his findings.

He said he’s given a copy of his latest study to Gov. Ted Strickland, who has made education a high priority of his administration, and has hopes the governor will take some action.

Hoover did briefly address the issue when Strickland visited YSU in September as pat of his Conversation on Education tour of the state.

Other teachers in the crowd also took some exception to the testing, prompting the governor to suggest that a change in the OGT might be considered.

“We don’t want to lessen accountability,” Strickland said at the time, but added that, if the current process is taking the joy out of education for both teachers and pupils, “That is a problem.”

Hoover said the state is holding educators accountable for results on the achievement and OGT tests and that isn’t fair.

If they are to be held accountable, it needs to be done on the things over which they have control, and the lived experience of kids isn’t one of them, he said.

Teachers in agreement

ODE may take no stock in Hoover’s findings, but some local educators say there is truth in his study.

The results are somewhat predictable as Hoover shows in his study, said Richard Denamen, superintendent of the Mahoning County Educational Service Center, but there still needs to be some method for districts to look at the test data with the overall goal of improving.

Do some teachers “teach to the test?” Yes, Denamen said, but pointed out that a coach coaches to win and that’s what teachers do too.

Still, just teaching to the state standards really narrows education and limits the depth one can go on a particular subject, he said.

Harold Wilson, an Austintown High School teacher, said he believes that life experience is a significant factor in a child’s performance results, just as Hoover says.

It’s related to what your family gives you in terms of education exposure and encouragement, he said, noting that economics are part of it.

Anderson said he sees some big flaws in the testing, suggesting it’s more about competition than it is about the kids.

“There’s truth in his study,” said Colleen Ruggieri, a National Board Certified teacher and college prep and senior honors English teacher at Canfield High School.

Pencil and paper tests are not always the best assessment tools, she said, pointing out that it is difficult to get an assessment equitable to all learners. The tests don’t always give an accurate assessment of student knowledge, she said.

Education reformers need to broaden the way they look at student assessment, Ruggieri said, suggesting that oral testing and project building could be incorporated in the process.

A pupil’s background is very important in relation to test performance, said Jennifer Colosimo, a middle school gifted teacher from Struthers.

Some kids’ families are able to provide computers, and some are able to take them places that expose them to the world.

Ohio’s current system of testing “stifles the way we have to teach,” she said, explaining that she can’t always go into the depth on a subject that she feels her pupils should have.

Terry Murcko, a Liberty High School English teacher with 32 years in the field, said he believes Ohio’s testing system squelches both critical and creative thinking.

Teachers wind up being under the pressure of pupil scores and might neglect the importance of developing thinkers, he said.

As Hoover says, the tests favor certain socioeconomic backgrounds, Murcko said.

gwin@vindy.com