Rankings scrutinized as fifth college this year comes clean

Bucknell University on Friday became the fifth college in a year to admit to submitting false admissions data to U.S. News & World Report, renewing scrutiny over college rankings and how those figures are reported.

The Pennsylvania college skewed its average SAT score by leaving off a few dozen students’ scores, Bucknell President John Bravman announced, which boosted the overall mean by up to 25 points. Bravman called the omission “relatively small” but serious. Higher education experts say the streak of confessions, including GW’s in November, has raised doubt about the reliability of college rankings.

Since last April, GW, Claremont McKenna College, Emory University and the Tulane University Freeman School of Business have also disclosed that they misreported admissions data.

Lynn O’Shaughnessy, a higher education writer, said after GW disclosed its misreporting in November, she predicted more to come forward.

“Other schools have been caught doing this. Who knows how many schools are doing this? I think a lot,” she said.

Part of the problem stems from universities’ decentralized and inconsistent reporting systems, Chronicle of Higher Education senior writer Eric Hoover said.

At many institutions, several departments calculate the same numbers but churn out different results.

“It’s just not a very evolved reporting system,” Hoover said. And when those figures are reported to U.S. News, he said, “there’s no data cop as far as these numbers go.”

Like any data-compiling institution, the publication is limited by personnel and time constraints, Hoover said.

Scott Jaschik, an editor at the online news site Inside Higher Ed, said U.S. News relies on the colleges to submit tallies of its enrollment data, and it’s unlikely the publication will change its policy and begin asking for raw data instead.

“It would cost them a lot – they would have to hire a lot of people to handle a lot of data,” Jaschik said.

But U.S. News & World Report doesn’t see a trend in misreporting. Following Tulane’s misreporting, the publication’s Director of Data Research Robert Morse described the incidents as “isolated cases” in a blog post.

“We have no reason to believe that other schools have misreported data,” Morse wrote.

Admissions directors tend to disagree. Ninety-one percent of admissions directors said they believed more universities were guilty of inflating data, according to an October 2012 Inside HigherEd survey – after two instances of misreporting.

In an email, Morse declined to comment on any trends in false data reports or Bucknell University’s recent disclosure.

“I am not doing interviews at this time,” he said in an email this week. He has refused interviews since November.

The rash of inaccurate data may be triggering a domino effect, Provost Steven Lerman said.

He said the problem of misreporting is not “better or worse” among colleges now than it was in the past.

“When we identify or others identify mistakes,” Lerman said, “it probably causes others to call up their people and say, ‘Let’s take look at the numbers.’ ”

Mike Reilly, executive director of the American Association of Collegiate Registrars and Admissions Officers, said that more universities are verifying their data after the recent instances of misreporting. He said that more universities would likely discover inaccurate data.

Some schools are now calling on their institutional research offices to supervise admissions data calculations, which could stem the problem, Jaschik said.

In contrast to a university’s admissions department, institutional research doesn’t face the pressure of meeting specific enrollment goals.

At Emory University, officials admitted to knowingly inflating admissions data, and an independent investigation implicated the university’s director of institutional research.

After GW came clean on its misreporting, University President Steven Knapp maintained that it was inadvertent.

Senior Vice Provost for Academic Affairs and Planning Forrest Maltzman also announced that the college’s academic assessment office would take over the data reporting and that outside firms will audit admissions data more regularly.

“We’ve taken away from [the admissions office] the responsibility for recording the data,” Maltzman said in November, after disclosing the error. “It was a bad system. It’s bad across administrative units to have the same unit responsible for bringing in the class and then telling us about the class. It doesn’t make sense.”

The audit firm Baker Tilly checked the most recent year of GW’s data after Maltzman first noticed a discrepancy in data this summer. While Knapp and Maltzman have said the firm found no other errors, both declined to provide a written audit report, saying it was delivered orally at a Board of Trustees meeting.

Besides GW, all other schools revealed that at least one employee knowingly inflated admissions data.

Reilly, a former enrollment manager, credited new leadership at the admissions level as a key factor in identifying faulty data.

Despite growing skepticism over the accuracy of rankings and their methodologies, which often focus on standardized testing and sometimes consider peer reviews, many prospective parents and students continue to focus on a school’s position.

“Ranking is not going away,” Reilly said. “I think it’s primarily because students and families value it.”

New York-based Iona College was caught in one of the most high-profile cases of U.S. News kicking a school off the rankings. It was tossed out of the top regional universities list two years ago after lying about test scores, graduation rates, freshman retention, student-faculty ratio, acceptance rates and alumni giving.

Sarah Ferris and Cory Weinberg contributed to this report.

The Hatchet has disabled comments on our website. Learn more.