A Word From the President

Meeting a Dual Obligation

Protecting the legal rights of individuals with mental or physical disabilities has been a topic of increasing attention in our society, and deservedly so. The last U.S. Census counted almost 50 million people with some type of long-lasting, disabling condition; that's 19 percent of US residents 5 years of age and older. Of those individuals, 12 million were listed as having a physical, mental, or emotional condition causing difficulty in learning, remembering, or concentrating. Healthcare professionals are, of course, well aware of how these disabilities can turn seemingly simple tasks into significant hurdles.

Federal law requires that no otherwise qualified student - including those with learning disabilities (LD) and Attention Deficit/Hyperactivity Disorder (ADHD) - be denied access to higher education programs by reason of his or her disability. The law also requires that educational institutions provide reasonable and appropriate accommodations for such students. In accordance with federal requirements, many testing services, including our own MCAT, accommodate students with certain documented disabilities by permitting extra testing time, providing separate rooms, or making other appropriate arrangements.

To ensure that the test scores reported to schools have psychometric integrity, the MCAT annotates, or "flags," the scores obtained when tests are taken under certain non-standard conditions. Many types of accommodations are not flagged, because they do not fundamentally alter the testing conditions. The only accommodations that MCAT flags are those that render the test results less reliable as predictors of a student's performance than non-flagged scores are; those accommodations are (1) extra time and (2) the use of tools not available to the standard test taker, such as a computer or a reader (e.g., for the visually impaired). Fewer than 1 percent of all scores are obtained under these conditions and warrant flagging.

The "flag" is designed to alert admissions committees to rely more heavily on other sources of information when making admissions decisions about students whose MCAT scores are so marked. But a number of disability rights groups contend that flagging should be discontinued because, in their view, it discourages students with disabilities from requesting special accommodations, stigmatizes those that do make such requests, and lessens the chance of admission to medical school.

These considerations, coupled with the legal challenge in 1999 of the flagging practices of the Educational Testing Service, convinced some (but not all) testing agencies to stop flagging their scores. The question raised by all this is whether the MCAT exam should continue its flagging policies. To help answer that question, we undertook a series of research projects over the past year. The first examined the impact of flagged accommodations on test scores; the second evaluated whether flagged scores predicted subsequent performance in medical school as well as did scores obtained under standard conditions; and the third looked at the impact of flagging on the probability of admission.

We concluded from the first two sets of studies that test scores obtained under nonstandard conditions are not comparable to those obtained under standard conditions. That conclusion was based on the following findings: a) flagged scores are, on average, higher than non- flagged scores; b) examinees who (i) took the test first under standard conditions, (ii) later applied for and received accommodation, and (iii) took the test again but with extra time, increased their scores, on average, more than four times as much as does the typical, non-accommodated test repeater; c) in an experimental study, standard examinees who were given extra time without forewarning improved their scores significantly in the two investigated sections of the examination (Verbal Reasoning and Physical Sciences); and d) examinees given extra time on the MCAT test performed less well on USMLE Step 1 and were more likely to encounter academic difficulty than non-accommodated students with comparable MCAT scores and undergraduate grades.

The third set of studies failed to confirm the view of some disability advocacy groups that examinees with flagged scores are discriminated against in the admissions process. In fact, taking both MCAT scores and undergraduate GPAs into account, we found that applicants with flagged MCAT scores were accepted into medical school at slightly higher rates than those whose scores were not flagged. This observation suggests that applicants who received MCAT accommodations impress admissions committees with attractive qualities not typical of other applicants.

The findings from all of these studies have convinced us that MCAT should continue to flag scores obtained under certain nonstandard testing conditions. Doing so is the only way we can meet our dual obligation to accommodate disabled test takers in accordance with federal law while assuring schools that the scores we report are valid estimates of their applicants' preparedness for medical school.

Jordan J. Cohen, M.D.
AAMC President