Ten "Must-Know" Facts About Educational Testing
By W. James Popham
All parents, especially those with children in school, need to know about the educational tests their children take these days with ever-increasing frequency. Typically, those tests will not only have a profound impact on a child's progress in school, but will also make a big difference in what happens to a child when school is finished.
The potential impact of educational tests rose dramatically when, in 2002, President Bush signed into law the No Child Left Behind Act. This important federal law makes students' test performances the chief determiner in how schools (and teachers) are to be evaluated. Given the heightened significance of today's tests, parents should become aware of what makes such tests tick. Presented here are 10 "must-know" facts about educational tests; each is followed by a brief explanation and an action implication for parents.
Fact 1. Educational tests are much less accurate than most parents believe. Students' scores on educational tests (whether teacher-built classroom tests or statewide or nationally standardized tests) are usually reported in numbers, which sometimes even include a decimal point. Such numbers, unfortunately, often convey to parents a picture of measurement precision that is simply unwarranted. A child's test performance on a given day can be greatly influenced by both physical factors (such as the child's having a cold) and emotional factors (such as the child's having had an argument with family or friends). In addition, the tests themselves only sample a child's skills and knowledge, and this sampling is often far from sufficient.
Action Implication: Never place all that much confidence in the accuracy of your child's performance on a single test. Only when the results of several different tests are reasonably consistent should you attach substantial significance to your child's test performances.
Fact 2. Educational tests allow teachers to make inferences about a student's unseen skills or knowledge, but these inferences may or may not be valid. The skills and knowledge children possess can't be seen. When Sally correctly spells all 25 words on her teacher's spelling test, the teacher infers that Sally is a good speller. Similarly, when Jane misspells more than half of the test's 25 words, the teacher infers that Jane's spelling skills are weak and need to be improved. If, on the basis of a student's test performance, an accurate inference is made about the student's unseen abilities, measurement experts call this a valid inference. But people, not tests, make these inferences. And people have been known to make mistakes.
Action Implication: When your child's teacher makes a test-based inference about your child's covert skills and/or knowledge, do not automatically assume that the teacher's inference is valid. It probably will be, but it may not be.
Fact 3. High-stakes tests, depending on the particular way they are constructed, can have a decisively positive or negative effect on a child's education. When students' performances on a test have important consequences for students (such as grade-to-grade promotion or diploma-denial) or for those students' teachers (such as test-based evaluations of a school staff's effectiveness), the test is referred to as a "high-stakes test." If an inappropriate high-stakes test is used, its impact on schooling can be quite harmful. Such high-stakes testing may cause important but untested content to be eliminated from the curriculum. Also, students will often be forced to take dull, excessive test-preparation drills. Some teachers, because of pressures to raise their students' test scores, may also engage in dishonest test-preparation or test-administration practices.
On the other hand, if an appropriate high-stakes test is employed, it cannot only supply accurate evidence for school evaluations, but can also help improve the quality of teachers' instruction. Everything hinges on the appropriateness of the particular high-stakes test that is used.
Action Implication: If your child attends a school in which a high-stakes test is used, try to discover (by speaking with teachers, administrators, and other parents) what effect, if any, that test seems to have on instruction. If inappropriate high-stakes assessments appear to have a negative impact on your child's schooling, join with other parents to demand more instructionally beneficial high-stakes tests. If it turns out that the use of inappropriate high-stakes tests (or, indeed, any other unsound use of tests) is based not on a district decision, but on a state-level requirement, offer to help district officials transmit your concerns to relevant state authorities.
Fact 4. Although achievement tests are designed to assess a student's skills and knowledge, and aptitude tests to predict a student's success in a subsequent academic setting, it is sometimes difficult to tell which is which.
The Iowa Tests of Basic Skills, the Stanford Achievement Tests, and the California Achievement Tests are examples of widely used nationally standardized achievement tests. These tests are intended to measure significant skills or bodies of knowledge that children should learn. A standardized aptitude test, on the other hand, is intended to predict a student's subsequent academic performance. The SAT and ACT, for example, are aptitude tests administered to high school students to predict what grades those students will earn in college. The ACT and SAT are widely used as college admissions exams.
Although the purposes of aptitude and achievement tests are different, the actual items used on the two types are often similar.
Action Implication: Both aptitude and achievement tests can give you a relative picture of your child's current performance levels. But because aptitude tests play a significant role in determining which colleges your child will be able to attend, you must give careful attention to those tests. (See Fact 7.)
Fact 5. Traditionally constructed standardized achievement tests can provide parents and teachers with useful information about a student's relative performance levels. A standardized test is administered and scored in a standard, predetermined manner. For more than 75 years, almost all of America's standardized achievement tests have been based on a comparative approach to measurement. In this approach, one student's score is interpreted according to the scores of a representative collection of other students who, having taken the test previously, are described as the test's norm group. For example, if a student's score is (according to the norm group) at the 65th percentile, the student outperformed 65 percent of the students in the norm group. Comparisons based on such traditionally constructed standardized tests can be very illuminating. For instance, when parents learn that their son has scored at the 87th percentile in mathematics, but only at the 23rd percentile in reading, those parents can work with their son's teachers to improve his relatively weak reading skills.
But, because a student's relative standing typically doesn't change rapidly, traditionally constructed standardized achievement tests need not be administered yearly to supply parents with this kind of comparative information.
Action Implication: If your district administers traditionally constructed standardized achievement tests at every grade level, this is overkill. Try to get the district's education policy makers to reduce the frequency of such testing.
Fact 6. Traditionally constructed standardized achievement tests—designed chiefly to compare a student's test score with other students' scores—should not be used to evaluate school quality. When using a comparative approach to measurement, it is essential that students' test scores be well spread-out by a test. Accordingly, the developers of traditionally constructed standardized achievement tests must include test items that produce the necessary score-spread. Examples of such items, however, are those more likely to be answered correctly by (1) students from economically advantaged backgrounds or (2) students with higher in-born academic potentials. Unfortunately, these items actually measure what students bring to school, not what students are taught in school. Such tests, of course, should not be used to evaluate a school's success. A school should be judged primarily by what students have learned there.
Action Implication: Find out if students' scores on a traditionally constructed standardized achievement test play an important role in evaluations of your child's school. If so, work with other parents to get more suitable, nontraditional standardized tests used for that purpose. Standardized achievement tests can, and should, contribute to the evaluation of your child's school—but only if those tests are appropriate for that purpose.
Fact 7. Even though only about 25 percent of a student's success in college is related to the student's score on aptitude tests such as the ACT and SAT, parents should (1) still help their children prepare for those tests, but (2) avoid conveying a negative impression to a child whose test scores are not particularly high. The research evidence on this point is quite conclusive.
There are many factors far more influential than aptitude-test scores in predicting a student's college performance. A student's motivation, study habits, and interpersonal skills play powerful roles in shaping collegiate success. Parents should not think that a child who doesn't earn super scores on a college admissions test is destined for failure—in college or beyond. Numerous students who earn lofty scores on the SAT or ACT take an academic tumble when they get into college. There are many important kinds of intelligence, and the "academic" intelligence measured by most standardized aptitude tests is only one.
Action Implication: First, if your child doesn't score well on the ACT or SAT, do not conclude that your child is "not bright." And definitely do not convey any such negative impression to your child. Second, because ACT and SAT scores do, in fact, play a significant part in current college admissions decisions, be sure to provide your child with at least some preparation for those tests.
Fact 8. Performance tests, although both difficult and costly to score, often yield the most valid inferences about a student's mastery of many significant skills. Some of the most important academic skills are difficult to measure. For instance, to see whether students can write an effective essay, the best way to do so is surely to have students sit down and write such an essay (rather than answering a flock of multiple-choice items.) When students generate an original response to an essay-writing task, we call this a performance test. Performance tests require students to construct their responses "from scratch" rather than merely selecting their responses from the already-presented options in multiple-choice items.
Action Implication: Find out the extent to which performance tests are being used to assess your child's mastery of significant skills. If these tests are not used, collaborate with other parents to urge that at least some key skills be assessed via performance tests.
Fact 9. Although an individual student's attitudes or interests are almost impossible to measure accurately, assessing such affective outcomes on a group-basis can provide accurate and powerful evidence of the quality of a school's instructional program.
If parents set out to judge a school's quality, they should look at lots more than mere test scores. For instance, are students genuinely excited about what they are learning? Do students really look forward to going to school? Are students gaining increased confidence in their ability to use the things they are learning in school? These are important questions, and their answers all revolve around the kind of affect being promoted in a school.
The use of self-report affective inventories—that is, questionnaires completed anonymously by students—can supply educators and parents with important insights about students' affect. Such inventories, however, are not suitable for identifying an individual student's affective status. Yet, when students' anonymous responses to such inventories are pooled to be analyzed all together, the responses yield a reasonably accurate picture of the affective status of a student group. Thus, it is possible to obtain, at little cost, satisfactory estimates of group-affect dealing with such variables as students' (1) perceived safety when at school, (2) interest in various school subjects, (3) attitudes toward learning, and (4) confidence in carrying out significant academic skills.
Action Implication: Check to see if there is any affective assessment in your child's school. If not, urge that some group-focused assessment of key affective variables be initiated.
Fact 10. Because educational tests (both teacher-made classroom tests and commercially developed standardized tests) vary in their quality, it should not be assumed that every educational test is a good one.
Many parents tend to defer to educational test-makers. That's unwise. Most of today's classroom teachers, during their teacher-preparation programs, were never required to complete even a single course dealing with test construction. It is not surprising, therefore, that some classroom tests will not yield valid score-based inferences about students' skills or knowledge. But even commercially developed standardized tests also vary in their quality. More and more standardized tests are now being required in education. Not surprisingly, there is a limited supply of test-construction expertise in our nation. As a result, today's standardized tests, even those created by reputable test-development firms, will sometimes fail to be at the high quality level that those firms would prefer.
Action Implication: Do not give automatic deference to any educational test, either one that's created by your child's teacher or one that arrives, professionally printed, from a commercial test-development company. Use your own good sense in judging whether a given test is likely to yield valid inferences about your child.
As you can see, these 10 must-know facts aren't all that intimidating. They're simply based on common sense and what's been learned over the years by measurement specialists. Do parents need to tackle all 10 action implications suggested here? Of course not. But undertaking one or two isn't a terrible idea.
W. James Popham, Ed.D. is author of Testing! Testing! What Every Parent Should Know About School Tests (Needham Heights, MA: Allyn and Bacon, 2000). He is professor emeritus at the University of California, Los Angeles.