Posts Tagged ‘assessment’
I finally read a tab I had open for about a week: A teacher’s troubling account of giving a 106-question standardized test to 11 year olds.
This Washington Post blog entry provided a blow-by-blow account of some terrible test questions and an editorial on the effects of such testing. Here are the questions the article raised:
- What is the purpose of these tests?
- Are they culturally biased?
- Are they useful for teaching and learning?
- How has the frequency and quantity of testing increased?
- Does testing reduce learning opportunities?
- How can testing harm students?
- How can testing harm teachers?
- Do we have to?
The article was a thought-provoking piece that asked several good questions. Whether or not you agree with the answers is moot. The point is to question questionable testing practices.
I thought this might be a perfect case study of what a poorly designed test looks like and what its short-term impact on learning, learners, and educators might be.
The long term impact of bad testing (and even just testing) is clear in a society like Singapore. We have teach-to-the-test teachers, test-smart students, and grade-oriented parents. We have tuition not for those that need it but for those who are chasing perfect grades. And meaningful learning takes a back seat or is pushed out of the speeding car of academic achievement.
We live in testing times indeed!
by Kevin Walter
I read with great interest the initial reactions to the post-PSLE assessment book/paper burning by a group of parents and kids as well as the forum letters and editorials that followed.
Some people were vehemently against the act and explored the historical reasons of book burning. Others said that the burning was simply one of catharsis.
I think that many of the responses were emotions disguised as logic. It is perfectly acceptable to be passionate, but that does not mean losing your head about what you are passionate about.
To liken the revision paper burning to the way Nazis burnt books is ridiculous. The former was at worst an ill-judged cathartic release. The latter was based on terrible ideology.
No doubt that both types of burning look the same, but they have different origins and purposes. We regularly incinerate old books, newspapers, and other paper-based material (along with other rubbish) instead of reusing or recycling. Where are the voices and arms raised then?
To paint both with the same black-or-white brush is like saying all killing is bad. We kill for food, greed, defence, revenge, etc. Depending on the context, some killings are easier to justify than others.
I agree that the parents who organized or facilitated the burning might have inadvertently sent the wrong message that burning is the thing to do. There are certainly other ways to express relief (perhaps less emphatic or dramatic) other than burning.
But to judge without first understanding and attempting to educate all parties is just as harmful. It is a way to burn the bridge that links both sides.
Like it or not we live in a world with more shades of grey than ever before. That is why is it less important to transmit values than to teach the next generation how to think critically and recreate the values that matter.
One of the announcements at this year’s National Day Rally was a wider spectrum of entry criteria for the Direct School Admission programme.
Some might say the DSA makes a mockery of standardized exams because it allows Primary school students to get into the Secondary school of their choice. While Primary School Leaving Examination (PSLE) results are still used as criteria once they are released, the student with entry via DSA already has a foothold that non-DSA students do not.
A few might wonder if the PSLE is even necessary if such an alternative form of evaluation exists. Others might argue that the DSA criteria are not enough.
That brings us back to increasing the selection criteria for DSA. What traits might students be evaluated on? Leadership? Character?
When those traits were bandied about in popular media, people asked if things like character and leadership could be measured among 12-year-olds.
You can measure just about anything, even fuzzy, hard to quantify things like happiness [happiness index]. But let us not kid ourselves into thinking that these measures are absolute, objective, or universal.
A trait like creativity is due to many things, and an instrument no matter how elaborate, cannot measure all aspects of creativity. Most fuzzy concepts, like beauty, are subjective no matter how much you quantify them. Ask anyone to define creativity or beauty and you will get different answers; there is no single understanding.
Whenever you measure anything, there are margins of error that originate from the measurer and the measuring instrument. Sometimes the object or subject measured introduces error. Consider what happens if person A measures a fidgety person B’s height with a tape measure.
Let us say that you could measure leadership or character precisely. Just because you can does not mean you should. How different is a person when he is 6, 12, 18, 24 or 36? What if a value judgement at 12 puts a child on a trajectory that s/he is not suitable for?
We learnt that the hard way when we started streaming kids when they were 10 (Normal, Extended or Monolingual). Thankfully that process has been removed from our schooling system. Actually, I take that back. We still test for “giftedness” at 10. Some schools start pre-selecting at 9.
That said, we would be foolish to think that we do not already gauge people on fuzzy traits like character. It happens in the hiring and firing of employees. Some might argue that we are just bringing that process up the line of development.
There are many ways to measure fuzzy traits. At a recent #edsg conversation, I tweeted:
Whether or not these measures to provide alternative evaluation are implemented, we will read in forum letters, blog entries, and Facebook posts rhetorical statements like “parents must change their mindsets.”
Of course they must. But they are not going to do so automatically.
Folks who highlight mindset sometimes fail to realize that you have to start somewhere with behaviour modification. In systemic change, you start with one or more leverage points. In our case, it is the way people are evaluated.
by Ms. Tina
Walking about in a mall, I saw a promotion to Pay Less, Taste More. Our Ministry of Education advocates Teach Less, Learn More.
After reading these three articles and reflections, I think we could and should Grade Less, Learn More.
- You Should Probably Just Grade Less
- Instead of Standardized Tests, Why Not Build the World Wide Web?
- My own archived blog entry Question The Model
The first piece by Bud the Teacher started the ball rolling . In his own words:
For starters, I think teachers, in general, grade too many things. So one way to streamline would be to “grade” less. And that doesn’t mean that teachers shouldn’t ask students to write, and write often. But we don’t need to grade everything that comes to us. In fact, we should grade very little of it.
In the context of writing, Bud went on to say that teachers have built up the wrong expectations that they are their students’ only audience and that they alone evaluate what that writing is worth.
He argued that technology should not be the means of streamlining grading when the problem lay with the assumptions, design, and expectations of grading.
Put another way, the questions he might have asked could include: Why are students writing for just one person? Why not leverage on authentic and public writing spheres? Why not approach the task as a fellow reader and learner in order to critique instead of just grade?
Cathy Davidson wrote the second article and she was as thought-provoking as always.
She started by describing seven reasons why standardized testing, while great for grading, are not that great for evaluating learning. So she suggested a key alternative: Get learners to create by learning how to code.
I do not think that all learners need to code, but I do think that all learners need to be given the freedom and opportunity to express themselves and to create. Only then are they putting theory into practice. Only then are they making mistakes. Only then are they really trying.
As a result of reading these and other articles, I had to ask myself if I had reflected on these issues before. I had in the third piece.
Two years ago, I had concluded that modernizing a problem (e.g., creating grading software) was not the solution. We might create more intelligent systems, but the problem of testing irrelevant skills in irrelevant ways would still remain.
No, the solution lies in questioning the established but problematic model. It starts with accepting that grading more sometimes leads to learning less.
Grading takes time and students do not get timely feedback. Students do not pick up as much content or as many skills as teachers wish. Teachers do all they can to keep up with the grading load and do not have the bandwidth to learn new strategies.
A solution might lie in figuring out how to grade less so that everyone learns more. By grading less I do not mean asking for lazier teachers. We need teachers who question, learn, and act on what they realize is better for their learners.
Teachers could grade less by redesigning tests and focusing on key things that need grading. They could leverage on their learners’ need to communicate, create, and critique. Teachers could model these life skills and teach their students to peer assess in authentic environments.
As they do this, I bet they would learn more about what their learners are capable of and what new value they as teachers bring to the table.
This article begins with an intriguing question: When is a test not a test? It cites a tweet by @Scott_E_Benson:
The future of testing will be tests that students, teachers and parents do not think of as tests.—
Scott Benson (@Scott_E_Benson) April 26, 2012
Then it dances around the benefits and pitfalls of tests before suggesting how one might assess and evaluate without the tests that we are most familiar with.
It suggests gamification and gaming strategies. It suggests portfolios, self-assessment, and peer accountability. It suggests measures that are more progressive than the quality control tests that are relevant only for the industrial age.
Thinking gamers might tell you that they are being tested all the time but the tests do not feel like tests. That is when a test is not like a test.
A game can be pure fantasy, be based on reality, or be a hybrid like the one featured above. Unlike a most video games, the game does not have obvious quests and thus mirrors much of life.
It is also been said that, unlike school, life throws tests at you whether or not you are ready. When that happens, you experience a knowledge gap and you need to problem-solve. That makes the seeking, analysis, and use of information relevant.
Despite the surprises that life throws at you, this form of insidious testing seems natural. School-based testing does not.
Like other creatures in the animal kingdom, we start learning by play. Why not be tested by play?
Two individuals have been debating in a local paper the merits and demerits of severing Singapore’s link with the GCE examination system.
Frankly, I think that the first person did not go far enough!
How about not just having our own brand of exams but breaking out of our outdated mode of assessment? Like this Chronicle article, I think that we should stop telling our students to study for (the sake of) exams. On a related note, this father wishes that he could tell his kids not to just study for exams [PDF].
We have set our system up like an unpleasant game where students try to move up from one level to another by defeating boss exams at the end of each level. Unlike a real video game, kids do not want to play the exam game willingly.
I am not saying that learning should not be hard. It already is. I am saying that we should not be learning just for the sake of exams. While that may be the point of schooling, it is not the purpose of education.
School is meant to prepare students for real life, but the exams rarely reflect real life. They do not set the child on the path to enjoyable or meaningful lifelong learning. They rely on paper and pen(cil) in an electronic world. They require you to work alone.
In the real world you can ask for help or collaborate. In the exam world that is cheating. In the real and e-assisted world, you can connect with others and content. Often you can do that faster than you can with the teacher right in front of you.
In the real world your performance and other factors you cannot always control (like whether your boss likes you) determine outcomes. In the real world we rely on portfolios, presentations, projects, and evidence of learning.
Our current assessment system is broken. But our industrial era and industrial strength system keeps trying to put it back together and patching over the cracks. It is a matter of time when the cracks get more noticeable.
I was happy to read an update about PERI’s assessment seminar in this MOE press release. I was disappointed that I could not learn more from the link, the PERI Holistic Assessment Website, provided in that release because it was password protected.
Why call it “peripheral”? There are just 16 schools trying out the PERI recommendations in any significant way, even though the programme was successful in the pre-trial school (the school that started the whole thing really!). A major venture might justify a more cautious approach and there is an obvious need for each school to design its own way. But we are talking about threatening the life of a sacred cow (Singapore assessment) here. Why not be more daring with the carving knives?
The PERI trial is not quite in the public eye in more ways than one and this is unusual given the impact on its stakeholders. The website I mentioned is not readily public for one. The other schools who are not involved do not seem terribly concerned for another.
When I ask primary school principals and teachers that I meet in non-PERI trial schools about their thoughts on PERI, I get 1) a shrug of the shoulders and 2) an arching of the eyebrows. The latter indicates that they are surprised that I know anything about it. The former indicate indifference (wait-and-see), ignorance (wait-for-someone-to-tell-me) or intolerance (nothing-will-change).
So much for change management. There is what you say and there is what you do. We are all held accountable by what we do. Let us see more of what is happening in that black box!
I am worried that we are addicted to exams. Look at one neighbour comparing grades with another. Look at both of them looking for tuition for their kindergarten-attending kids. Look at schools sacrificing vacation time to prep students for exams and even to conduct exams. It’s a sad fact of Singapore life.
Just as disturbing was a letter a young man wrote to the Straits Times forum yesterday, Why Primary 1 exams are necessary. The writer, Sean, said that parents are to blame. But I think that he was equally culpable since he supported exams.
There are a few ways to respond at this point. One is, “What is wrong with exams?” and another is “Sean does not know any better, so give him a break”.
So what is wrong with exams as we currently know them? Nothing, if you were tested in life exactly like those exams. Nothing, if what you knew only in your head sufficed in life. Nothing, if the joy of learning weren’t drained out of you as if it were poison. Nothing, if a few hours frantically scribbling at a table did not result in a few letters that labelled you a success or failure regardless of your character, drive or other abilities. Nothing is wrong with exams if they did not create an addiction to high stakes testing.
I’m tempted to give Sean a break. As a pawn of the system, he is unlikely to know any better. If you are a frog born at the bottom of a dark well, you know not how other frogs live. But Sean, his peers, their parents and teachers have a choice if they bother to look and change. Unfortunately, they are all addicted to exams.
The demand for exams is created by policy makers, administrators and bean counters. They are so convincing that teachers not only design the exams, they also feed the exam machinery. Exams grip students and parents by fear and they live by that fear. These are our exam drug lords, unwitting peddlers and helpless addicts.
Another response to my rant might be, “You and I were a result of our exam system, but look how well we turned out!” To that I say: We probably did well not because of the system, but despite of it. Think about it.
I’ll state the obvious:
- Assessment is a key leverage point for educational change.
- Our teachers teach to the test because they are pressured to.
- If our modes and methods of assessment do not change, very little else will.
High stakes exams are designed to sort just like quality control in a factory. Exams are a legacy of the Industrial Age. Is there are time and place for tests and exams? Maybe. But not all the time and in every classroom. The folks at Harvard (thanks for the link, Steve!) are discovering this for various reasons.
Now if the winds of change blow new ideas our way, how will we respond? If we choose to remain addicted, not well.
But I detect gentle breezes that we casually label alternative assessments. These alternatives include meaningful portfolios, performances and projects.
I do not think that they should be alternatives like substitutes on a football bench are alternatives. These alternatives get little or no playing time in the real game. These alternatives are young and inexperienced. I will do what I can to see many of these alternatives become mainstream while turning exams into alternatives. After all, the exam player is getting old and has tested positive for drugs.