Another dot in the blogosphere?

Posts Tagged ‘exams

The breaking news that refused to die was about the A-level Chemistry papers that were stolen last year. This time ministers in Parliament discussed how to prevent this from happening again.

The suggestion: Scan the papers and mark them electronically.

For me this was braking news — I had to stop to think about what was actually going on.

Superficially, the issue was about the security of high stakes examinations. While student results are important, the larger messages were missed, i.e.,

  • The exams are still handwritten on paper.
  • They are still reliant on factual recall.
  • The assessment is inauthentic — there is no referencing, no cooperating, etc.

This pays lip service to the supposed 21st century competencies that we are supposed to develop in learners. If we are to do this, we need to pull assessment into the same century.

Like it or not, assessment is the tail that wags the dog. Summative forms of assessment like end-of-course examinations are terminal activities — they are the tail. However, they dictate what is taught, how it is taught, and shape how students opt to learn — they wag the dog.
 

 
The examination in question was the GCE A-Levels. These are taken by girls whose next destination is likely university, and boys who become men via military service (if they are citizens and permanent residents).

However, these students take paper-based exams much the same way they did ten years before when they were in primary school. Heck, I took my A-levels on dead trees and I am older than some trees!

I now mentor, advice, and teach some future faculty who still clutch at paper as the be-all and end-all technology. They teach and test like a book and by the book. The assessment tail does not just wag the dog; it trains the dog and shapes its psyche as it rewards and punishes the dog.

Am I overreacting? After all, the issue was exam paper security and not assessment redesign. But why was the latter not the issue?

Just consider the logistics and costs. The papers had to be transported to the United Kingdom. They had to be stored and provided with some modicum of security. They also had to be transported securely to graders and then brought back centrally for more processing.

Even if every script was scanned and marked electronically, there is still the cost of scanning every page and retraining the graders.

These exercises help the agencies involved in the processes — question-setting, grading, analysing, transport, storage, security, administration, etc. You might think of this as an assessment mill that is dependent on paper mills.

But what of the current student and future employee who has to rely less and less on paper and paper-led habits? Our duty is not to keep the assessment and paper mills alive. It is to help our learners thrive in their future, not our past.

Take writing for example. We still have to write, but how much on paper and how often?

The medium is part of the message and shapes the way we think and craft those messages. For example, I am drafting this reflection in MacOS Notes, I have a web browser with these tabs open: WordPress (for the blog entry), ImageCodr (for the CC-licensed images), and several online references.

The writing skills might be the same — for example, logical paragraphing — but the need to write shorter paragraphs is the new expectation. This reflection is already too long for most people. TLDR. So I also break the message up into chunks with photos (aww, cute doggies and baby!).
 

 
But back to the main topic of changing assessment. I am not suggesting that we throw the baby out with the bath water. I am pointing out that the bath water is still there, getting filthier by the minute, and threatening to drown the baby.

If this analogy is not clear, the paper-based exams are the problem because we do not question their purpose. They solved the problem in the past of how to sort students, and they still do that. But they also create unnecessary stress and entrench old mindsets, neither of which are good for our students.

It is time to throw the bath water out, not build a better receptacle, replace the water, or somehow have self-cleaning water.

When students cheat on exams it's because our school system values grades more than students value learning. -- Neil deGrasse Tyson

I created this image quote several months ago from Neil deGrasse Tyson’s original tweet.

This week I stumbled upon this video about exam “techniques”.


Video source

There is so much value placed in exams that some students resort to cheating and some examiners resort to extremes.

If you are assessment literate, you might realise that paper-based exams are a practice inherited from the industrial age. They have become a mainstay because we have forgotten to question and critique them. One big question is WHY we still have them. One big critique is the narrow academic-only ability they measure.

Despite viable alternatives like performance assessment and e-portfolios, there are groups that keep fanning into the embers of exams, e.g., the TES crowdsourcing A-Level exam questions.

Recently, Manu Kapur wrote an excellent opinion piece on why we should not over-rely on exams as we know today.

I paraphrase his main points. Test and exams:

  • May measure what we know, but not what we can apply with that knowledge or to create new information.
  • Do not guarantee transfer. The acquisition of information does not guarantee conversion to knowledge, and this in turn does not guarantee usage in real contexts.
  • Prevent students from using resources they would otherwise use in the wider world, e.g., their mobile phones.
  • Limit problem-solving to minutes at a time, and do not encourage persistence or perseverance.
  • Mould students to think and act under test conditions. They do not encourage deep learning and mastery.
  • May not match the cognitive developmental stage of the learner.

Tests are not authentic, they encourage superficial learning, and they are not forgiving.

Tests are outdated and it is not surprising why some students opt to cheat. They have so much stacked against them cognitively, ethically, and holistically.

I am not suggesting that students cheat. I am saying we must start operating outside the the test and exam paradigm.

How? Kapur’s article briefly outlines the approaches involving policies, people, and practices. It is well worth the read. It is even better to take action.

The news that caused ripples in Singapore schooling last week was the official announcement from the Ministry of Education (MOE) of the new scoring system that will be implemented in the Primary School Leaving Examination (PSLE) in 2021.

There was a slew of news following the announcement. Some people made tsunamis out of the ripples, some rode the waves as they were [small sample of both].

Beneath the surface was an undercurrent that did not get much attention, but was the most significant change in terms of education. According to STonline, one of the changes was the switch from norm-referenced testing (NRT) to standards or criterion-referenced testing (CRT).

PSLE2021: From NRT to CRT

What are NRT and CRT in layman terms? Why is the switch an important driver of change?

In NRT, the results of a cohort of students are reduced to scores — T-scores in the case of PSLE — and lined up from the highest to the lowest (or vice versa). The result is a bell-shaped curve of scores: There will be a few very low and very high scores, and many somewhere-in-the-middle ones.
 

 
Reviewers of these scores typically use this distribution to create an even curve (a normal distribution, ND), and to rank and sort. In the adult world of work, this method might help determine who gets promotions or bonuses, what appraisal grade you get (if you are in the civil service), or who gets fired.

For example, a large organisation can first rank the performances of all its employees. If an ideal ND does not result, it can statistically massage it into an ideal bell curve. So if there are too many A-graders, some will be pushed into Bs, and as a result Bs become Cs and so forth. Once there is an ideal bell curve, someone can decide cut-offs and consequences, say, the top 5% get promotions and the bottom 15% are let go.

If this seems unfair to working adults, then what more for the 12-year-old children who take the PSLE but have no idea what is going on?

The core problem is that people are compared one against the other with or without their knowledge. If with, this can result in unhealthy competition because they want to be on the right part of the ND curve. If without, the people become victims of processes not transparent to them and circumstances beyond their control.

Is there a better way? Yes, it is called CRT (standard-based assessment and/or evaluation).

Modern corporations like Accenture are abandoning the outdated practice of norm-referencing [1] [2] and embracing comparisons of one. The fundamental principle is this: How one improves and contributes individually over time is more important than how one is measured against others.

For example, a worker might show evidence of specific skills that indicate that he or she is a novice, intermediate, or advanced worker. There is no comparing of all the workers regardless of their skill group or even comparing within each skill group.

To make this work, there must be standards or criteria that identify each skill group, e.g., skills A to J for novices to master; K to R for intermediates; S to Z for advanced plus five potential managerial markers.

Back to PSLE 2021. The switch is from NRT to CRT. It is more about the standards or specific criteria that indicate the test-based achievements of the child, and less about the comparison of one child with another.

This is a fundamental shift in mindset from sifting and sorting to measuring performance. The former is about what is good for the system and how to feed it; the latter is about where the learner is at and what is good for the learner.

However, this piecemeal change of the CRT system of academic levels (ALs) still falls short. I share thoughts on these in more reflections on PSLE2021 over the next few days.

Read Part 2: The Dark Side.

Neil deGrasse Tyson might have lost some credibility for tweeting a false claim and facing this scientific backlash.

He is certainly as expert in astrophysics. But he should not have ventured into biology.

When students cheat on exams it’s because our school system values grades more than students value learning. -- Neil deGrasse Tyson

That said, he is still a smart man. His observation about why students cheat during exams is spot on.

I do not cheat with images and attribute this CC-licensed image for my image quote.


Video source

In this video Noam Chomsky explains the problems with assessment: The way they are misused, misaligned, and misappropriate.

It is no surprise then that a Secret Teacher wrote the following article in The Guardian about how tests seemed to be dumbing down her students.

The teacher bemoans:

My students are bright, engaged and well-behaved, but there is something missing: they cannot think.

The Secret Teacher goes on to blame a focus on exams and I agree with the teacher for the most part. But tests are not the only thing to blame for students who do not know how to think independently.

Teachers who spoon feed, stifle thought, or fail to stay relevant are just as culpable.

For instance, the teacher said:

Last week I caught another of my A-grade students using his phone in the lesson. As a starter exercise, I told them to think of as many advantages as they could of being on the UN security council. “What are you doing?” I asked. “I’m googling the list of advantages,” came his wary reply. I was flabbergasted. I tried to explain that there is no list of advantages, but that I wanted his own views.

I am confident that the Secret Teacher is also a Good Teacher. But she also sounds like a traditional one in that she is averse to searching for Googleable answers. Perhaps she did not know how to take advantage of a now natural behaviour to show her students how to think, act, and write critically after Googling.

Most people would eventually realize that the most important factor in a schooling or educational system is the quality of its teachers. Those that join the profession are self-selecting by choice and pre-selected by institutes of teacher education.

But only the exceptional step up to deal with the problems with assessment or learn how to skilfully promote critical and creative thinking in a conservative system. The rest need professional development and the mindset of lead learners to do this.

Cracked pot by mirsasha, on Flickr
Creative Commons Attribution-Noncommercial-No Derivative Works 2.0 Generic License  by  mirsasha 

Two individuals have been debating in a local paper the merits and demerits of severing Singapore’s link with the GCE examination system.

The first person [PDF] suggested that we have our own examination system. The responder [PDF] gave the usual “if it ain’t broke, don’t fix it” reply.

Frankly, I think that the first person did not go far enough!

How about not just having our own brand of exams but breaking out of our outdated mode of assessment? Like this Chronicle article, I think that we should stop telling our students to study for (the sake of) exams. On a related note, this father wishes that he could tell his kids not to just study for exams [PDF].

We have set our system up like an unpleasant game where students try to move up from one level to another by defeating boss exams at the end of each level. Unlike a real video game, kids do not want to play the exam game willingly.

I am not saying that learning should not be hard. It already is. I am saying that we should not be learning just for the sake of exams. While that may be the point of schooling, it is not the purpose of education.

School is meant to prepare students for real life, but the exams rarely reflect real life. They do not set the child on the path to enjoyable or meaningful lifelong learning. They rely on paper and pen(cil) in an electronic world. They require you to work alone.

In the real world you can ask for help or collaborate. In the exam world that is cheating. In the real and e-assisted world, you can connect with others and content. Often you can do that faster than you can with the teacher right in front of you.

In the real world your performance and other factors you cannot always control (like whether your boss likes you) determine outcomes. In the real world we rely on portfolios, presentations, projects, and evidence of learning.

Our current assessment system is broken. But our industrial era and industrial strength system keeps trying to put it back together and patching over the cracks. It is a matter of time when the cracks get more noticeable.

Romatic Reels by wvs, on Flickr
Creative Commons Attribution-Noncommercial 2.0 Generic License by  wvs

A year ago, I blogged about a BBC article on how the Danish use the Internet in their exams. The minister of Danish education was reported to have said:

Our exams have to reflect daily life in the classroom and daily life in the classroom has to reflect life in society. The internet is indispensible, including in the exam situation.

Stephen Heppell’s take was:

As a nation we’ve been really good at embracing technology – we’ve been really at the forefront of doing this well in the classroom. Then they go into the exam room and all that’s taken away and they’re given a fountain pen and a sheet of lines paper and a three hour time limit. It’s time to get real, isn’t it?”

Oh, those “crazy” Europeans… such progressive risk-takers!

But the naysayers say students are cheating. If you read the article, you’ll realize that they can’t afford to. Even if they could communicate with others, what is wrong with that? As other bloggers have pointed out, what we label “cheating” in a classroom we call “collaboration” elsewhere.

There’s real life and then there is reel life. Schools operate in a movie-like bubble where things do not mirror real life and move in dreadfully slow motion. Burst that bubble!


http://edublogawards.com/files/2012/11/finalistlifetime-1lds82x.png
http://edublogawards.com/2010awards/best-elearning-corporate-education-edublog-2010/

Click to see all the nominees!

QR code


Get a mobile QR code app to figure out what this means!

My tweets

Archives

Usage policy

%d bloggers like this: