Another dot in the blogosphere?

Posts Tagged ‘assessment

We live in testing times. Not just politically or environmentally, but also in terms of actual tests.

So here is a basic tip with multiple-choice questions like the one above: Use LETTERS as options instead of numbers.

This is the MOE press release that accompanied the announcement on reducing tests in Singapore schools.

First comes the policy shift (long overdue, in my opinion). Then might come the years-long mindset shifts. Next is the decades or generations-long behavioural shifts.

The press release ends as most documents that herald change do.

You could apply points 15 and 16 to any change in schooling, but that does make them any less true.

The stakeholders hardest to reach and change lie immediately outside the school arena, i.e., parents and enrichment tuition centres. This is what makes the change process arduous.

Like teaching, the policy announcement is neat. And like learning, the actual change processes are messy. It is time to muck about.

One of the replies to my tweet about the parliamentary response to stolen exam papers — electronic scanning and and marking of scripts — was this tweet.

I had to look up the product and service and found a UK-based website and YouTube video.

Apparently SurpassPaper+ allows students who opt to take electronic versions of an exam on their own devices alongside their peers who opt to take the paper version.

There are several advantages of taking the electronic version. The ones that stood out for me were:

  • Students use a platform they are already accustomed to.
  • The submissions are immediate and do not incur physical handling, storage, security, and transport costs.
  • Proctors can monitor student progress with an app and intervene if necessary.
  • Students can continue on an alternate device should their own fail them.

If all this seems innovative compared to the old-school method of high-stakes exams, then we should cast our eyes on how some standardised tests are regularly taken on Chromebooks in US school districts.

The change is also just an incremental one. Evolutionarily speaking, the new test animal is not that different from the generation before. It has not replaced the old one and actually lives alongside the incumbent species as a minority and novelty.

The bottomline is this: The medium has changed, but the method has not. Changing the medium is comparatively less disruptive and easier than changing the method of assessment.

To change the method is to face the usual suspects of barrier statements. I share just three and pose three questions as responses.

The first barrier statement is: We should not abandon what is good about the old or current method. My questions are: What is objectively good about it? From whose perspective is “good” defined?

The second barrier is an excuse: Now is not the time. My response are: If not now, then when? How will you know when the right time is? What if the right time is too late? How can we make it the right time?

The third barrier is a generalisation: Change will take time. My response is:
Of course it does. But when will you start?

The breaking news that refused to die was about the A-level Chemistry papers that were stolen last year. This time ministers in Parliament discussed how to prevent this from happening again.

The suggestion: Scan the papers and mark them electronically.

For me this was braking news — I had to stop to think about what was actually going on.

Superficially, the issue was about the security of high stakes examinations. While student results are important, the larger messages were missed, i.e.,

  • The exams are still handwritten on paper.
  • They are still reliant on factual recall.
  • The assessment is inauthentic — there is no referencing, no cooperating, etc.

This pays lip service to the supposed 21st century competencies that we are supposed to develop in learners. If we are to do this, we need to pull assessment into the same century.

Like it or not, assessment is the tail that wags the dog. Summative forms of assessment like end-of-course examinations are terminal activities — they are the tail. However, they dictate what is taught, how it is taught, and shape how students opt to learn — they wag the dog.
 

 
The examination in question was the GCE A-Levels. These are taken by girls whose next destination is likely university, and boys who become men via military service (if they are citizens and permanent residents).

However, these students take paper-based exams much the same way they did ten years before when they were in primary school. Heck, I took my A-levels on dead trees and I am older than some trees!

I now mentor, advice, and teach some future faculty who still clutch at paper as the be-all and end-all technology. They teach and test like a book and by the book. The assessment tail does not just wag the dog; it trains the dog and shapes its psyche as it rewards and punishes the dog.

Am I overreacting? After all, the issue was exam paper security and not assessment redesign. But why was the latter not the issue?

Just consider the logistics and costs. The papers had to be transported to the United Kingdom. They had to be stored and provided with some modicum of security. They also had to be transported securely to graders and then brought back centrally for more processing.

Even if every script was scanned and marked electronically, there is still the cost of scanning every page and retraining the graders.

These exercises help the agencies involved in the processes — question-setting, grading, analysing, transport, storage, security, administration, etc. You might think of this as an assessment mill that is dependent on paper mills.

But what of the current student and future employee who has to rely less and less on paper and paper-led habits? Our duty is not to keep the assessment and paper mills alive. It is to help our learners thrive in their future, not our past.

Take writing for example. We still have to write, but how much on paper and how often?

The medium is part of the message and shapes the way we think and craft those messages. For example, I am drafting this reflection in MacOS Notes, I have a web browser with these tabs open: WordPress (for the blog entry), ImageCodr (for the CC-licensed images), and several online references.

The writing skills might be the same — for example, logical paragraphing — but the need to write shorter paragraphs is the new expectation. This reflection is already too long for most people. TLDR. So I also break the message up into chunks with photos (aww, cute doggies and baby!).
 

 
But back to the main topic of changing assessment. I am not suggesting that we throw the baby out with the bath water. I am pointing out that the bath water is still there, getting filthier by the minute, and threatening to drown the baby.

If this analogy is not clear, the paper-based exams are the problem because we do not question their purpose. They solved the problem in the past of how to sort students, and they still do that. But they also create unnecessary stress and entrench old mindsets, neither of which are good for our students.

It is time to throw the bath water out, not build a better receptacle, replace the water, or somehow have self-cleaning water.

One basic assessment design principle is: Do not provide answers in your question.

If you provide answers, it is your fault. If learners take advantage of this, you cannot penalise them (see example above).

Instead of telling the student to see you, you should see yourself, i.e., reflect on the error of your ways and sign up for professional development on Assessment 101.

The tweet below is wrong.

The girl in the image is not five metres tall. She is taller than that.

Including a human figure is a physics problem does not make it friendlier. Claiming that she is a giant does not make it authentic.

Apparently, the diagram was from a textbook. This means that more than one person was involved in designing, editing, and approving the question. All of them have provided a free lesson on how not to design assessment questions.

Far wiser and more articulate people have shared their thoughts on assessment, grading and feedback. So I reshare what they shared.

From these and the work of others, I distill some wisdoms into these image quotes.

Formative feedback

Quantitative grading ends learning. Quality feedback sustains learning.

 
The word “evaluation” might have been ill-defined and misused.

I was surprised to read someone like Senge reportedly saying this about evaluation.

Evaluation is when you add a value judgment into the assessment. Like, ‘Oh, I only walked two steps. I’ll never learn to walk.’ You see, that’s unnecessary. So, I always say, ‘Look, evaluation is really optional. You don’t need evaluation. But you need assessment.

Evaluation is about adding a value judgement into assessment. That is why it is called eVALUation. But that does not make evaluation negative or optional.

Student A might get an assessment score of 60/100. Student B might get an assessment score of 95/100. One way to evaluate the students is to compare them and say that student B performed better than A. More is better and that is the value, superficial as doing that may be.

If you consider that Student A previously got a score of 20/100 and B a previous score of 90/100, the evaluation can change. Student A improved by 40 points; student B by 5 points. The evaluation: Student A made much more improvement than Student B.

The value judgements we bring into assessments are part of evaluations. Assessments alone are scores and grades, and not to be confused with the value of those numbers and letters.

In the context of working adults who get graded after appraisals, a B-perfomer is better than a C-performer. The appraisal or assessment led up to those grades; the worker, reporting officer, and human resource manager place value in those letters (no matter how meaningless they might actually be).

The assessments of children and adults are themselves problematic. For kids, it might be a broad way of measuring a narrow band of capabilities (academic). For workers, it might be an over simplistic way of assessing complex behaviours. So the problem might first lie with assessment, not evaluation.

As flawed as different assessments may be, they are simply forms of measurement. We can measure just about anything: Reasoning ability, level of spiciness, extent of love, degree of beauty, etc. But only evaluation places value on those measurements: Einstein genius, hot as hell, head over heels, having a face only a mother could love.

I have noticed people — some of them claiming to be teachers or educators — not understanding the differences between assessment and evaluation. As the terms have not been made more distinct, evaluation has been misunderstood and misused.

Evaluation is not a negative practice and it is not optional. If evaluations seem overly critical (what went wrong, how to do better), they merely reflect the values, beliefs, and bias of the evaluator. We do not just need assessment, we also need evaluation to give a measurement meaning.


http://edublogawards.com/files/2012/11/finalistlifetime-1lds82x.png
http://edublogawards.com/2010awards/best-elearning-corporate-education-edublog-2010/

Click to see all the nominees!

QR code


Get a mobile QR code app to figure out what this means!

Archives

Usage policy

%d bloggers like this: