Another dot in the blogosphere?

Posts Tagged ‘exam

Yes, kids should learn from mistakes. But they are not likely to this as a result of high stakes summative exams.

Tests are not the best method for developing resilience and critical reflection. A major exam like the PSLE has one main purpose — to sort. 

Assessment and evaluation experts know this. The layperson does not. It will take a lot of re-education of learners young and old to beat the exam mindset into submission. I doubt we can do this. But we might be able to get enough people to realise the limits of tests and exams.

Last month, there was the furore over the International Baccalaureate results. They were adjusted algorithmically because the exams were affected by the current coronavirus pandemic.

What was the problem? The Wired article reported:

Teen regrets about grades aren’t unusual, but the way the foundation behind the IB Diploma Programme calculated this year’s grades was. The results, released Monday, were determined by a formula that IB, the foundation behind the program, hastily deployed after canceling its usual springtime exams due to Covid-19. The system used signals including a student’s grades on assignments and grades from past grads at their school to predict what they would have scored had the pandemic not prevented in- person tests.

Now there is an uproar over the English GCE A-Level results. This was history quickly repeating itself because that was a similar problem just last week with the Scottish exams.

Video source

According to BBC news, the exams were also cancelled because of the pandemic and replaced with teacher assessment. The results in England were moderated centrally so that 39% of grades were marked down, while only 2% marked up.

This system of decentralised testing and centralised decision-making was supposed to keep results in line with previous years. But you cannot have your cake and eat it, too. The systems works only if you keep all its parts intact and in play.

I have a different view. There was no real need to make adjustments so that this year’s results were comparable to previous years’. The circumstances were different and so was the method. The outlier of better than expected results should simply go on record as exceptional circumstances.

The burden should fall on universities to adjust their entry requirements. They can be as selective as they wish to be because they are in the business of higher education. They can conduct their own entry tests or require portfolios of work.

The worst and best thing that could happen is that the education system learns that standardised exams are not that important for university entry. Or perhaps powerful entities already realise that and want to protect the sacred cow that is assessment.

A news article led with the headline “More than a third of GCE coursework subjects to have assessment tasks reduced”.

If you read the details, that “more than a third” is stratified. The 12 academic subjects are:

  1. N(T)-Level Music Syllabus T
  2. N(T)-Level Food Studies
  3. N(T)-Level Design & Technology (Revised)
  4. N(A)-Level Food & Nutrition
  5. N(A)-Level Design & Technology (Revised)
  6. N(A)-Level Design & Technology (Legacy)
  7. O-Level Food & Nutrition
  8. O-Level Design & Technology (Revised)
  9. O-Level Design & Technology (Legacy)
  10. O-Level Exercise & Sports Science
  11. O-Level Drama
  12. A-Level H2 Theatre Studies & Drama

If the subjects share anything, they are performative or skills-based.

If anything at all, this is a reminder not to play the quantitative game. The qualitative details matter.

If you actually read the article, this segment might give you pause:

SEAB said the move aims to “alleviate stress on students and teachers… without compromising the validity of the assessment”.

If I was to challenge that statement, I would simply cite the examples in the same article about not performing group drama pieces or sports science students forced to work with truncated training plans.

If this reduction is possible with compromising on quality, why was there that much assessment before?

If such a reduction is possible with skills-based subjects, might the same be said about the other subjects? If the immediate answer is no, why (really!) not?

If the examination board is “allowing special provisions” like “footage recorded as far back as last year”, might the COVID-19 lockdown be a push to portfolio-based evaluation instead of paper-based assessment?

This is the MOE press release that accompanied the announcement on reducing tests in Singapore schools.

First comes the policy shift (long overdue, in my opinion). Then might come the years-long mindset shifts. Next is the decades or generations-long behavioural shifts.

The press release ends as most documents that herald change do.

You could apply points 15 and 16 to any change in schooling, but that does make them any less true.

The stakeholders hardest to reach and change lie immediately outside the school arena, i.e., parents and enrichment tuition centres. This is what makes the change process arduous.

Like teaching, the policy announcement is neat. And like learning, the actual change processes are messy. It is time to muck about.

I would like to critique this move fairly, but I cannot as the rest of the article is behind a paywall.

However, the SEAB has a track record of siding with caution. It moves so slowly, if at all, that molasses in a jar looks like speed demon.

Article screenshot: More trials before switch to electronic marking of exam scripts.

The SEAB seems to favour changing the medium and not the method, and as a result, not change at all.

This example of electronic marking would presume electronic test-taking were simple transitions from paper to screens. This is what happened with early e-books. Going electronic in this manner did (and does) not take advantage of hyperlinking, searching, and collaborating.

To push the boundaries where they need to be, the method must also change. The test should not just be about individual accountability, but also about the ability to communicate, cooperate, and collaborate. The challenge should not just be about low level thinking, but about contextual application, evaluation, and creation.

The superficial change in medium and not the method reveals this: The SEAB is neither prepared (state of mind) nor ready (state of being) to design and implement meaningful change. It is about jogging on the same spot to create the impression of work, but not move in any particular direction.

We live in testing times, not least because of people like Trump and the consequences of their thoughtlessness.

Last week, the local press bragged about how Singapore universities were moving towards electronic examinations.

This sounds oh-so-progressive until you read excerpts like:

  • “laptops to replace pen-and-paper exams because students are losing the ability to write by hand”
  • “online exams save paper”
  • “efficiency in distribution of exam papers, marking and collating results”

The reasons for changing the medium of exams were relatively superficial. Legibility of writing and saving paper are natural shifts in switching media. That is like saying switching from a bicycle to a plane lets you travel further and faster, and allows you to have a bird’s eye view. Of course you would!

There was no mention of how switching to electronic forms was not only more aligned with how we consume media today and how many students take their notes. The latter, in turn, is linked to the practice medium matching the task medium. If you do not understand the last point, consider a common response from teachers: Why should we use computers when students still have to take exams with papers and pens?

“Efficient” or “efficiency” was mentioned at least four times in the short article. Apparently, more effective ways of measuring learning were not on the radar.

The paper claimed that universities were “adopting more creative ways of assessment… audio or video segments, and interactive charts and graphics”. Again, that those are functions of richer media.

But can students also respond in equally creative and critical ways? Apparently not since “the students will have a ‘lock-down browser mode’ to prevent cheating, which cuts access to the Internet”.

Those that prepare the e-exams would rather set the same type of lower level Google-able, app-solvable questions than to change their methods and set unGoogle-able questions or tasks instead.

I said it in my tweet and I will say it again: This is a change in exam media, but not a shift in method or mindset.

Still on the topic of tests, I tweeted a WaPo article last night.

TLDR? Here is a comic I found in 2014 that summarises the take home message.

Tests. I can take tests.

The WaPo article did an excellent review of a national exam in the USA and tested the test with the help of three researchers. The researchers were experts in the content area of the test (history) and of assessment in general.

The researchers found that the tests only functioned to teach test-takers how to take tests. The questions did not necessarily test critical thinking skills like:

  • “explain points of view”
  • “weigh and judge different views of the past,” and
  • “develop sound generalizations and defend these generalizations with persuasive arguments”

Those tests were also going electronic or online. But again the change in medium was apparent; the change in method was not.

If we are going to design better forms of assessment and evaluation, we need to think outside the traditional test. This Twitter jokester gives us a clue on how to do this.

The test looks like a simple two-choice series of questions. However, the test-taker has the liberty of illustrating their answers. This provides insights into their mindsets, belief systems, and attitudes.

This makes such tests harder to quantify, but this is what changing the method entails. It is not just about increasing the efficiency of tests, it is also about being more effective in determining if, what, and how learning takes place.

Last week I volunteered for a school’s career guidance event for students. The students chose who they wanted to interview in half-hour intervals and we shared our hearts out.

A teacher inserted himself into my group’s discussion. As we meandered from one topic to another, that teacher declared this about his students: “They prefer pen and paper.”

No, they do not. Students have been conditioned to accepting them. The Principal of Change articulated this in a recent blog entry:

many of our students are so used to “school” that something outside the lines of what they know terrifies them just as much as any adult. If school has become a “checklist” for our students (through doing rubrics, graduation requirements, etc.), learning that focuses on creation and powerful connections to learning, not only take more effort, but more time, which sometimes frustrates many students.


The problem with... by horrigans, on Flickr
Creative Commons Creative Commons Attribution-Noncommercial 2.0 Generic License   by  horrigans 

There is an immediacy to paper which needs to be examined in more ways than one. If you have a notebook or scrap ready, there is quickness of use and illustration that is hard to beat. Some might also value the feel of paper as a form of immediate recognition.

But paper is also immediate in that what happens there immediately stops there. There is no hyperlinking or ease of editing, copying, and sharing. The affordances of paper, as wonderful as they are, also have limited pedagogy: Go where I say, stay where you are, write neatly, do not copy, do not share, etc.

Paper is comforting because it does not push pedagogy to new ground, so schools use a lot of it. Other than the newspaper industry, I do not know any other organization that relies on so much paper. OK, maybe the toilet paper industry.

Ultimately, schools justify the pen and paper mode of instruction because of the pen and paper exams. This is despite the increasingly non-pen and paper life and work that awaits learners when they move on.

Video source

Saying kids prefer pen and paper is like saying they would rather use encyclopedias instead of Google or Wikipedia. (BTW, they know the limitations of those tools better than teachers might expect).

Just ask enough of our kids the same questions the teens were asked in the video above. Their responses might not be as colourful as the ones in the video, but you are likely to get similar responses.

Like the video, you will get a few responses about paper-form books based on nostalgia and tactility. But these come from a place of honesty and passion, not from conditioning by a schooling system that refuses to change. These come from students who learn how to think by themselves instead of providing conditioned responses.

Do we assume nostalgia to be important enough not to change? Do teachers have a right to recreate their own world instead of preparing kids for theirs? I borrow another quote from The Principal of Change in the same blog entry:

if we do not challenge our students in the learning they do in school, what are we preparing them for? What mindset will we actually create in our students? It is important, if not crucial, to really listen and act upon student voice…

How much longer are we going to maintain conditions for “pen and paper” conditioned responses?

The press had a blast last Friday when the Singapore Examinations and Assessment Board (SEAB) announced that entire Primary School Leaving Examination (PSLE) papers from the last three years would be sold to the public.

The response from the public was predictable.

Entire GCE O and A level papers have been available for generations of students, so why did the SEAB do this after years of holding out? The official reason was to reduce over-preparation

However, this response creates more questions than answers. Has anyone asked and convincingly found out if the release of high stakes exam papers at the secondary and junior college levels has reduced over-preparation? If so, might a comparison of pre-teen and teenaged learners be valid?

Perhaps the move is a social engineering experiment which is part of a larger plan for changing the mindsets of stakeholders. The larger plan is a jigsaw puzzle of changes in the last three years:

  • the push to label every school a good school
  • not announcing the country’s top PSLE students
  • de-emphasizing the importance of grades and refocusing on values
  • the rise of vocational education
  • the posturing on the value of basic degrees

Social experiments take time and face serious obstacles.

This social experiment will take years to monitor even as we hope to gain from increased transparency and lowered stress of exam preparation. The ones that gain immediately from the release of full PSLE papers are publishers like Popular, the neighbourhood publishers (nudge-nudge, wink-wink), and test preparation centres. There is money to be made by meeting the demand for these papers and stoking the fear and anxiety that high-stakes exams bring.

A social experiment is not a scientific one because it is less controlled. Other ingredients in the pot, like kiasu parents, are confounding factors. Kiasuism is a staple of our diet and telling parents not to hot-house their kids is like telling them to stop eating rice.

Our kiasu mentality and tuition culture are formidable obstacles against change. The main weapon that leads the charge for change is transparency.

Being transparent means being to see through something. In this case, the exam question transparency allows for more open discussion about the quality of the questions. Schools, tuition centres, parents, and most importantly, students, will be able to better prepare for the exams.

The stress of taking a high-stakes exam will not go away. However, the stresses associated with over-preparation or under-preparation might be reduced.

In the long run, I would like to see the continued transparency and discussion leading to better questions. Not exam questions, but questions about the PSLE as a sorting mechanism when children are just 12-years-old. This transparency is a long-term solution to looking past symptoms of kiasuism and unnecessary tuition, and dealing with the root problems instead.

Ta Prohm Roots (view on black) by mendhak, on Flickr
Creative Commons Creative Commons Attribution-Share Alike 2.0 Generic License   by  mendhak 

How did I arrive at that conclusion?

Our kiasuism is a result of our notion of meritocracy which starts with how well our kids play the schooling game. Success is measured almost solely on test and exam results, hence the premium placed on playing the game well.

If parents feel that teachers and schools cannot help their kids succeed, they purchase that insurance or enable the result with tuition, be it remedial or enrichment or both. When kids do well on paper, this justifies the kiasuism and reinforces tuition-dependency. This perpetuates the processes of testing and private tuition.

Some people see excessive testing and unnecessary tuition as problems. They are not. They are symptoms of more insidious problems with conventional schooling.

One need only revisit recent history of test design and schooling to realize that they were designed for the industrial age. Industry is about mass production and standardization. It is driven by one-size-fits-all efficiency. While that has put cars on our roads, smartphones in our hands, and nearly identical Big Macs in our stomachs, the quality control based on standardization and efficiency suit machines and food but not people.

Put people through schools based on industrial processes and there will be kick back. Teachers struggle with limited production (curricular) time and mass treatment (large classes) because they are counterintuitive to development time as well as nurturing and coaching small groups or individuals. Mass implementation and standardization go against personalization and individualization.

Teachers eventually realize that their students are not products of teaching that undergo quality control (tests) at the end of production lines. But people are malleable and adapt to the circumstances. People learn to be products. Parents with the means outsource coaching and preparation so that their children pass those tests.

The problem is the design of schooling. Testing and tuition are merely symptoms. The transparency of how exams like the PSLE are designed could lead to the questioning of our quality control and sorting mechanisms. In concert with other systemic changes, this could lead to the questioning of the design and purpose of schooling.


Usage policy

%d bloggers like this: