Another dot in the blogosphere?

Posts Tagged ‘psle2021

Before the new PSLE scoring system was announced two weeks ago, it was described by Acting Education Minister (Schools) Ng Chee Meng in May as “no silver bullet”.

That cliché aptly describes the changes. So do “a mixed bag” and “to have your cake and eat it too”. In my sixth and final reflection on PSLE2021, I explain why the restructuring does not go far enough.

As if to pre-empt this line of argument, Mr Ng said:

“Some things are best evolved and not revolutionalised,” he said noting that Singapore’s education system is a strong and robust one as educators have done very well over the last 50 years in building a strong system.

The PSLE2021 is an evolution, not a revolution. Again, very apt.

The most important but undersold change is the switch from norm-based testing to criterion-based testing (see Part 1 of my reflection). However, the PSLE retains its summative testing and sorting nature. These counteract the messages of the restructure being less stressful, not being a source of competition, and focusing on the learner and learning (Part 2).

Those that study change and are familiar with the literature will describe the proposed changes as piecemeal. This contrasts with systemic change.

Piecemeal change is often top-down and tacked on to an existing system. It might make incremental improvements and it does not disrupt the status quo. That is why such change is evolutionary and not revolutionary.

Systemic change is often the opposite, although its leadership and sustainability can stem from a mix of top-down, bottom-up, and middle-up-and-down. Such change takes place by first identifying key leverage points of a system.

In schooling, one critical leverage point for systemic change is assessment. Change this and everything else has to change. It is the tail that wags the dog.

If the PSLE2021 was systemic, it could start with changes in the assessment at the end of Primary 6 — if there was one at all — and cascade changes to educational policies, curriculum, teaching methods, school support, stakeholder behaviours, and more.

Piecemeal change often leads to little appreciable change or no change at all.

The changes in PSLE2021 will not include curricula (see point 10 of this article). It is also relatively easy to get used to the Achievement Levels (ALs) since we were all schooled to think that way — they are like O-Level grades!

Teachers can keep drilling in the latter stages and tutors can keep “enriching”. Enrichment tuition centres need only replace their trophy heads’ grades with AL1s instead of As or A*s. Parents can keep pushing their kids to compete and subject them to hothousing.

Consider another example. With regard to the PSLE2021 changes, a school principal said:

…this would reduce the previous “pressure points” of comparing against peers and chasing the last few marks. Instead, the focus can be on grasping and having a “mastery over content”, and striving towards one’s personal best.

The change from conventional grades to ALs will do little to stop the paper chase. The ALs are not actionable because they are products of a terminal activity (Part 2).

Trawl what leaders in education are saying online about grading and you will see something like this emerge.

Quantitative grading ends learning. Quality feedback sustains learning.

It is possible to do very well in a test or exam by drill, rote, and formulaic thinking. It matters little if you have “mastery of content” if you do not hone thinking skills.

The changes in PSLE2021 have not been accompanied by changes in curriculum to address student thinking and skills, or professional development for teachers to teach differently.

For example, the curriculum is still designed to be learning about Mathematics or Science. It is hardly about learning to be or think like a mathematician or scientist. Some teachers want critical, creative, and independent learners, but they either do not know how to model or nurture these traits, or are not willing to let go.

So I am critical of the piecemeal change. The vision for change is not met by its currently proposed implementation. Mr Ng’s vision was:

…to move this school system forward so that we reduce the competitiveness of it, and encourage creativity and collaboration of succeeding together.

How is retaining and polishing an old and increasingly outdated assessment and sorting system an attempt to “encourage creativity and collaboration of succeeding together”? Creative answers are not encouraged and not appreciated in grading rubrics. There is are names for “collaboration” and “succeeding together” in exams — they are cheating and colluding.

If we remain rooted in the domain of summative assessment, we operate by its rules and language.

Here is another cliché: Fortune favours the brave. Did Finland worry when it implemented keyboarding over handwriting? Did it wonder what others would think about:

No. It focused on what its students need today and tomorrow. It takes care of its citizens so that they take care of themselves and their country.

I am not suggesting that we adopt Finland’s strategies wholesale because our schooling contexts might be different. Our schooling system already has so-called alternatives like DSA, e-portfolios, institutional entry tests, interviews, performances, and train-through like integrated programmes. Why not empower and support these more?

I say we put our money where our collective mouth is. If we say we must value creativity, innovation, critical thinking, and collaboration, then we must implement processes that nurture and measure these things.

I poke and prod some quotes from various stakeholders of PSLE2021. Links to source material of the quotes are in the headers.

Quote 1

Currently, the T-score (short for transformed score) reflects how well a student did in relation to others in the cohort — using a mathematical formula. A student may have got high marks for a subject, but would receive a lower T-score if most of his peers performed better than him.

This encapsulates the main issue with the current PSLE. It is a sorting model based on bell curve or normal distribution.

There is nothing wrong about making the assumption of normal distribution for a large population. What is doubtful is if a cohort of Primary 6 students is representative of that population.

Quote 2

MOE said the changes are part of a larger shift to nurture well-rounded individuals and move away from an over-emphasis on academic results. They will reduce fine differentiation of students – a key complaint of the current scoring system; reflect a student’s level of achievement regardless of how his peers have done; and encourage families to choose schools based on their suitability for the child’s learning needs, talents and interests.

This quote ticks all the right boxes, but we need to read in between the lines.

Student achievement as measured by standards or criteria instead of comparison with other students in the sample is a good move. I wonder if we have studied the USA’s implementation of Common Core and its testing regimes.

The kickback there is how testing has affected curriculum, restricted teaching, influenced teacher appraisals, and increased stress levels of stakeholders. The only ones that seem to have benefitted from the programme are test companies.

As for the desired change in parental mindsets, read one example in the quote below.

Quote 3

This was also an issue raised by Jean Lim, a former teacher with more than 30 years’ experience. “In the past, if a student scored 75, we could tell parents that their child scored an A, and there were happy. But now, if they score an AL4, which is still considered an A in the old system, they will not be happy, because an AL4 just doesn’t sound as nice,” she said.

The message that the focus will be on the learner and learning will fall on deaf ears if PSLE2021 comes across as only about changes in scoring and school selection.

The current PSLE regime has created a cultural monster that feeds on kiasu-ism and is fueled by enrichment tuition for competition. Numbers like T-scores, aggregates, and cut-off points are the well-understood rules.

We wait with bated breath on what MOE and schools will do to deal with these. If they take action, we do not need more dialogues on what PSLE2021 means. We can read and think. We need MOE and schools to listen and reflect first.

Quote 4

Time needed for parents to change mindset of chasing ‘good schools’

This was an awful title for a forum letter. It is not just time that will change mindsets, as if the influence is somehow automatic. It will take a lot of concerted effort.

There was a plan for the original PSLE in 1960. It changed over time, but it was people that communicated, forced policies through, and implemented the regime. Once enculturated, the PSLE took a life of its own when schools and parents responded to the increased stakes and competition with hot-housing and tuition.

Better than the headline was conclusion of the letter:

The greatest change that this new system is supposed to elicit is a mindset change. With the clock ticking away from now until 2021, more things can be done by schools and the Education Ministry to alleviate the fear and uncertainty that parents feel, to help them have more confidence in the new system.

Quote 5

It depends on what is best for her, not what the best school is,” said Ms Ho. “Ultimately, you want your child to grow up to be a good person with good character, good morals and if you’re always focusing on the academics, you will miss out on other things.

If I was facilitating a change management effort, this quote would be an integral part of the visioning process. Change agents need to visualise what they want to achieve otherwise they will be running blindly.

We need need more parents with this perspective. MOE needs more parents with this perspective.

There already are some who have this mindset. How many are there? What is MOE going to do take advantage of this?

I am being realistic, not blindly optimistic, about the changes in and around PSLE2021. It is piecemeal change, not systemic change. It is evolutionary change, not revolutionary change. It is not enough. More thoughts on this in Part 6 tomorrow.

The first three parts of my reflections on PSLE2021 was like reviewing the good, the bad, and the ugly.

  • Part 1: The good change is the move to criterion-based testing
  • Part 2: The bad is that the assessment is still summative
  • Part 3: The ugly is how T-score differentiation goes away only to be replaced by course granularity

 

 
Most people know how the current A grade in the PSLE spans scores of 75 or more. They have pointed out how the new Achievement Levels 1 to 4 will be equivalent to the current A.

The concern seems to be that the old A was attainable at 75 while straight As (75s) under the new scheme results in four AL4s and aggregate of 16. The new aggregate will not look and feel as pretty.

Others have focused on the disparity of score spans for each AL. I illustrate the score spans for each AL in the table below.
 

AL Raw score range Score span
1 ≥90 11
2 85 to 89 5
3 80 to 84 5
4 75 to 79 5
5 65 to 74 10
6 45 to 64 20
7 20 to 44 25
8 <20 19

 
But those who think this way are missing the point.

Not only do the ALs try to introduce some granularity to the grades, I speculate that they are an attempt to 1) prevent grade inflation, and 2) insidiously reintroduce the bell curve.

Grade inflation is the ease with which students get an A or even an A* for each of their examinable subjects. It is more commonly referred to in the context of school, university, or workplace admission offices. The people who work here help decide which students get entry and they struggle to distinguish between numerous diplomas filled with straight As.

This is the source of the snippet on grade inflation that I tweeted last year.

The “finer” grained ALs help separate the good As from the not so good As. This punctures grade inflation, and very likely, egos and morales too.
 

 
In Part 1 of this series, I wrote about how the future standards or criterion-based testing was better than the current norm-referenced testing. I described it as an important fundamental shift in the PSLE. Implemented well, the focus could shift centrally to the learner and learning instead of focusing on sorting.

However, administrators and policymakers like “God views” of their system. Reducing people to numbers, data points, statistics, and diagrams are their work (and could be their idea of fun). The bell curve is too sexy to let go because phenomena only seem normal if there is a normal distribution.

Things seem neater and safer under the umbrella of a bell curve. You can be sure that one or more groups have crunched numbers with existing data to see if the ALs might insidiously recreate a normal distribution.

With some logical guesswork, you might see how this pattern might emerge as well.

It is a fairly safe assumption to say that many kids taking PSLE have be “tuitioned” and/or drilled in school. Quite a few will get As. The ALs 1 to 4 will spread them out: There will be fewer AL1s than AL4s. The curve draws itself with greater granularity.

TL;DR? The uneven AL bands in PSLE might not just be for increasing the granularity of measuring achievement. It might actually help administrators and policymakers prevent A-grade inflation and recreate the bell curve.

In Part 1 of my analysis of the new PSLE assessment system, I highlighted the important fundamental switch from norm-referenced testing to criterion-referenced testing.

In Part 2, I described how the summative design and implementation of the high-stakes exam counters positive change.

In this part, I reflect on MOE’s and the public’s obsession with differentiation when we all should be more concerned about granularity.

The current PSLE uses transformed scores (T-scores) and their aggregates as outcome measures for the exam. There are at least three major problems with doing this.

  • The scores can be normalised (see Part 1) and this process is not transparent to the students or the public.
  • A score might indicate where a student stands relative to his or her peers, but it does not indicate what that number means. The student does not know what areas of learning need to be addressed because the exam papers are not returned and there is no feedback loop. This is typical of summative assessment (see Part 2).
  • The aggregate scores seem to finely differentiate students who are competing for places in Secondary schools. For example, a student with an aggregate score of 221 gets in, but another with 220 does not. Someone sets an entry score, but no one can really explain why that benchmark is and what it means. That is one way we play the cruel numbers game.

PSLE2021 is supposed to take away that differentiation because exam papers are graded with Achievement Levels (ALs). Students are assessed on four academic subjects and each subject can be graded from AL1 to AL8. This results in 29 discrete categories of aggregate AL scores of 4 to 32. There is a better, but still not adequate, granularity of scores.

It is no surprise that other bloggers and their mothers have pointed out that the competition will now be for low aggregate scores. Tuition centres might tweak their marketing material to focus on lowering AL scores.

The schooling arms race used to be to get the highest aggregate T-score possible. The next battle is to get the lowest aggregate AL score possible. This is like a Pokémon Go game, just not as fun. You do not want to collect all the scores. You only want the very rare AL1 Pokémon.
 

 
The “new” PSLE does not change or break the summative assessment mould. It is still a sorting tool. With the ordered choice of Secondary schools by students playing a more important role than the current model, it is being tweaked to be a decision-making tool.

If the focus is on student achievement and learning — as claimed by the MOE PSLE2021 microsite and echoed the press — students need even greater granularity. By this I am not referring to exact scores of each paper, although this will provide more coarse insight. I am also referring to feedback and remediation on areas of weakness.

At the risk of painting a overly simplistic dichotomy, we have these divergent paths:

  1. Summative assessment model, T-scores, fine differentiation for the purpose of sorting.
  2. Alternative assessment model, high granularity for the purpose of meaningful learning.

Our MOE seem to be designing a hybrid, at least on paper. In its wish list is: Summative assessment, some granularity, a focus on learning. This is a very elusive Pokémon, it is exists.

What paths will we take? What game will we play? What is at stake?

In Part 1 of my reflection on PSLE2021, I elaborated on why the move from norm-referenced testing (NRT) to criterion-referenced testing (CRT) was a fundamental shift. It could set the tone for the desired change from unhealthy competition and comparisons to a focus on individual achievement. To the latter end, PSLE2021 will have Achievement Levels (ALs) 1 to 8 instead of scores.

To claim that ALs will help students know where they stand is one thing. To say that ALs will help students focus learning is another.

TODAYonline claimed that the new system will “help children focus on learning instead of marks”. It mirrored the official message at MOE’s PSLE microsite.

The current PSLE does not help children focus on learning. The PSLE2021 with its ALs does not guarantee that either. This is because the:

  • PSLE is summative
  • Preparation for exams discourages it
  • Comparison and competition lead to stress
  • ALs are not actionable

Summative nature of PSLE
The ALs are part of summative assessment, which typically happens at the end of a course of study. Summative assessment is sometimes referred to as assessment of learning (AoL). The PSLE ALs should indicate what the student has learnt at the end of Primary school.

Summative assessments tend to focus on the products of learning, e.g., grades or ALs, instead of processes of learning.

The PSLE in its old or new form is like the quality control near the end of a production line. It rates the product of schooling (fail or pass; if pass, then how well) and determines where it goes next (to recycle or discard; to channel to which Secondary stream).

Exam preparation
With so much at stake, kids are drilled as the PSLE draws near. However, there is much to take in because an entire Primary schooling experience boils down to exam papers.

Students learn to game a system that is stacked against them. They learn shortcuts from their classrooms and tuition centres. Memorisation is key because it is the path of least resistance and learning is for the short term. The time-honoured strategy is GIGO — garbage in, garbage out.

Comparison, competition, stress
ALs will not stop comparison and competition. This is what creates the unnecessary PSLE stress.

At least two news agencies [1] [2] explored the effect of stress. One parent reportedly said that “students will not have to stress about having to outscore their peers under the new system”. I disagree.

Children are taught to compare by adults. They are conditioned in the classroom, home, and other social environments to do this. Students find out how their peers have done. Parents compare notes because they realise their children compete for limited places in their Secondary school of choice. As a result, students are told they can do better than themselves and someone else.

Enrichment tuition centres already take advantage the desire to compare and compete. The tuition industry will tweak its marketing messages and promise better ALs instead of better letter grades or scores.

ALs are not going to remove comparisons.

ALs are not going to stop enrichment tuition as the chase will be for AL1.

There is a simpler way to examine this issue. Some have remarked on the similarity of ALs to O-level grades [1] [2]. Is there less or no competition at the Singapore-Cambridge O-Level Examinations?

Reality bites and at least one parent put it plainly in this report:

Ms Deborah Giam, whose daughter is a student at Methodist Girls’ School, agreed it will be fairer to judge a student’s own performance but also felt the PSLE will “continue to be a source of stress”. “In the case of the T-score, a lot of it was out of people’s control … so all you can do is always push your kid to do the best that they can. With this new system, it’s still the same because … at end of the day, it’s still about how many points you score,” she said.

ALs are not actionable
The PSLE is taken at the end of a child’s Primary school experience. What difference can a student make if he or she has an AL4 in an academic subject? It is not as if there are any lessons of note after the examinations. The papers are not returned to students so that they can address what they have not mastered.

Short of retaking the PSLE, a student cannot try to get a better AL. The Secondary schools students are posted to are also not necessarily going to take note of the AL and remediate.

PSLE2021 Yoda.

The fictitious Master Yoda once said, “Fear is the path to the dark side. Fear leads to anger. Anger leads to hate. Hate leads to suffering.”

If there was an Edu Yoda, he might say, “Summative assessment is the path to the Dark Side of PSLE2021. ALs lead to comparisons. Comparisons lead to competition. Competition leads to stress.”

There is a lot of noise amidst the signals of PSLE2021. Strip away the noise and you might detect the important signals.

One signal is the shift towards on standards or criteria of learning. The focus is the learner and the mastery of learning. The other is the fact that the design and implementation of PSLE2021 is still summative. The result of this is still quality control and sorting, and with it the socio-cultural baggage of comparing, competing, and unnecessary stress.

The two signals do not harmonise. One hints at a new tune, the other repeats an old refrain.

If I can keep my figurative noise-cancelling headphones on long enough, I will pick on a few more PSLE2021 notes in more reflections.

Read Part 3: Differentiation vs granularity.

The news that caused ripples in Singapore schooling last week was the official announcement from the Ministry of Education (MOE) of the new scoring system that will be implemented in the Primary School Leaving Examination (PSLE) in 2021.

There was a slew of news following the announcement. Some people made tsunamis out of the ripples, some rode the waves as they were [small sample of both].

Beneath the surface was an undercurrent that did not get much attention, but was the most significant change in terms of education. According to STonline, one of the changes was the switch from norm-referenced testing (NRT) to standards or criterion-referenced testing (CRT).

PSLE2021: From NRT to CRT

What are NRT and CRT in layman terms? Why is the switch an important driver of change?

In NRT, the results of a cohort of students are reduced to scores — T-scores in the case of PSLE — and lined up from the highest to the lowest (or vice versa). The result is a bell-shaped curve of scores: There will be a few very low and very high scores, and many somewhere-in-the-middle ones.
 

 
Reviewers of these scores typically use this distribution to create an even curve (a normal distribution, ND), and to rank and sort. In the adult world of work, this method might help determine who gets promotions or bonuses, what appraisal grade you get (if you are in the civil service), or who gets fired.

For example, a large organisation can first rank the performances of all its employees. If an ideal ND does not result, it can statistically massage it into an ideal bell curve. So if there are too many A-graders, some will be pushed into Bs, and as a result Bs become Cs and so forth. Once there is an ideal bell curve, someone can decide cut-offs and consequences, say, the top 5% get promotions and the bottom 15% are let go.

If this seems unfair to working adults, then what more for the 12-year-old children who take the PSLE but have no idea what is going on?

The core problem is that people are compared one against the other with or without their knowledge. If with, this can result in unhealthy competition because they want to be on the right part of the ND curve. If without, the people become victims of processes not transparent to them and circumstances beyond their control.

Is there a better way? Yes, it is called CRT (standard-based assessment and/or evaluation).

Modern corporations like Accenture are abandoning the outdated practice of norm-referencing [1] [2] and embracing comparisons of one. The fundamental principle is this: How one improves and contributes individually over time is more important than how one is measured against others.

For example, a worker might show evidence of specific skills that indicate that he or she is a novice, intermediate, or advanced worker. There is no comparing of all the workers regardless of their skill group or even comparing within each skill group.

To make this work, there must be standards or criteria that identify each skill group, e.g., skills A to J for novices to master; K to R for intermediates; S to Z for advanced plus five potential managerial markers.

Back to PSLE 2021. The switch is from NRT to CRT. It is more about the standards or specific criteria that indicate the test-based achievements of the child, and less about the comparison of one child with another.

This is a fundamental shift in mindset from sifting and sorting to measuring performance. The former is about what is good for the system and how to feed it; the latter is about where the learner is at and what is good for the learner.

However, this piecemeal change of the CRT system of academic levels (ALs) still falls short. I share thoughts on these in more reflections on PSLE2021 over the next few days.

Read Part 2: The Dark Side.


http://edublogawards.com/files/2012/11/finalistlifetime-1lds82x.png
http://edublogawards.com/2010awards/best-elearning-corporate-education-edublog-2010/

Click to see all the nominees!

QR code


Get a mobile QR code app to figure out what this means!

Archives

Usage policy

%d bloggers like this: