Another dot in the blogosphere?

Posts Tagged ‘numbers


Video source

This video is timely given the misleading way some people use the efficacy numbers of different COVID-19 vaccines.

The efficacy of the a vaccine is not the same as its effectiveness. I recommend this NYT article for an explanation of how something like “95%” efficacy is derived.

Vaccine trial efficacy is not the same as real use effectiveness. A trial use of the vaccine includes a placebo for one sampled group of people and the vaccine for another group. Actual use only includes the vaccine and is applied across a much larger group of people.

The J&J vaccine trials were also conducted in South Africa and Brazil. Vox video (https://www.youtube.com/watch?v=K3odScka55A) on why the vaccine efficacy numbers cannot be compared.

The J&J trials were also conducted outside the USA — in South Africa and Brazil.

The J&J vaccine trial was conducted over a more severe infection period. Vox video (https://www.youtube.com/watch?v=K3odScka55A) on why the vaccine efficacy numbers cannot be compared.

The J&J vaccine trial was conducted over a more severe infection period.

Back to the video — it explains why efficacy numbers cannot be compared. For example, the Moderna trial was only in the USA. The Johnson & Johnson (J&J) trial also included countries outside the USA (Brazil and South Africa) where variants of SARS-CoV-2 emerged. It was also conducted over a more severe infection period compared to the Pfizer/BioNTech and Moderna trials.

Here is something the video did not point out. The Pfizer/BioNTech and Moderna vaccines have high efficacies after two doses. The J&J vaccine is a single-dose shot.

Screenshot of the range of outcomes after vaccination. From Vox video (https://www.youtube.com/watch?v=K3odScka55A) on why the vaccine efficacy numbers cannot be compared.

The range of outcomes after vaccination.

The video also highlighted that all the vaccines are not designed to absolutely prevent COVID-19 symptoms. If after vaccination people got mild to moderate symptoms, the vaccine is considered effective.

During the trials, all the vaccines mentioned in the video prevented hospitalisation and death among sampled participants. By that measure, all the vaccines were just as good. If we focus only on trial efficacy numbers, we lose sight of this more important outcome.

One general takeaway that applies in any problem-solving and policy-making is this: Numbers are a start, but they are not the end. The explanations and narratives that accompany them provide depth, nuance, and exceptions. If we do not go beyond the numbers, we risk misinforming ourselves and others.

One of the post-lockdown rules we have in Singapore is that groups of people should be no larger than five [1] [2].

So why did I spot a group of eight at an eatery over the weekend? My guess is that one group of four scored one table and the other group of four got another table right beside the first table.

People look for loopholes and take advantage of them. They know what the rules are, but they do not care why the rules exist. The rule-of-five is meant to reduce the number of people interacting physically while providing a sense of normalcy.
 

 
If you are taught to listen and comply, you hear the number. If you are not taught to think and care, you do know know why that number exists nor do you behave responsibly.

My reflection is not so much about how wilfully ignorant we can sometimes be. It is about how we condition that mindset by the way we teach.

If we rely on the pedagogy of answers, we tend to provide the facts and figures. But if we learn to use the pedagogy of questions, we model for our learners and we teach them to ask important questions. The most important of which is why.

What exactly does 1,320km of cycling paths mean? How does that compare with what we have as roads?

This photo was the second image accompanying this tweet from the LTA.

While I look forward to people depending less on cars and more pedal power, I wonder what exactly 1,320km of cycling paths means.

How does that compare with what we have as roads? What does that mean to commuters who might actually want to cycle? How connected and convenient will these paths be?

If cyclists, pedestrians, and other non-car commuters still have to contend with a car-dominant mentality, all the cycling paths in the world will not make a difference.

Numbers are easy to tout. They tend to be the first thing that administrators, policymakers, and leaders start with. But impressive as the numbers might be, we need to ask what those numbers mean.

By the same token, reporting that MOE lent 12,500 devices to students for home-based learning describes an effort. It was an important effort, but that does not answer or address the issue of WHY the conditions made that effort necessary.

As with most things, the important question is not about how much or how many. It is about finding out why.

Tags: , , ,

The local press is fond of highlighting the number of new COVID-19 cases each day. According to this data visualisation tool [desktop] [mobile], Singapore had 110 cases as of yesterday. Note: The data is from official sources in each country.

What the press conveniently leaves out from its headlines is the number of recoveries. According to the data, 78 people in Singapore have recovered and none have died as of yesterday. This makes our known pool of COVID-19 patients 32.

Focusing on the number of infections without also emphasising the recoveries feeds the fear. While this does not contravene any POFMA rule, it is still irresponsible behaviour.

POFMA might deal with misinformation and disinformation, but it can do nothing about partial information. We need to be better and do better. We might do this by uncovering the processes behind each product.

Screenshot of graph showing COVID-19 infections and recoveries worldwide.

When I first visited the data visualisation site almost a month ago, I did not notice this other visualisation of COVID-19 cases. The new cases (orange and yellow lines) are plateauing at the moment. The recoveries (in green) are on the rise.

The graphs might change if there are new cycles of infections, but the more complete use of numbers tells a more complete story.

As I view this again from the perspective of an educator, I return to product and process. Headlines and graphs are products. But these have underlying processes that need to be examined and critiqued. We remain ignorant if we take products at face value without demanding better processes that create them.

Insist on seeing the processes. Demand to shape them.

This tweet reminded me of how people in power play the numbers game to oversimplify complex issues.

In this case, the number (USD 5000) looked good because the mathematical average supported a predisposed conclusion. This is not how to analyse data or conduct research. The data might reveal one or more conclusions; bias or mindset should not dictate the analytical process.

One way to counter the misleading conclusion could be to use the median gain instead. This takes into account the number of beneficiaries and will reveal that there are many more ordinary folks receiving less than claimed.

Countering such a numbers game is relatively easy. Pundits on Twitter and news channels alike can make the same point as I have. But some activities-by-numbers are more insidious.
 

 
Consider the claim that the administrative load of teachers here has been reduced on average. However, they might have been lowered only from a policymaker’s or administrator’s spreadsheet, e.g., timetables, co-curricular duties, committee work, special projects, etc.

Such spreadsheets do not consider how school leaders and managers replace the “void” with smaller and more numerous tasks that do not look like administration. Consider how a teacher might be told to follow up on an event by writing a report, counsel a sensitive parent-student case, or chaperone an overseas trip.

All these can be quantitatively defined in a spreadsheet (if they are at all), but not qualitatively justified. A post-hoc report might involve gathering data, sorting though photographs, and drafting documents, all of which take more time than anticipated.

Dealing with a difficult parent and/or student can be emotionally draining and this affects all other work. One might bean-count a two-hour contact time and ignore the lasting effects of such an engagement.

The solutions for countering such a blind and cruel numbers game are not easy. They might include having empathetic leaders, conducting frank and open communication between teachers and their managers, and crafting policies that look into the quality of work and not just the quantity.

Many of our teachers a self-selecting because they are empathetic nurturers. They care for others, but in the process, forget to care for themselves. They do not care about the numbers game or know how to counter it when assaulted by unexpected responsibilities. Might their leaders and managers nurture these nurturers by not playing the numbers game?

Late last year I made a screen shot of this odd headline.

Ambiguous headline.

The headline was ambiguous. It was meant to highlight how a youth turned his life around. Instead it could also mean that he needed to be chemically induced to help others.

Words matter. The choice of words and how you order them helps you mean what you say. If only headline writers could say what they mean.

Now consider this passive-aggressive app rating. It was a complaint about the app disguised as a five-star rating.

This is how NOT to write a review.

Such a review is counter-productive. The bean counters only care about what works in a spreadsheet. They will not process the qualitative feedback.

Numbers matter, too. Your choice of numbers is a cryptic summary of what you value. You let others in on the secret by your choice of words. The numbers and words should be congruent. If there are inconsistencies, you lose the trust of your readers.

Tags: ,

If you operate in training in the corporate, governmental, or NGO worlds, then 80:20 or 70:20:10 makes sense to you.

To the rest of the world, particularly those in schooling and higher education, those ratios and what they stand for make about as much sense as the tweet above.

The issue is not so much the proportion of, say, sales and active customers, it is how the numbers came to be. When you discover that this “principle” has been applied to management, marketing, sales, and even life, you have to ask: What research helped establish these ratios?


If you dive into that rabbit hole, you might realise how shallow it is. It is a pothole that does not go anywhere. I would rather pave over that hole.

I did something similar in the past by pointing out how wrong the numbers were in Dale’s cone of experience. This critical resource describes how Dale did not put any numbers to his model and the numbers were made up.

It is tempting to try to make complex phenomena like “how people learn” simple. The mistake is trying to boil this down to numbers and formulae.

If you are going to play the numbers game, do it right. Embrace details and nuance.


Video source

An absolute person understands and operates by absolutes. A nuanced person looks for subtlety and context.

If both play the numbers game, the first person is going to look like an absolute fool. The second is going to look reasonable and logical.

Some people’s eyes glaze over when shown figures and statistics. Some hearts might harden as it is easy to focus on the numbers instead of considering the people that make up those numbers.

Case in point: The debate about the number of deaths resulting from Hurricane Maria last year.


Video source

This segment from the video above was one method to estimate the number of deaths in the aftermath the hurricane that hit Puerto Rico last year. But they might not mean much to the neutral or uninformed without a human face and story.


Video source

The journalist near the end of the news report took care to describe a couple he had followed from last year. The couple survived the initial onslaught, but the husband succumbed to “the stress (and) the fumes from the generator”.

Narratives do not just complement, they illustrate more richly. They connect readers, listeners, and watchers with what matters — the human condition.

Last Saturday, STonline reported the International Baccalaureate (IB) performances by Singapore schools. As usual, it featured pass rates and the number of perfect scores.

The local rag does this with our PSLEs, GCE O-Levels, and GCE A-Levels, so the article almost writes itself with a template. To be fair, the template has been updated to include human interest stories — the people behind the numbers — but these can seem like afterthought or filling newspaper space for a few days.

The IB result article fit the mould perfectly. It featured the usual suspects with the usual stellar results. Then it zoomed in on twins from the School of the Arts who got perfect scores.

What is wrong with doing this?

There is nothing wrong with human interest stories provided the children are not coerced into doing them and if the overcoming-the-odds stories inspire others.

What is wrong is the almost perverse fascination with quantitive results. It is one thing for schools and the Ministry of Education to keep track of these statistics, it is another to tout them and lead stories with them.
 

 
The health of our schooling system is not just measured by numbers. This would be like diagnosing sick patients by measuring only their temperatures and blood pressure. Even a layperson would say that stopping at such triage is irresponsible. The same could be said of the STonline reporting.

About five years ago, the MOE stopped revealing the names and schools of the top Primary School Leaving Examination (PSLE) students. It also discouraged the ranking and banding of schools into socially-engineered leagues in order to operate by its “every school a good school” principle. The move was meant to emphasise the holistic development of each child.

The IB results article and its ilk hold us back. Yes, the template includes human interest stories and background information about the IB. But the newspaper conveniently forgets or ignores that the IB practically an alternative form of assessment. From the article:

Founded in Geneva in 1968, the programme is now available in 4,783 schools in over 150 countries and territories.

IB diploma students take six subjects and Theory of Knowledge, a course that combines philosophy, religion and logical reasoning. They also learn a second language, write a 4,000-word essay and complete a community service project.

Why not focus on how the thinking and value systems are nurtured? What are the impacts of the community service projects on all stakeholders? How might the rest of the schooling system learn from the IB process? Finding these things out is not easy. Then again, nothing worth doing is easy. Using a writing template is easy.

STonline might think it has the perfect template for reporting academic results. It might. But this template has lost relevance given MOE policy changes. In emphasising the numbers game, it creates speed bumps and barriers in a schooling system that is trying to plod slowly forward.


Archives

Usage policy

%d bloggers like this: