Another dot in the blogosphere?

Posts Tagged ‘testing

This is an example of doing the same thing differently: Taking exams online instead of in-person and being subject to algorithmic proctoring.

What is the problem with algorithmic proctoring? The rules, regulations, and repercussions are worse than conventional proctoring. But don’t take my word for it, read this piece by the Electronic Frontier Foundation, Proctoring Apps Subject Students to Unnecessary Surveillance.

I concluded yesterday’s reflection with Alfie Kohn’s call to take our “chance to turn a(n) epidemiological crisis into an educational opportunity”. A testing vendor took the opportunity to reinforce the status quo, i.e., the old and increasingly irrelevant way of holding students accountable by first holding them captive. Progressive educators who focus on learners and learning can do better because we know better. We could start by rejecting such processes and products.

Crash Course provided a ten-part series called Navigating Digital Information. But what good is claiming to be information literate if you cannot prove it?

Video source

This TED-Ed video is a quick test on applying some of that knowledge by evaluating misleading headlines.

The video title states that this test is Level 1. So will there be more difficult tests?

We live in testing times. Not just politically or environmentally, but also in terms of actual tests.

So here is a basic tip with multiple-choice questions like the one above: Use LETTERS as options instead of numbers.

We live in testing times. Literally.

Tests and exams are a fact of life in school, so teachers have devised strategies like drill-and-practice to help their students cope over the short term.

Any strategy that helps students actually do better in tests is good, right? So how about creating some metacognitive awareness?

Video source

The video above highlights how students might procrastinate and sabotage themselves during test preparation. The psychology behind this is behavioural and claimed self-handicapping.

Why do learners sleep late, not study, or distract themselves even though their teachers might have taught them test-taking strategies?

According to the video, students anticipate failure or poor performance, and create an excuse in advance. This way they can blame something else other than ability.

Test-taking is stressful enough. Other than reassuring students than academic tests are a narrow measure of worth, parents and teachers should nurture student self-awareness as a metacognitive strategy.

We live in testing times, not least because of people like Trump and the consequences of their thoughtlessness.

Last week, the local press bragged about how Singapore universities were moving towards electronic examinations.

This sounds oh-so-progressive until you read excerpts like:

  • “laptops to replace pen-and-paper exams because students are losing the ability to write by hand”
  • “online exams save paper”
  • “efficiency in distribution of exam papers, marking and collating results”

The reasons for changing the medium of exams were relatively superficial. Legibility of writing and saving paper are natural shifts in switching media. That is like saying switching from a bicycle to a plane lets you travel further and faster, and allows you to have a bird’s eye view. Of course you would!

There was no mention of how switching to electronic forms was not only more aligned with how we consume media today and how many students take their notes. The latter, in turn, is linked to the practice medium matching the task medium. If you do not understand the last point, consider a common response from teachers: Why should we use computers when students still have to take exams with papers and pens?

“Efficient” or “efficiency” was mentioned at least four times in the short article. Apparently, more effective ways of measuring learning were not on the radar.

The paper claimed that universities were “adopting more creative ways of assessment
 audio or video segments, and interactive charts and graphics”. Again, that those are functions of richer media.

But can students also respond in equally creative and critical ways? Apparently not since “the students will have a ‘lock-down browser mode’ to prevent cheating, which cuts access to the Internet”.

Those that prepare the e-exams would rather set the same type of lower level Google-able, app-solvable questions than to change their methods and set unGoogle-able questions or tasks instead.

I said it in my tweet and I will say it again: This is a change in exam media, but not a shift in method or mindset.

Still on the topic of tests, I tweeted a WaPo article last night.

TLDR? Here is a comic I found in 2014 that summarises the take home message.

Tests. I can take tests.

The WaPo article did an excellent review of a national exam in the USA and tested the test with the help of three researchers. The researchers were experts in the content area of the test (history) and of assessment in general.

The researchers found that the tests only functioned to teach test-takers how to take tests. The questions did not necessarily test critical thinking skills like:

  • “explain points of view”
  • “weigh and judge different views of the past,” and
  • “develop sound generalizations and defend these generalizations with persuasive arguments”

Those tests were also going electronic or online. But again the change in medium was apparent; the change in method was not.

If we are going to design better forms of assessment and evaluation, we need to think outside the traditional test. This Twitter jokester gives us a clue on how to do this.

The test looks like a simple two-choice series of questions. However, the test-taker has the liberty of illustrating their answers. This provides insights into their mindsets, belief systems, and attitudes.

This makes such tests harder to quantify, but this is what changing the method entails. It is not just about increasing the efficiency of tests, it is also about being more effective in determining if, what, and how learning takes place.

Two recent reads articulated what I sometimes struggle to put into words: What seems to work in schools is sometimes an illusion.

I elaborate on the first today, an Edsurge article, that explained how much “education” research is based on flawed designs.

One example was how interventions are compared to lectures or didactic teaching. With the baseline for comparison so low, it was (and still is) easy to show how anything else could work better.

Then there is the false dichotomy of H0 (null hypothesis) and H1 (hypothesis). The conventional wisdom is that if can prove that H0 is false, then H1 is true. This is not the case because you might be ignoring other contributing or causal agents.

Finally, if there is no significant difference (NSD) between a control and the new intervention, then the intervention is judged to be just as good. Why is it not just as bad?

This makes it easy for unscrupulous edtech vendors to sell their wares by fooling administrators and decision-makers with a numbers game.

There was something else that the article skimmed on that was just as important.

This graph was the hook of the article. If the data are correct, then the number of movies that Nicholas Cage appeared inform 1999 to 2009 eerily correlates with the number of swimming pool drownings during the same period.

No one in their right minds would say that Cage being in movies caused those drownings (or vice versa). Such a causal link is ridiculous. What we have is a correlation of unrelated phenomena.

However, just about anything can be correlated if you have many sources and large corpuses of data. So someone can find a correlation between a product use and better grades. But doing this ignores other possible causes like changes in mindsets, expectations, or behaviours of stakeholders.

So what are educators, decision-makers, and concerned researchers to do? The article recommends a three-pronged approach:

  1. Recognise that null hypothesis significance testing does not provide all the information that you need.
  2. Instead of NSD comparisons, seek work that explains the practical impacts of strategies and tools.
  3. Instead of relying on studies that obscure by “averaging”, seek those that describe how the intervention works across different students and/or contexts.

This is good advice because it saves money, invests in informed decision-making, and prevents implementation heartache.

I have seen far too many edtech ventures fail or lose steam in schools not just because the old ways accommodate and neutralise the new ones. They stutter from the start because flawed decisions are made by relying on flawed studies. Pulling the wool away from our eyes is long overdue.

If this tweet was a statement in a sermon, I would say amen to that.

Teachers, examiners, and adminstrators disallow and fear technology because doing what has always been done is just more comfortable and easier.

Students are forced to travel back in time and not use today’s technologies in order to take tests that measure a small aspect of their worth. They bear with this burden because their parents and teachers tell them they must get good grades. To some extent that is true as they attempt to move from one level or institution to another.

But employers and even universities are not just looking for grades. When students interact with their peers and the world around them, they learn that character, reputation, and other fuzzy traits not measured in exams are just as important, if not more so.

Tests are losing relevance in more ways than one. They are not in sync with the times and they do not measure what we really need.

In an assessment and evaluation Ice Age, there is cold comfort in the slowness of change. There is also money to be made from everything that leads up to testing, the testing itself, and the certification that follows.

Like a glacier, assessment systems change so slowly that most of us cannot perceive any movement. But move they do. Some glaciers might even be melting in the heat of performance evaluations, e-portfolios, and exams where students are allowed to Google.

We can either wait the Ice Age out or warm up to the process of change.

By reading what thought leaders share every day and by blogging, I bring my magnifying glass to examine issues and create hotspots. By facilitating courses in teacher education I hope to bring fuel, heat, and oxygen to light little fires where I can.

What are you going to do in 2014?

I finally read a tab I had open for about a week: A teacher’s troubling account of giving a 106-question standardized test to 11 year olds.

This Washington Post blog entry provided a blow-by-blow account of some terrible test questions and an editorial on the effects of such testing. Here are the questions the article raised:

  • What is the purpose of these tests?
  • Are they culturally biased?
  • Are they useful for teaching and learning?
  • How has the frequency and quantity of testing increased?
  • Does testing reduce learning opportunities?
  • How can testing harm students?
  • How can testing harm teachers?
  • Do we have to?

The article was a thought-provoking piece that asked several good questions. Whether or not you agree with the answers is moot. The point is to question questionable testing practices.

I thought this might be a perfect case study of what a poorly designed test looks like and what its short-term impact on learning, learners, and educators might be.

The long term impact of bad testing (and even just testing) is clear in a society like Singapore. We have teach-to-the-test teachers, test-smart students, and grade-oriented parents. We have tuition not for those that need it but for those who are chasing perfect grades. And meaningful learning takes a back seat or is pushed out of the speeding car of academic achievement.

We live in testing times indeed!

I was thinking about how testing and grading were contributing to things like competitive tuition syndrome here and the race to the bottom in the US.

I was also wondering why politicians and policymakers were hitting the panic button when students in their countries did not do well in international tests like TIMMS and PISA. Was it really possible to draw a straight line from grades to economic success?

I seriously doubted it as there are many more important factors that contribute to the well-being of a country. Put another way, who cares if your students are not test-smart but are world leaders and world beaters in various fields?

Video source

I gained some perspective when I watched the closing ISTE 2012 keynote by Yong Zhao. The video is long, but the important bit starts at the 54-minute mark.

In short, Yong Zhao illustrated how there was a negative correlation between test scores like PISA and entrepreneurial indicators. A country whose students did well in tests would not guarantee economic success.

But correlations do not explain phenomena. Low test scores do not cause high entrepreneurial capability. The numbers do not reveal truths, but neither do they lie. They merely hide deeper issues that need to be explored and explained.

Yong Zhao did this by asking and answering three questions in his keynote:

  1. What matters more? Test scores or confidence?
  2. Are you tolerant of talent? Do you allow it to exist? Do you support it?
  3. Are you taking advantage of the resources you have? Or do you impoverish yourself in the pursuit of test scores?

I think he had one statement that practically addressed all three questions. In explaining why US students did poorly in tests but well on the world economic stage, he said:

Creativity cannot be taught… but it can be killed. American schools don’t teach it better. We kill it less successfully.
Video source

Our schooling system pins creativity to the ground and mindless tuition applies the coup de grĂące.

I wish we could be less successful killers of creativity too…

This article begins with an intriguing question: When is a test not a test? It cites a tweet by @Scott_E_Benson:

Then it dances around the benefits and pitfalls of tests before suggesting how one might assess and evaluate without the tests that we are most familiar with.

It suggests gamification and gaming strategies. It suggests portfolios, self-assessment, and peer accountability. It suggests measures that are more progressive than the quality control tests that are relevant only for the industrial age.

Thinking gamers might tell you that they are being tested all the time but the tests do not feel like tests. That is when a test is not like a test.

Video source

A game can be pure fantasy, be based on reality, or be a hybrid like the one featured above. Unlike a most video games, the game does not have obvious quests and thus mirrors much of life.

It is also been said that, unlike school, life throws tests at you whether or not you are ready. When that happens, you experience a knowledge gap and you need to problem-solve. That makes the seeking, analysis, and use of information relevant.

Despite the surprises that life throws at you, this form of insidious testing seems natural. School-based testing does not.

Like other creatures in the animal kingdom, we start learning by play. Why not be tested by play?

Click to see all the nominees!

QR code

Get a mobile QR code app to figure out what this means!

My tweets


Usage policy

%d bloggers like this: