Posts Tagged ‘assessment’
TV show ratings and assessment
Posted on: March 26, 2013
When I read this Wired article, The Nielsen Family is Dead, I saw parallels in the de-emphasis and modifications to Nielsen ratings for TV and assessment in schools.
Here is something from the article to chew on:
Since the 1970s, television has been ruled by the Nielsen Family—25,000 households whose TV habits collectively provide a statistical snapshot of a nation’s viewing behavior. Over the years, the Nielsen rating has been tweaked, but it still serves one fundamental purpose: To gauge how many people are watching a given show on a conventional television set. But that’s not how we watch any more. Hulu, Netflix, Apple TV, Amazon Prime, Roku, iTunes, smartphone, tablet—none of these platforms or devices are reflected in the Nielsen rating.
To borrow from the structure of the article, the dominant and traditional assessment system has been tweaked, but it still serves one fundamental purpose: to gauge how many student are paying attention in class and able to regurgitate information or perform drills.
This is not the only way we learn any more. Google, Khan Academy, Coursera, iTunes U, YouTube Edu, TED-Ed, smartphones, slate PCs — almost none of the skills and values that result from leveraging on these resources appear on the radar of traditional assessment.
At the risk of sounding like a TV programme rerun, I have to say teachers and textbooks are no longer the fountains of knowledge that students must make their way to and drink from in a particular way and at a particular time.
Later in the article I read this:
Nielsen and others have been scrambling to generate a new kind of TV rating, one that takes into account all of the activity that occurs on screens other than a television.
One such “new kind of TV rating” is social media-based metrics.
Are schools scrambling to generate new ways to measure learning? Ways that take into account the learning that occurs in situations outside the classroom for example?
We do not learn just because someone talks to us or attempts to teach us in a formal context. Sometimes we learn despite our schooling, not because of it. Why are we not trying to measure learning as and where it happens?
Measuring these elements is difficult. Thinking of alternatives is difficult. Creating buy-in for alternatives is difficult. But often the things most worth doing are difficult.
Really good reads
Posted on: September 2, 2012
Last week’s finds provided some really good food for thought.
Humanizing Our Organizations Through Social Media was a great reminder on why institutes of higher learning should leverage on Facebook and Twitter.
Higher Ed and the Monastic Space provided a perspective on how to better use the face-to-face time in classes.
Measuring what you contribute rather than what you collect, my favourite of the lot, provided some concrete examples of so-called alternative assessments. I think they should be mainstream assessments as they are relevant now.
So as not to get mentally fat, I plan on acting on what I consumed!
Grade less, learn more
Posted on: June 21, 2012
Walking about in a mall, I saw a promotion to Pay Less, Taste More. Our Ministry of Education advocates Teach Less, Learn More.
After reading these three articles and reflections, I think we could and should Grade Less, Learn More.
- You Should Probably Just Grade Less
- Instead of Standardized Tests, Why Not Build the World Wide Web?
- My own archived blog entry Question The Model
The first piece by Bud the Teacher started the ball rolling . In his own words:
For starters, I think teachers, in general, grade too many things. So one way to streamline would be to “grade” less. And that doesn’t mean that teachers shouldn’t ask students to write, and write often. But we don’t need to grade everything that comes to us. In fact, we should grade very little of it.
In the context of writing, Bud went on to say that teachers have built up the wrong expectations that they are their students’ only audience and that they alone evaluate what that writing is worth.
He argued that technology should not be the means of streamlining grading when the problem lay with the assumptions, design, and expectations of grading.
Put another way, the questions he might have asked could include: Why are students writing for just one person? Why not leverage on authentic and public writing spheres? Why not approach the task as a fellow reader and learner in order to critique instead of just grade?
Cathy Davidson wrote the second article and she was as thought-provoking as always.
She started by describing seven reasons why standardized testing, while great for grading, are not that great for evaluating learning. So she suggested a key alternative: Get learners to create by learning how to code.
I do not think that all learners need to code, but I do think that all learners need to be given the freedom and opportunity to express themselves and to create. Only then are they putting theory into practice. Only then are they making mistakes. Only then are they really trying.
As a result of reading these and other articles, I had to ask myself if I had reflected on these issues before. I had in the third piece.
Two years ago, I had concluded that modernizing a problem (e.g., creating grading software) was not the solution. We might create more intelligent systems, but the problem of testing irrelevant skills in irrelevant ways would still remain.
No, the solution lies in questioning the established but problematic model. It starts with accepting that grading more sometimes leads to learning less.
Grading takes time and students do not get timely feedback. Students do not pick up as much content or as many skills as teachers wish. Teachers do all they can to keep up with the grading load and do not have the bandwidth to learn new strategies.
A solution might lie in figuring out how to grade less so that everyone learns more. By grading less I do not mean asking for lazier teachers. We need teachers who question, learn, and act on what they realize is better for their learners.
Teachers could grade less by redesigning tests and focusing on key things that need grading. They could leverage on their learners’ need to communicate, create, and critique. Teachers could model these life skills and teach their students to peer assess in authentic environments.
As they do this, I bet they would learn more about what their learners are capable of and what new value they as teachers bring to the table.
Insidious testing
Posted on: May 27, 2012
This article begins with an intriguing question: When is a test not a test? It cites a tweet by @Scott_E_Benson:
The future of testing will be tests that students, teachers and parents do not think of as tests.—
Scott Benson (@Scott_E_Benson) April 26, 2012
Then it dances around the benefits and pitfalls of tests before suggesting how one might assess and evaluate without the tests that we are most familiar with.
It suggests gamification and gaming strategies. It suggests portfolios, self-assessment, and peer accountability. It suggests measures that are more progressive than the quality control tests that are relevant only for the industrial age.
Thinking gamers might tell you that they are being tested all the time but the tests do not feel like tests. That is when a test is not like a test.
A game can be pure fantasy, be based on reality, or be a hybrid like the one featured above. Unlike a most video games, the game does not have obvious quests and thus mirrors much of life.
It is also been said that, unlike school, life throws tests at you whether or not you are ready. When that happens, you experience a knowledge gap and you need to problem-solve. That makes the seeking, analysis, and use of information relevant.
Despite the surprises that life throws at you, this form of insidious testing seems natural. School-based testing does not.
Like other creatures in the animal kingdom, we start learning by play. Why not be tested by play?
Two individuals have been debating in a local paper the merits and demerits of severing Singapore’s link with the GCE examination system.
The first person [PDF] suggested that we have our own examination system. The responder [PDF] gave the usual “if it ain’t broke, don’t fix it” reply.
Frankly, I think that the first person did not go far enough!
How about not just having our own brand of exams but breaking out of our outdated mode of assessment? Like this Chronicle article, I think that we should stop telling our students to study for (the sake of) exams. On a related note, this father wishes that he could tell his kids not to just study for exams [PDF].
We have set our system up like an unpleasant game where students try to move up from one level to another by defeating boss exams at the end of each level. Unlike a real video game, kids do not want to play the exam game willingly.
I am not saying that learning should not be hard. It already is. I am saying that we should not be learning just for the sake of exams. While that may be the point of schooling, it is not the purpose of education.
School is meant to prepare students for real life, but the exams rarely reflect real life. They do not set the child on the path to enjoyable or meaningful lifelong learning. They rely on paper and pen(cil) in an electronic world. They require you to work alone.
In the real world you can ask for help or collaborate. In the exam world that is cheating. In the real and e-assisted world, you can connect with others and content. Often you can do that faster than you can with the teacher right in front of you.
In the real world your performance and other factors you cannot always control (like whether your boss likes you) determine outcomes. In the real world we rely on portfolios, presentations, projects, and evidence of learning.
Our current assessment system is broken. But our industrial era and industrial strength system keeps trying to put it back together and patching over the cracks. It is a matter of time when the cracks get more noticeable.
PERIpheral?
Posted on: August 30, 2010
I was happy to read an update about PERI’s assessment seminar in this MOE press release. I was disappointed that I could not learn more from the link, the PERI Holistic Assessment Website, provided in that release because it was password protected.
Why call it “peripheral”? There are just 16 schools trying out the PERI recommendations in any significant way, even though the programme was successful in the pre-trial school (the school that started the whole thing really!). A major venture might justify a more cautious approach and there is an obvious need for each school to design its own way. But we are talking about threatening the life of a sacred cow (Singapore assessment) here. Why not be more daring with the carving knives?
The PERI trial is not quite in the public eye in more ways than one and this is unusual given the impact on its stakeholders. The website I mentioned is not readily public for one. The other schools who are not involved do not seem terribly concerned for another.
When I ask primary school principals and teachers that I meet in non-PERI trial schools about their thoughts on PERI, I get 1) a shrug of the shoulders and 2) an arching of the eyebrows. The latter indicates that they are surprised that I know anything about it. The former indicate indifference (wait-and-see), ignorance (wait-for-someone-to-tell-me) or intolerance (nothing-will-change).
So much for change management. There is what you say and there is what you do. We are all held accountable by what we do. Let us see more of what is happening in that black box!
Rant: Exam addiction
Posted on: July 21, 2010
Like a peat fire, a colleague of mine reignited this deep-seated rant when we exchanged tweets yesterday:
I am worried that we are addicted to exams. Look at one neighbour comparing grades with another. Look at both of them looking for tuition for their kindergarten-attending kids. Look at schools sacrificing vacation time to prep students for exams and even to conduct exams. It’s a sad fact of Singapore life.
Just as disturbing was a letter a young man wrote to the Straits Times forum yesterday, Why Primary 1 exams are necessary. The writer, Sean, said that parents are to blame. But I think that he was equally culpable since he supported exams.
[Click to see larger version]
There are a few ways to respond at this point. One is, “What is wrong with exams?” and another is “Sean does not know any better, so give him a break”.
So what is wrong with exams as we currently know them? Nothing, if you were tested in life exactly like those exams. Nothing, if what you knew only in your head sufficed in life. Nothing, if the joy of learning weren’t drained out of you as if it were poison. Nothing, if a few hours frantically scribbling at a table did not result in a few letters that labelled you a success or failure regardless of your character, drive or other abilities. Nothing is wrong with exams if they did not create an addiction to high stakes testing.
I’m tempted to give Sean a break. As a pawn of the system, he is unlikely to know any better. If you are a frog born at the bottom of a dark well, you know not how other frogs live. But Sean, his peers, their parents and teachers have a choice if they bother to look and change. Unfortunately, they are all addicted to exams.
The demand for exams is created by policy makers, administrators and bean counters. They are so convincing that teachers not only design the exams, they also feed the exam machinery. Exams grip students and parents by fear and they live by that fear. These are our exam drug lords, unwitting peddlers and helpless addicts.
Another response to my rant might be, “You and I were a result of our exam system, but look how well we turned out!” To that I say: We probably did well not because of the system, but despite of it. Think about it.
I’ll state the obvious:
- Assessment is a key leverage point for educational change.
- Our teachers teach to the test because they are pressured to.
- If our modes and methods of assessment do not change, very little else will.
High stakes exams are designed to sort just like quality control in a factory. Exams are a legacy of the Industrial Age. Is there are time and place for tests and exams? Maybe. But not all the time and in every classroom. The folks at Harvard (thanks for the link, Steve!) are discovering this for various reasons.
Now if the winds of change blow new ideas our way, how will we respond? If we choose to remain addicted, not well.
But I detect gentle breezes that we casually label alternative assessments. These alternatives include meaningful portfolios, performances and projects.
I do not think that they should be alternatives like substitutes on a football bench are alternatives. These alternatives get little or no playing time in the real game. These alternatives are young and inexperienced. I will do what I can to see many of these alternatives become mainstream while turning exams into alternatives. After all, the exam player is getting old and has tested positive for drugs.
Trust the teachers
Posted on: January 11, 2010
The BBC has an interesting article on educational policy: Signs of a turning tide on tests. It was interesting to me because the Commons Schools Committee advocated that stakeholders “trust the teachers” instead of relying heavily on things like standardized tests.
From the article:
The report did not argue for an end to all external assessment. But it called for a shift toward more within-school, teacher-led assessment. This, it said, would not only save money but also a lot of the teaching time that is lost to exam preparation and administration.
And this is the key point: it is not about dropping school accountability altogether, but about making sure it does not obstruct teaching and learning.
I hope that the UK does this while the rest of the world watches and learns. I also hope that we in Singapore act on this same issue before it is too late.
I think a scheme like this will work only if a) teachers are treated/nurtured as professionals, b) we expect them to behave as such and c) we hold them accountable for what they do. The measure of accountability should not just be exam results otherwise the test tide will return.
Instead, evidence of student ability, attitudes and skills could be recorded in portfolios, community involvement, personal and group projects, etc. In other words, more authentic, meaningful and rigorous assessments.
Question the model
Posted on: November 27, 2009
It’s almost the end of a long teaching semester. For reasons too long and boring to mention, some of my colleagues and I had to start next semester’s teaching this semester.
The two things that usually happen at the semester’s end are I fall ill and I think about what to do next. So I type now before the flu completely takes over!
One thing that doesn’t usually happen at the end of semester is a huge grading load to process during the break and before the second half of the course resumes next year. This is why I found Siemen’s recent comments on grading and evaluation particularly relevant. Some snippets:
Grading is a waste of time. We only do it in schools and universities. It’s a sorting technique, not truly an evaluation technique. Iterative and formative feedback is what’s really required for learning.
Agreed! Our teacher education university is still in sorting mode but for reasons that are no longer relevant. Why? First, they are selected by interviews (coarse sorting). Second, a few bad apples that beat this filter or the trainees who cannot handle teaching will drop out on their own (self sorting). Third, the sorting is only based on academic results. If anyone wants to sort them out, do so along the lines of their values and attitudes because they must be role models and lifelong learners (yes, values and attitudes can be measured). Lastly, even after they are sorted, teacher trainees graduate and end up in schools irrespective of their grades. It is not as if A-grade teachers end up in some schools and C-grade teachers end up in another. So why sort at all?
Siemens concludes with:
The authors of the HASTAC post are not trying to do away with grading (as I would suggest we should). They are trying to use technology to make grading more “modern” or “in line” with society’s needs today. I think that’s exactly the wrong way to go about it. Question the model, don’t modernize it.
Thought-provoking and something I thoroughly agree with. If you consider the concepts of assessment of, for and as learning, I’d argue that most of what we do is, at best, only assessment of learning. Furthermore, assessment is just a number. Unlike evaluation, the value of that number is not made clear.
So what’s on my agenda? This year my guiding principle as I facilitated the ICT course was to get my teacher trainees to use what their students were already using in terms of technology. Next year my approach will likely be “question the model, don’t modernize it”.
Alternative assessment?
Posted on: September 28, 2009
I started thinking about how we assess teacher trainees here in NIE. While my thoughts are still forming, I think that we should be evaluating them instead.
It never ceases to amaze me how well some of my teacher trainees write when they reflect in their blogs. It also never surprises me how poorly constructed some of their written assignments are. I am reminded of the latter now that I have collected essays from all four of my classes.
I have blogged before about how I enjoy reading my trainees’ blogs because it allows me to get inside their heads. Blogging seems to give them the time to think deeply and write thoughtfully.
So should an essay assignment. But an assignment is not necessarily as meaningful as blog entries. The assignment is high-stakes, defined by a small group of “experts” and heavily influenced by university policies. I think that an assignment makes the most sense to the teacher and not the student, but students do them because because they have to and are conditioned that way.
Assessments only put numbers or grades to performances. They measure cognitive ability and often only just that. Evaluations, on the other hand, determine the VALUE of a test, a portfolio, a performance, etc. If we want to start thinking about “alternative assessments”, we should really be asking ourselves how we are evaluating teachers-to-be.
I would love to see a portfolio system become the norm in teacher education. After all, teachers are practitioners and they must show evidence of what they can do, not just write about it or create flashy presentations of what they might do.
I think that reflective blogging has a place in the portfolio system. Both the teacher trainee and I can monitor how their knowledge, skills, attitudes and values change over time. Much of what is written is self reported and skeptics might argue that teachers-to-be might say only the “right” things. I’d argue that these educators do not get the idea of portfolios nor have they taken the opportunity to build trust and to emphasize core educator values.
