The Ottawa Citizen by Claire Davies 26 October 2016
Last week we set up Grade 10 students across the province for failure. We used them as guinea pigs to test a deeply flawed online system that already has a history of letting us down.
The Ontario Secondary Schools Literacy Test has always been a pen-and-paper affair. This year, for the first time, 100,000 young people would trial the test online.
In addition to practice tests, teachers at my 15-year-old son’s school instructed me on how to help him prepare. At home the night before, the anxiety was building. After all, students only have a maximum of two attempts to pass, and they all must do so to graduate.
The test is deemed so important that last Thursday, only Grade 10 students were allowed on school property to ensure no distractions.
As each student logged on to a computer terminal, the system began to crumble. Each was met by a blank, white screen. If lucky, the white screen would change after 10 seconds to enable login. If not, the screen would stay blank. When a question did appear, the system would freeze when the “next question” button was pressed. Another “white screen of death” would present and frequently, a teacher would have to reboot the system.
It was a disaster. But for those of us who research online access, it wasn’t entirely a surprise.
In February, for example, the long-awaited IBM Phoenix payment system went live for many Canadian government departments. There was a promise that it would save millions of dollars. As of last week, there were over 30,000 unresolved cases of employees being underpaid, overpaid or not at all. Fixing the system is expected to cost up to $50 million. IBM knows a lot about computers, and even they couldn’t get it right the first time.
Here’s another: On May 2 of this year, the website for the census opened up to a flurry of activity and within minutes, it had crashed. The congestion surprised the site’s technicians with Statistics Canada suggesting that Canadians’ “enthusiasm” for filing their census data simply overwhelmed the system.
On the website for Educational Quality and Accountability – the body that oversees the Ontario Secondary Schools Literacy Test – there is considerable research into student achievement, school effectiveness and best assessment practices. Not one article, however, provides any evidence that online testing is beneficial to students. In university settings, studies are inconclusive as to the benefit of online testing and caution is recommended.
If there is no evidence within Ontario that online testing is better for the student, why is it being rolled out province wide?
My son was one of the “lucky” ones as he was able to complete his test (after rebooting his computer eight times). Even he is extremely worried that his test won’t count. He wants to know what will be recorded on his transcript. He wants to know whether he’ll have to re-do the test in March, and whether that will affect his university applications. And what, he wants to know, if it happens again? As a parent – and one who works in academia at that – it’s hard to reassure him.
It’s clear Ontario is not ready for online testing. Let’s avoid increasing student anxiety by giving them the paper version of the test in March (or even this week – we know they’re prepared!) But let’s skip another online testing fiasco until we know the system works.
We teach our students that they should be prepared to experience failure, but providing them with a system that won’t allow them to succeed is not the best method. At this rate, they may have a greater fear of system failure than of failing the test on which they are being assessed in the first place.
Claire Davies is an assistant professor at Queen’s University and researches optimal design of web-based surveys. Her 15-year-old attends high school in Kingston.