I like requiring exams be taken on school dumb terminals. If students are just answering questions, they shouldn't need anything fancier. If they need their laptop to look things up or run calculations, fine, but copy pasting is not an answer.
This is an extremely robust and simple solution. I'm amazed it's not already done.
The hardware is cheap keyboards and displays. What, $15 BOM? And a 30 cent microprocessor.
You just need to make sure the 'dumb terminals' are *really* dumb to the point where they can't be hacked. A bit tricky these days - buying something that primitive costs extra. And you have to be mathematically certain there's no backdoor from the VT100 to the MCU. [which is the very first thing the marketing dept. will ask for)
A really stupid simple display interface means the display and keyboard cables could be moved to USB connectors on the student's laptop. But the MCU should be able to detect this and flunk the test.
Somehow we're talking past each other. I'm imagning physical terminals (dumb) in the classroom. Not connected to any student's computer. The students use them to take the test. The terminals are made of cheap PC displays and keyboards. They are not computers, not Turing complete even. Dumb, and therefore hack resistant. Internal construction should not use consumer interfaces like USB (to make *hardware* hacking nontrivial).
OK, fine, that's exactly what I imagine. Now I come in, Joe Lazy Student, disconnect the communication cables to the dumb tube, and keyboard if separate, plug them into my laptop, and use a program to simulate that dumb tube on my screen. I'm not hacking the dumb tube or the school computer it's connected to. I'm hacking the cables.
It's been a long long time since I've dealt with serial RS-232 cables. Myt memory says there is no way for the computer to tell if a serial cable is connected, or whether the device it's connected to is powered on and functioning. The only way to prevent Joe Lazy Student from moving the cables is some protocol, like USB, that can tell if the connection is lost.
Absolutely need SOME solution. Since cursive is no longer taught or practiced, and block printing is unacceptably slow, pen and paper (blue books) are simply no longer fit for purpose in scholastic examinations.
> The software can, Examsoft does, take over the computer that it is running on in order to prevent the user from accessing material on his hard drive or online. To evade that, create a virtual computer on your physical computer, load Examsoft on that, and switch between the padlocked virtual computer and the unlocked physical computer at will.
Or even easier--have a second physical computer nearby.
> The context was not taking exams at home but in school
In that context, I would think a locked down computer owned by the school, not the student, would work better.
That said, switching between the padlocked virtual computer and the unlocked physical computer would also be observable in a school setting, particularly if video cameras running AI algorithms were used.
> 4. Write exams designed to be taken open book and open computer.
This was the solution I was most often subjected to in college: the exams were simply written so that even having the book open in front of you wouldn't help you to answer the question if you didn't really understand the material. But that was before the Internet and LLMs.
Microsoft Windows for years has supported an 'exam' functionality for locking down a device to restrict application and internet access - as set by the school policy. Given a modern 'trusted boot' system that is appropriately configured, a school should be able to establish a reasonably secure test environment.
My daughter has taken her professional engineering licensing exams at commercial testing sites. They certainly had the environments well locked down.
Of course, neither addresses helping the grader.
When I was a teaching assistant reviewing senior mechanical engineering reports I would mark up the reports with a permanent red ink pen and slot them into folders - A+, A, A-, ... . After I had done one pass across all the reports I would do a second scan, comparing the reports with the other reports in the same and adjacent grade reports. As you noted, exposure to the various reports during a grading run would cause a drift in my grading thresholds.
Only then would I mark the reports with their grades and report them.
Odd that LLMs were not mentioned. That is a new issue with making the exam open book.
Being able to answer correctly with the assistance of an LLM is not trivial. They are now famous for producing slop, though perhaps it is awkward grading off just for slop, when no clear mistake is involved.
What is the point of tests? Of classes? Do we want to know whether facts have been memorized, or whether the student can solve novel problems? Or what?
In the abstract, we start off with a goal, and the solution mainly consists of making a plan that can accomplish the goal using available tools. Technology occasionally makes particukar tools obsolete, but the underlying framework remains.
Stopping cheating, even with Respondus Lockdown browser and Respondus Monitor is not fool proof, but its better than nothing. In the end, independent testing with continuous proctoring is probably as good as it gets. Something like SAT testing or bar exam testing or FINRA testing.
Yes. Cheating isn't a software problem; it's a human problem. The solution isn't technology. The solution is independent testing. We all know that professors grading their own students is a conflict of interest. Now, Ai is forcing the issue. Finally. Thank goodness.
I like requiring exams be taken on school dumb terminals. If students are just answering questions, they shouldn't need anything fancier. If they need their laptop to look things up or run calculations, fine, but copy pasting is not an answer.
This is an extremely robust and simple solution. I'm amazed it's not already done.
The hardware is cheap keyboards and displays. What, $15 BOM? And a 30 cent microprocessor.
You just need to make sure the 'dumb terminals' are *really* dumb to the point where they can't be hacked. A bit tricky these days - buying something that primitive costs extra. And you have to be mathematically certain there's no backdoor from the VT100 to the MCU. [which is the very first thing the marketing dept. will ask for)
A really stupid simple display interface means the display and keyboard cables could be moved to USB connectors on the student's laptop. But the MCU should be able to detect this and flunk the test.
Don't use USB interfaces.
What would you use? It either has to be permanently attached, or something the computer can detect when it’s disconnected.
Somehow we're talking past each other. I'm imagning physical terminals (dumb) in the classroom. Not connected to any student's computer. The students use them to take the test. The terminals are made of cheap PC displays and keyboards. They are not computers, not Turing complete even. Dumb, and therefore hack resistant. Internal construction should not use consumer interfaces like USB (to make *hardware* hacking nontrivial).
OK, fine, that's exactly what I imagine. Now I come in, Joe Lazy Student, disconnect the communication cables to the dumb tube, and keyboard if separate, plug them into my laptop, and use a program to simulate that dumb tube on my screen. I'm not hacking the dumb tube or the school computer it's connected to. I'm hacking the cables.
It's been a long long time since I've dealt with serial RS-232 cables. Myt memory says there is no way for the computer to tell if a serial cable is connected, or whether the device it's connected to is powered on and functioning. The only way to prevent Joe Lazy Student from moving the cables is some protocol, like USB, that can tell if the connection is lost.
Absolutely need SOME solution. Since cursive is no longer taught or practiced, and block printing is unacceptably slow, pen and paper (blue books) are simply no longer fit for purpose in scholastic examinations.
The funnier example is cybersecurity exams. If you can’t cheat, are you really any good?
> The software can, Examsoft does, take over the computer that it is running on in order to prevent the user from accessing material on his hard drive or online. To evade that, create a virtual computer on your physical computer, load Examsoft on that, and switch between the padlocked virtual computer and the unlocked physical computer at will.
Or even easier--have a second physical computer nearby.
That is observable. The context was not taking exams at home but in school, everyone in one room and proctors observing.
> The context was not taking exams at home but in school
In that context, I would think a locked down computer owned by the school, not the student, would work better.
That said, switching between the padlocked virtual computer and the unlocked physical computer would also be observable in a school setting, particularly if video cameras running AI algorithms were used.
> 4. Write exams designed to be taken open book and open computer.
This was the solution I was most often subjected to in college: the exams were simply written so that even having the book open in front of you wouldn't help you to answer the question if you didn't really understand the material. But that was before the Internet and LLMs.
Microsoft Windows for years has supported an 'exam' functionality for locking down a device to restrict application and internet access - as set by the school policy. Given a modern 'trusted boot' system that is appropriately configured, a school should be able to establish a reasonably secure test environment.
My daughter has taken her professional engineering licensing exams at commercial testing sites. They certainly had the environments well locked down.
Of course, neither addresses helping the grader.
When I was a teaching assistant reviewing senior mechanical engineering reports I would mark up the reports with a permanent red ink pen and slot them into folders - A+, A, A-, ... . After I had done one pass across all the reports I would do a second scan, comparing the reports with the other reports in the same and adjacent grade reports. As you noted, exposure to the various reports during a grading run would cause a drift in my grading thresholds.
Only then would I mark the reports with their grades and report them.
Odd that LLMs were not mentioned. That is a new issue with making the exam open book.
Being able to answer correctly with the assistance of an LLM is not trivial. They are now famous for producing slop, though perhaps it is awkward grading off just for slop, when no clear mistake is involved.
What is the point of tests? Of classes? Do we want to know whether facts have been memorized, or whether the student can solve novel problems? Or what?
In the abstract, we start off with a goal, and the solution mainly consists of making a plan that can accomplish the goal using available tools. Technology occasionally makes particukar tools obsolete, but the underlying framework remains.
This sort of functionality is available in a state of the art LMS like Canvas.
How does it solve the cheating problem?
Are you talking about the functionality in my proposed software?
Stopping cheating, even with Respondus Lockdown browser and Respondus Monitor is not fool proof, but its better than nothing. In the end, independent testing with continuous proctoring is probably as good as it gets. Something like SAT testing or bar exam testing or FINRA testing.
Yes. Cheating isn't a software problem; it's a human problem. The solution isn't technology. The solution is independent testing. We all know that professors grading their own students is a conflict of interest. Now, Ai is forcing the issue. Finally. Thank goodness.