One of my relatives heard some strange stories when working on a healthcare helpline during the Covid pandemic. Her job was to help callers complete the rapid lateral flow tests used millions of times during lockdown. But some callers were clearly confused by the procedure. “So, I’ve drunk the fluid in the tube. What do I do now?” asked one.
That user confusion may be an extreme example of a common technological problem: how ordinary people use a product or service in the real world may diverge wildly from the designers’ intentions in the lab.
Sometimes that misuse can be deliberate, for better or worse. For example, the campaigning organisation Reporters Without Borders has tried to protect free speech in several authoritarian countries by hiding banned content on the Minecraft video game server. Criminals, meanwhile, have been using home 3D printers to manufacture untraceable guns. More often, though, misuse is unintentional, as with the Covid tests. Call it the inadvertent misuse problem, or “imp” for short. The new gremlins in the machines might well be the imps in the chatbots.