articles · studio notes

Usability evidence: designing the file alongside a device that keeps changing.

For regulated devices, "users like it" isn't a finding. It's a liability. The human-factors work has to produce evidence a submission can stand on. And as more devices go on changing through software after launch, some adding AI features users have to interpret, that evidence can't be assembled once and shelved. It has to move with every release.

skeelx · updated oct 2026 · 4 min read

Two processes, or one

The common failure pattern in device programs: design runs on taste and iteration, and then, months later, a usability-engineering file is reverse-engineered for the submission, reconstructing rationale nobody wrote down. It's slow, it's expensive, and it shows: retrofitted files read like alibis. The alternative is structural: run the human-factors work so its natural exhaust is the evidence: use-related risks identified up front, design decisions traceable to them, evaluations documented as they happen with protocols, participants and findings.

two processes: the retrofit Design on taste & iteration rationale lives in conversation Device done The file, reverse-engineered reconstructing what nobody wrote down months later rationale reconstructed backwards; retrofitted files read like alibis one loop: the exhaust is the evidence Use-risk first Requirements trace to risks Formatives: small, frequent Summative confirms stressed, gloved, interrupted: what goes wrong? each decision answers a risk dated: tested · found · changed well-evidenced, not a gamble the usability file, assembled as it happened: risk analysis · trace · protocols & findings · the summative record the device and its file come out of the same loop: cheaper, faster, and considerably more convincing
the retrofit reads like an alibi; the loop's exhaust reads like evidence

What evidence-grade HF work looks like

It starts from use-error thinking rather than preference: what could a stressed, gloved, interrupted user do wrong with this device, and how badly would it matter? Where the device offers an AI-generated suggestion or score, the list grows: a user who over-trusts a confident output, or dismisses a correct one, has made a use error like any misread display. Those risks drive the design requirements. Formative evaluations along the way are small, frequent and documented, each one a dated record of what was tested, what was found, what changed in response. By the time a summative evaluation arrives, it's confirming a well-evidenced design, not gambling the program on a first real test. The same rigs and trials that make the product fit real hands produce the paper trail, because they were designed to.

What this asks of your design partner

Documentation discipline that survives audit, protocols written before sessions rather than after, and comfort working alongside your regulatory and quality people. Their process is the destination, and the design work should drop into it cleanly. It's the standing philosophy of our medical device work: the device and its file come out of the same loop, which is cheaper, faster and considerably more convincing than either done alone. If your program already has the device but not the file, closing evidence gaps before a submission window is common, unglamorous work, and the earlier it starts, the less archaeology it involves.

The agent-native lens: warnings that travel, participants who are real

When the buyer sends an assistant. Clinical and procurement teams are starting to ask assistants to shortlist equipment and to answer how-do-I questions about it. Those assistants read the instructions for use, the labelling and the training material, then restate them, and a restatement can drop a warning that sits pages away from the step it governs. So write the instructions the way the risk analysis reads: each warning attached to the step it protects, intended users and use environments stated plainly, the training a new user needs said once. Published as current, structured documents rather than a scanned PDF, they leave less room to paraphrase into a use error. And if an agent can read data from the device or trigger a function, that call path is a user interface in all but name. It belongs in the use-related risk analysis beside the screen and the buttons.

When the business runs on agents. Agents can keep the trace honest: every use-related risk linked to a requirement, every requirement to an evaluation, and a flag whenever a design change breaks a link. They can draft a session protocol from the risk list and turn notes into a first-pass findings table. Two limits don't move. A participant simulated by a model is not a participant; it can help rehearse a protocol, never stand in for the people the device is for. And nothing enters the file as a finding until the human-factors lead has reviewed it.

A file that keeps up

Software updates now reshape screens, workflows and sometimes what a device advises, long after the first submission. Each release that touches how the device is used carries its own small piece of human-factors work: what changed, which use-related risks it affects, what was evaluated and what was found. Run the loop continuously and that is routine. Retrofit the file, and every release becomes another round of archaeology. The wider method behind this is our evidence-loop process. Regulated usability is simply the version where the evidence has a second customer.

next

One loop. Device and file together.

Start a project