Human Centered Strategies

The FDA's Human Factors group has moved within the FDA's CDRH organization to the Office of Device Evaluation (ODE) futher underscoring the importance of human factors in the regulatory process of reviewing PMAs, 510(k)s and other manufacturer submissions. In particular, ODE reviewers are looking for evidence of device interaction risk management in the form of a final simulated use user interface validation study and a solid use error risk assessment regarding any failures observed in the validation study.

The device approval process, centered in the Office of Device Evaluation of FDA/CDRH, focuses on the manufacturer's proof of the "safety and effectiveness of the device." Traditionally, this has required due diligence and submission of testing results for component reliability, software and hardware fail-safe capability, manufacturing process quality, and evidence of proof of clinical claims of the device. Risk management concerning user interactions with the device has been a concern, but not systematically evaluated on a daily basis by ODE reviewers. That all changed when CDRH moved several human factors scientists to the Office of Device Evaluation in 2008.

What's Different Now About Human Factors at FDA?

The FDA had always stressed the importance of employing human factors techniques in understanding and controlling use error risks in medical device design and development. So what's different now?

Human factors reviewers have become part of the overall review team and process at ODE. This means they now are looking at many more submissions for evidence of validation that use errors have been systematically removed from the user interfaces of medical devices. Members of the human factors group are now given "consults", that is, a voice at the table determining whether a submission is adequate for clearance, especially where the user interface and device interaction are critical to safety.

What are they looking for?

The major piece of evidence that human factors reviewers need for assurance that use related hazards have been removed is a carefully designed and executed validation study with participants from the appropriate user populations.

What makes a good, acceptable validation study?

The validation study should be the final, or near-final result of a continuous process of identifying and mitigating use-related hazards in device interaction. By validation stage time, you should have been engaged in a variety of studies, analyses, and user research methods that result in an "air-tight" design. The validation study is analogous to "putting your Grand Prix race car on the track, after you've put it on the test track for months beforehand" as stated by one FDA human factors reviewer. In validation, you're looking for "residual risk", patterns or even isolated instances of remaining use errors that could be problematic, should you find them.

How many participants, what do they do?

The question of sample size in validation studies is a common one and depends on the "end customer" for the overall validation effort. Typically, clinical trials require large sample sizes to ensure appropriate statistical power to substantiate the device's efficacy. Marketing may want to validate certain "claims" which will require adequate size to provide evidence of the device's superiority over competitive devices.

In a human factors validation, fifteen (15) participants performing critical tasks involving high risks is currently the accepted, minimum sample for validation studies per FDA guidance. Many comment that this seems too low a number, however, the human factors case departs from the cases discussed above in that the major emphasis is on exposing problems in the user interface design at a relatively late, near production stage of development. We want to discover any "show stoppers" resulting from faulty user interaction at validation. When discovery of problems is the aim, a long history of human interface usability study research has shown a clear "law of diminishing returns" for employing more than 12-15 people in these "discovery" type of study efforts. We are asserting the hypothesis that 15 people, from the target population, will reveal common, across-the -population types of trouble with the device, if it exists.

Human Centered Strategies looks forward to discussing how use error may be identified and controlled in the design of your medical device and how to structure a human factors section of a pre-market submission. Often, this may include facilitation of dicsussions with the FDA's Human Factors Team to identify the best design for the simulated use validation study and supporting analysis.