PMDA’s overall HFE mindset is aligned with international regulatory frameworks, but expectations in certain areas can differ. In particular, PMDA takes a distinct approach to topics such as sample size determination and test environment considerations. This guide outlines key points based on the implementation of JIS T 62366-1:2022 and current PMDA expectations.
Sample Size
PMDA does not specify a minimum sample size requirement. Unlike regulatory frameworks that mandate specific participant counts (e.g., “15 per user group”), Japan’s approach centers on rational justification rather than numerical compliance.
Evaluation Criteria
PMDA evaluates sample size adequacy across three dimensions:
1. User Representativeness
- Appropriate representation of intended user types (physicians/nurses/patients/general users)
- Adequate distribution across experience levels (novice vs. experienced users)
- Alignment with actual use environments (hospital vs. home use settings)
2. Coverage of Hazard-Related Use Scenarios
- Comprehensive testing of critical use errors identified through JIS T 14971 risk analysis
- Evaluation of operational and decision-making scenarios where errors could manifest
3. Rational Justification
- Logical explanation for why the selected sample size is adequate
- Consistency with existing knowledge, prior evaluations, or post-market data
Conditions Supporting Smaller Sample Sizes
Reduced sample sizes may be justified when the following conditions are met:
- Limited number of hazard-related use scenarios
- Simple operational sequences with minimal decision branches
- Comprehensive formative evaluation resulting in substantial design improvements
- Availability of supporting evidence from existing products, overseas validation data, or post-market surveillance
Conditions Requiring Larger Samples
Certain device characteristics need either larger sample size or enhanced rational justification.
- Software as a Medical Device (SaMD): Particularly when dependent on user judgment or data input interpretation
- Consumer/patient-use devices: Non-professional user populations with variable skill levels
- Diagnostic or treatment-critical devices: Use errors that directly impact clinical decision-making
- Class III/IV devices: Strict scrutiny on scenario selection rationale and residual risk acceptability
Documentation
PMDA frequently identifies the following documentation deficiencies:
- Arbitrary participant numbers without justification
- Failure to demonstrate user representativeness
- Inadequate scenario coverage
Example of Sample Size Rationale:
This summative evaluation was conducted under conditions where anticipated use errors would manifest, targeting hazard-related use scenarios. The operations and decisions being evaluated are limited; therefore, it was determined that verification of use errors is achievable with a small number of participants.
Test Environment Requirements
Access to medical facilities for external usability testing in Japan is limited, and dedicated usability laboratories are relatively rare. As a result, conference room settings with appropriate simulation elements are commonly used and generally accepted by PMDA. The agency does not require high-fidelity clinical simulation environments.
What matters most is whether the evaluation environment allows researchers to realistically determine whether use errors may occur within hazard-related use scenarios.
Evaluation Criteria
The primary criterion is whether the environment allows potential use errors to manifest under realistic conditions. PMDA evaluates whether:
- The evaluation focuses primarily on interface interaction and decision-making, where environmental dependency is relatively low.
(e.g., button operation, screen transitions, understanding displayed information, recognizing alerts or warnings) - Critical elements necessary for hazard-related scenarios are adequately reproduced.
(e.g., operation flow, timing of information presentation, display conditions that could lead to misinterpretation or incorrect decisions) - The study setup avoids over-idealized conditions that could mask potential use errors.
(e.g., an excessively quiet environment, displays or controls positioned or emphasized in ways that make them easier to perceive or operate than in real use)
Use Environment Attribute Considerations
According to JIS T 62366-1 Section 3.2, the simulated use environment should reproduce attributes that could influence user performance. These may include:
- Physical Attributes: Hygiene requirements, lighting, ambient noise, temperature.
- Social Attributes: Team vs. individual use, organizational chaos vs. order, stress levels, work duration.
The goal is to simulate attributes that directly affect the potential for use errors, rather than achieving complete environmental fidelity.
Documentation
PMDA frequently identifies the following documentation deficiencies:
- Absence of environment justification
- Over-idealized conditions without acknowledgment
- Missing critical environmental factors
Example of Test Environment Rationale:
This summative evaluation was conducted to reproduce conditions where use errors in hazard-related use scenarios would manifest. Although a conference room was used, operational procedures, information presentation, and decision conditions were configured to be equivalent to the actual use environment.
Conclusion
PMDA’s approach to sample size and test environment requirements places strong emphasis on well-reasoned justification. Successful submissions clearly explain how study design decisions align with the device’s risk profile and integrate with JIS T 14971 risk management documentation, demonstrating that the chosen validation approach is appropriate for the device and its intended use context and provides sufficient evidence that potential use errors have been adequately evaluated.
Uism offers end-to-end support for planning, conducting, and interpreting usability studies in Japan. Whether you are looking to validate your device with local users, strengthen your global HFE/UE documentation, or gain a deeper understanding of how your product is used in the Japanese context, we help bridge the gap between global standards and local regulatory expectations. Let’s connect!
*This guide is based on JIS T 62366-1:2022 requirements and current PMDA regulatory expectations as of 2024-2025. Manufacturers should engage qualified regulatory consultants for device-specific guidance.
