Penetration testing, commonly referred to as pen testing, is a critical process in the cybersecurity landscape. When it comes to medical devices, this process takes on added significance due to the potential impact on patient safety and privacy. This blog will delve into the basics of pen testing, explore its distinctiveness in the medical device industry, and examine its integration within the product lifecycle.
What is Pen Testing?
Penetration testing is a simulated cyber attack to evaluate the security of an application, system, or product performed by trained security experts known as ethical hackers to simulate an attack from malicious parties such as black-hat hackers and disgruntled employees. The primary objective is to identify and exploit vulnerabilities that could compromise the confidentiality, integrity, or availability of the system before a malicious party can do so, enabling the developer or manufacturer to identify and develop appropriate security controls to reduce risk. “Pen testing” also serves to perform security control validation, ensuring security controls are designed to effectively perform as specified.
In the context of medical devices, pen testing aims to ensure that these devices can withstand cyber threats that might compromise the safety or efficacy of the device or patient data.
Pen Testing in the Medical Device Industry
Medical devices operate in an environment where security vulnerabilities can severely impact patient safety and continued care delivery. Additionally, these devices are highly interconnected, increasing the risk and complexity of managing their security. Furthermore, medical devices that are networked can be the weak link into the broader healthcare environment in which they operate. This underscores the necessity for rigorous and thorough pen testing.
Key Differences from Testing in Other Industries:
1. Scientific and Engineering Rigor:
- Medical device manufacturing is dominated by scientists and engineers who prioritize unbiased and carefully executed testing methodology. The expectations for rigor and demonstrable facts are high, and penetration testing must meet these standards to be credible.
2. Variability Among Testers:
- The results of pen testing can differ widely depending on the tester’s experience, skill set, focus, and tools. The clinical operation, intended use, and environment of a device play a crucial role in shaping the design of pen tests. Testers with different levels of expertise may identify varying types of issues based on their unique approach. Additionally, testers who are less familiar with the stringent requirements of highly regulated medical devices may overlook critical vulnerabilities.
3. Cybersecurity Risk Assessment:
- In medical device security, the primary role of pen testers is to identify vulnerabilities, determine what data can be accessed or manipulated, and assess whether these actions can be detected. Pen testers should not conduct risk assessments beyond this scope, especially when it comes to evaluating patient safety or device effectiveness. Risk assessments tied to 14971 processes, which focus on safety and clinical considerations, are the responsibility of qualified safety risk management professionals and clinical teams.
Pen Testing and the Product Lifecycle
Pen testing is not a one-off task but an integral part of the medical device product lifecycle, encompassing design, and development on the premarket side, and maintenance and surveillance on the postmarket side.
1. Design and Development (Premarket):
- Incorporating security measures early in the design and development stages is essential to building a secure medical device. Threat modeling exercises help identify potential vulnerabilities and guide the development of effective security features. The outputs from these exercises can be used to scope initial penetration testing activities, focusing on key assets and interfaces to identify any gaps in security controls.
- Pen testing should evolve alongside the product’s development, with the final premarket test validating that security controls align with both the threat model and the product’s security risk assessment.
- This approach ensures traceability from high-level user needs through the product’s risk management and security architecture, offering confidence in the product’s readiness for real-world deployment.
- Regular iterative testing may also be appropriate, depending on the complexity of the device and its connectivity.
- It’s critical that the unit under test (UUT) accurately represents the final product intended for market deployment to ensure testing results reflect the actual device environment.
2. Post-Market Surveillance and Maintenance:
- After the release to market, continuous monitoring with periodic pen testing isare essential to address new vulnerabilities that emerge. This ongoing process combined with updates and other post market surveillance activities ensures the device remains secure throughout its operational life.
Methodological Rigor and Reporting
Penetration testing must be rigorous, repeatable, and evidence-based, regardless of the industry. In the context of medical devices, this rigor is particularly crucial given the evolving threat landscape. To stay ahead of potential attackers, pen testing methodologies need to be flexible and adaptive, evolving alongside the threat environment. For medical devices, this means developing methodologies that are not only robust but also capable of being clearly documented and replicated. Although it can be challenging to establish standardized testing protocols due to the ever-changing nature of cybersecurity threats, high-level approaches should remain consistent to ensure thorough and effective testing.
Key Elements of Rigorous Reporting
The elements below are based on FDA testing requirements outlined in their Premarket Guidance:
1. Description:
- Provide an overview of the findings and context, helping technical audiences understand the issue and its implications.
2. Scope:
- It’s crucial that the report clearly outline what was in scope for the testing as well as what was not in scope. Regulatory agencies require full end to end testing of the system.
3. Timeframe:
- It’s important to ensure the test firm states the length of the engagement in the final report. This should line up with the complexity of the system under test. Three days of testing on a system that includes numerous, complex technologies is not an acceptable length of time to perform an appropriate test.
4. Methodology:
- Detail how the finding was identified, including test steps, code snippets, commands, tools, and procedures, enabling replication by experienced professionals.
5. Technical Risk Factors:
- Evaluate findings against objective technical criteria to support risk management decisions.
6. Technical Detail:
- Include evidence such as screenshots, videos, photographs, raw output, or code snippets to provide deeper insights.
7. Whitebox Testing:
- In order to provide traceability, it is also imperative to provide the test firm with the threat model, risk assessment, product requirements and other detailed design documentation. Work with the tester to determine which controls in the risk assessment they can test and validate.
8. Point in Time:
- Pen testing provides a snapshot of the device’s security at a specific moment. It’s important to indicate the date the test was performed. Given the rapid evolution of threats, findings can become outdated quickly, necessitating continuous post-market surveillance and updates. FDA has raised concerns about pen testing that was performed more than a year prior to submission and in some cases, required a new pen test.
9. Credentials of Testers:
- It is crucial that any report from a penetration test include details regarding the skills and credentials of the testers, the equipment and methods they use, etc., to ensure that the manufacturer and regulator understand the appropriateness of testing applied and of the skills of the tester. Inappropriate application of testing can be hard for an MDM to identify when procuring services and can lead to regulatory risks.
Risk Rating and Risk Scoring
While CVSS (Common Vulnerability Scoring System) is a valuable tool for scoring identified vulnerabilities, it is not designed to assess the potential harm to patients or operators in the context of medical devices. CVSS should be used to evaluate the technical severity of a vulnerability, but it must not be applied to assess safety impacts, as that falls under the domain of ISO 14971 risk management processes.
Alternative Approach:
1. Technical Effect:
Describe the state or condition created during testing, highlighting implications for safety, effectiveness, and other critical factors.
2. Conditions of Testing:
- Outline the circumstances under which the finding was generated, such as proximity to the device, interfaces tested, and tools used.
3. Regular Status Updates and Preliminary Documentation:
- Maintain ongoing communication with stakeholders through regular meetings and preliminary documentation to enhance the testing process and ensure clarity.
Conclusion
By adopting these principles and building these requirements into the quality management system, medical device manufacturers can ensure their pen testing efforts are thorough, scientifically rigorous, and aligned with the high standards expected in the industry.
Pen testing for medical devices is a complex but essential component of cybersecurity. By understanding its unique challenges and integrating it throughout the product lifecycle, manufacturers can safeguard their devices against evolving threats and ensure patient safety and data integrity.
AEC provides proactive cybersecurity solutions for medical device manufacturers. Take AEC’s FDA Readiness Assessment or email us at info@aeciso.com to learn how we can help your organization develop a security strategy that meets FDA requirements and protects patient safety.
