Penetration Testing Methodologies compares the existing Methodologies and Frameworks within the world of vulnerability detection. It explains the different IT infrastructure landscapes (General ICT, Cloud, IoT, OT/ICS and cyber-physical and the NIS2 categories), describes the landscape of vulnerability detection and focusses in depth on Penetration Testing. It describes many methodologies and frameworks in detail and compares them via defined criteria.
Written to be a valuable resource for Member State NCCs, it is also intended to individual Pen Testers, aspiring Pen Testers, Pen Testing Organizations and End Users – Cybersecurity teams. It would provide them with in-depth knowledge of Penetration Testing practices, enabling them to enhance national cybersecurity strategies, support research and innovation, harmonize standards, influence policy, and improve the overall cyber resilience of their country. By understanding and utilizing the methodologies you evaluate, NCCs can more effectively fulfill their mandate to support cybercapacitybuilding in their respective National Communities of Competence[1].
The report is also written for organisations and individuals that perform Penetration Testing as a service and their respective teams of ethical hackers. It offers a wealth of knowledge that can use to enhance their service quality, drive innovation, support compliance, and improve market positioning. It also aims to equip them with the tools and insights needed to refine their methodologies, better serve their clients, and stay competitive in a rapidly evolving cybersecurity landscape.
We identified two types of practices: methodologies and frameworks. Methodologies are focused on the organisation and process of Penetration Testing, how to ensure a good quality from the process point of view. Frameworks are technical and are focused on the vulnerabilities and techniques to identify them.
Within the section of methodologies, the coverage of the process also differs. Some only focus on a single Penetration test while others also include the process of organising multiple Penetration tests within a company. In this latest category, the methodology “A Guide for running an Effective Penetration Testing Programme” from CREST is clearly outstanding. In the former category, the “Web Security Testing Guide” authored by OWASP is the assessed as the best framework. This guide is actively supported by a large community.
Within the section of frameworks there is a large difference in breadth and depth of the framework. Few are comprehensive. Of the 13 frameworks we evaluated, only one framework scored more than 7 out of 10. The most recommended framework is the OWASP Mobile Application Security Testing Guide, but this is specific to mobile devices. As a general conclusion, we recommend working with a combination of the MITRE Attack framework in combination with the OWASP Web Security Testing Guide. This combination will however demand a large investment of time from the Ethical Hacker to learn both frameworks and be able to combine them.
We can summarize that there are many good sources of information, whereby most are open and all documentation is freely available. There is however no single methodology, nor framework that covers all that is needed. We therefore recommend that NCC’s and Penetration testing organisations build their proper approach to Penetration Testing and base this on the existing methodologies and frameworks, such that they can assure that the vulnerabilities are identified in a complete and optimal way. This to the end of providing the most optimal cyber security support to protect our European SME’s.
We identified several gaps in the frameworks, especially towards critical infrastructure (IOT, OT, Cyber physical). Especially for this last one, we found no frameworks specifically designed for this.
[1] mandate arising from ECCC Regulation (Regulation (EU) 2021/887)