The health of your software development life cycle (SDLC) is an important indicator of your organizations’ quality assurance, cost effectiveness, customer satisfaction, and compliance. While the executive order (EO) on improving the nation’s cybersecurity issued in May 2021 only required software Bill of Materials (SBOM)s for federal agencies, it has resulted in widespread promotion and adoption of SBOMs as an industry best practice. The EO has also driven many organizations to re-evaluate their software development processes and implement measures to improve the security and resilience of their software supply chains. It also underscores that organizations need to pay closer attention to their development practices and health, as SBOM creation can expose “dirty laundry” to the world. This is why it’s time to stop using pen tests to simply find problems in your software, and start using them to find problems in your processes.
Pen testing as SDLC best practice
While software developers have long used third-party web app and API pen tests to find application security defects, pen tests are also a great way to gauge the health of an SDLC. Third-party automated pen tests, which are often mandated by regulatory agencies, require very little experience with tooling or training on the part of the customer, and are fairly thorough in identifying security flaws that need to be fixed. This is why they’re popular. From both parties’ points of view, it’s a pretty fulfilling activity; the pen tester gets to hand over a number of alarming-sounding vulnerabilities, and the developers receive a number of defects to add to the backlog.
However, what third-party automated pen tests cannot replace is the human behind pen testing execution. While humans are slow and more expensive than automated defect discovery tooling, because they can mimic human hackers, they are better at evaluating an application’s response to a pen test, and can potentially catch responses that automated software may miss.
One way to think about how you are using pen tests is to look at the cost of finding defects. For example, finding a cross-site scripting defect using a SAST tool is much more efficient than finding it using a pen test. Using pen tests for vulnerability detection means that each one costs a percentage of your limited testing time, a fraction of your checklist, and a portion of your report writeup effort. And when companies run pen tests on an annual basis, the cost per defect increases as vulnerabilities get patched, and the critical and high vulnerabilities dwindle to nothing. So if pen testing shows diminishing returns as a defect discovery tool, how can you use the expensive but more accurate human element of pen tests to take full advantage of their value?
Where pen testing really begins to pay off is when organizations leave routine defect discovery to automated tools and shift the human effort toward AppSec program assurance. Just as there are multiple points in a piece of software that could validate, detect, or encode a cross-site scripting attack, there are multiple points in an SDLC that can anticipate, prevent, detect, or mitigate various classes of vulnerabilities. By shifting pen testing efforts to finding those inflection points in your SDLC where vulnerabilities can be prevented, you’re using the human and financial costs of pen testing more efficiently than if you’re just focused on defect discovery. Using pen testing this way can help you detect the processes in your SDLC that allow vulnerabilities to creep in, so if you begin fixing those processes, you’ll also minimize future vulnerabilities.