Outsourced penetration testing: improving product security from the outside in

Penetration testing conducted by external vendors can often be seen as just another compliance checkbox. However, when tailored to the unique security needs of a product, outsourced penetration testing (OPT) can become a powerful strategic asset. By leveraging the diverse skill sets of specialized testing firms– whether for large language models (LLMs) or other complex systems – organizations can gain a more objective view of their security posture from an outside perspective.

At Adobe, we have a comprehensive security testing program dedicated to rigorously testing our products from an adversary-aware perspective. As part of this testing regime, we recognize the value of collaborating with the external security community and researchers to strengthen our product security for broader coverage. In this blog, we will explore the strategic value of OPT and how Adobe's hybrid approach—combining authenticated black-box and source-code assisted testing—helps our internal product security teams more effectively identify and remediate vulnerabilities.

The strategic value of outsourced penetration testing (OPT)

While internal engineering teams are essential for building secure systems, bringing in outside expertise—specifically through OPT —offers an invaluable advantage. At Adobe, we recognize the strategic value of leveraging external testing capabilities and partnering with pentesters who can provide unique approaches.

Fresh eyes = fresh perspectives

Internal testing is a crucial part of securing our products, but Adobe recognizes that relying solely on our in-house teams can limit the full potential of our security assessments. Internal testers typically have a deep familiarity with an organization’s systems and products, which is invaluable for identifying issues rooted in Adobe’s unique infrastructure and workflows. To complement these internal efforts, our outsourced experts bring in diverse experience and nuanced perspectives from testing across various industries and environments, helping test Adobe’s products from multiple angles.

To conduct holistic testing, we regularly rotate and diversify our pool of external pentesters to support us in approaching our systems with a fresh set of eyes to spot potential weaknesses and vulnerabilities.

Specialized testing for focused results

Outsourcing penetration testing allows us to match specific skill sets to the unique needs of each product. At Adobe, our diverse product ecosystem presents unique security challenges, and outsourcing allows us to select penetration testing teams with specialized expertise tailored to each product's specific needs. Given Adobe's latest focus on AI innovation, we rigorously test our AI products, like Adobe Firefly, for LLM-related risks by engaging specialized AI experts to identify security vulnerabilities, safety concerns, and ethical issues.

Authenticated black-box testing

Authenticated black-box testing is a type of penetration testing we employ where the external testing team is given limited access to the application, typically in the form of low-privilege user credentials. The goal is to simulate the perspective of an authenticated user, allowing testers to identify vulnerabilities that could be exploited by a legitimate user with malicious intent. We find this methodology valuable because it allows us to more closely mirror real-world scenarios, providing a realistic simulation of how an attacker would attempt to exploit vulnerabilities.

When engaging with OPT vendors, Adobe follows a structured, end-to-end process designed to align testing efforts with the unique characteristics of its products, environments, and security objectives. This approach enables Adobe to maintain a consistent and streamlined process so that we prioritize the right risks and deliver relevant, actionable results.

For authenticated black-box testing to be successful, it’s essential to establish well-defined rules of engagement to maintain stability during testing, particularly when working in production environments. These rules allow penetration testers to explore the application in a free, yet structured, manner while simultaneously preventing inadvertent harm to the system’s integrity or data.

Source-code assisted white-box testing

For applications requiring a deeper analysis, our OPT program also includes a source-code assisted approach. In this method, we work with known, vetted, and trusted third-party penetration testing team to conduct a thorough review of our application’s source code to identify potential vulnerabilities.

Unlike black-box testing, where testers rely solely on external probing, reviewing source code can better enable testers to identify flaws that might not be easily exploitable but still pose significant risks. This approach provides greater insight into the application’s security posture, uncovering issues that might not be visible through traditional testing methods. Additionally, by leveraging the source code of an application, we can engage specialized, research-driven pentesters with expertise tailored to the specific needs of our products, helping enable a more comprehensive and targeted security assessment.

Conclusion

By adopting a hybrid testing methodology that combines authenticated black-box testing with source-code assisted testing, our OPT program achieved a 25 percent increase in discovered vulnerabilities over the course of a year. This highlights the value of a carefully curated approach –leveraging both testing methods, a well-defined test plan, and the right expertise – to maximize the effectiveness of our OPT efforts.