TL; DR: The more information provided for pen testing the better the outcomes.

This blog explores the value in white box penetration testing. It focuses on explaining the concepts and why you should be doing white box testing. Keep an eye out for future posts that delve into the technical details of white box testing.

Penetration testing is an essential part of most organisations security programs. It helps to identify gaps in defences so companies can better protect themselves, but not all pen testing is created equal. At Tanto Security we favour white box testing and the reasons why are outlined below. So, why should you be opening up the white box on your next pen test?

If you have found yourself reading pen test reports full of generic issues you would recognise that there are probably better ways to do things. If you think that the tester didn’t get something or felt that they didn’t fully understand how the application was built and the technologies it uses, then it might be time to start thinking about white box testing.

The reality is that Penetration Testing is delivered under the constraints of a commercial engagement. Consultants are assigned an amount of time to deliver the work and often only given the bare minimum information needed to access what they are testing. Thinking about this from a threat model point of view - yes it will take away the low hanging fruit and make your business a slightly harder target - but it doesn’t fully address the other threat of a motivated attacker who has time on their hands to learn about the system without an external time constraint. The threat model should also consider that an attacker needs to find one vulnerability whereas a pen tester is trying to find as many as possible in the time allocated.

With any software development project there is generally huge amounts of ambient information understood by those involved without having been made crisp or specific. Which is why it’s best to provide as much information as possible as it will provide the tester with a blueprint of how the system works. The chances of a tester not getting a full picture or missing an issue increase exponentially as the amount of information provided decreases. The best way to even up the ledger and hand the advantage back to the people you want to find the security issues is to give them as much information about the system as possible.

What is White Box

Varying definitions and different interpretations of terms at this point is almost a feature of the cyber security market. But for this article let me define what we classify as white, grey and black box security testing here at Tanto Security:

• White box testing involves the consultant doing the pentest having the same access to information that a developer working on the app has. This typically includes technical documentation, user accounts for every level of privilege, guides on usage and importantly the source code.
• Grey box testing is when we are provided with some information about the app. For example the technologies it uses, user accounts and other general information that most users would have access to. This is information that given enough time we or an attacker could find.
• Black box testing usually involves very limited information such as only the target URL. This is generally unauthenticated and while it may be a good starting point if you have never done security testing before or budget is very limited we typically do not recommend this.

Why White Box?

There are benefits in moving to whitebox testing across all of your key systems. The clearest reason is efficiency - why should the pen tester be spending their limited time trying to guess how things work. In the hands of a skillful pen tester, information like network diagrams and source code will provide insights that might take days or weeks of trial and error to exploit.

Figure 1. The progression of pen testing in meme form

As people have become more aware of the implications of poor security the chances of having the business agree to allow whitebox penetration testing has increased. It was true that in the past there would be objections to sharing this type of information. By outlining the potential benefits coupled with the maturing of security in general we think every pen test should start with the explicit aim of being white box. Then if there are legitimate reasons some information can not be shared, it can be moved back to grey or black box testing.

Root Cause Analysis

The other key outcome that you’ll start to see in your reports is that root cause analysis becomes more meaningful. The report will contain the relevant code snippets and as the consultant has a deeper understanding they are in a better position to explain what the issue is and how it potentially came about. For example a security vulnerability might relate to the integration between the authentication and the backend database. The consultant may have been led to look at those entry points via an architecture diagram and the root cause analysis beyond just the vulnerability could identify problems with how the systems are integrated.

The other huge advantage is that developers can see the code because we can provide the snippets with the findings. This means the developer who is tasked with fixing the issues can see exactly where it needs to be fixed. This also helps to provide greater context for the issues found and might mean the developers of the app will be able to apply what they learnt from the pen test report to future iterations of the app.

When to do it

We think you should be doing white box testing all the time. If this isn’t feasible then any application that is critical to your business should be the first in line to be tested with a white box approach. Even if grey box is as far as you can go, think of ways that you could help the consultant delivering the work to focus their attention on the right areas. As we mentioned earlier in the article ambient information which developers know about the system can help provide insights to the tester. For example we recommend sharing how you would attack the app and the main areas of concern. This information is helpful to a tester and can be discussed during the scoping phase or prior to the tester starting.

But if we tell you everything you are just cheating?

People can be hesitant to hand over the keys to the kingdom to any third party. Trust is an important part of these engagements and sometimes it can take time to build. But the honest answer is that security providers who want to do white box testing are requesting those types of engagements because they know that the results are better.

We also concede that by allowing us access to the source code it can be seen as a ‘gimme’ and dismissive of other defensive controls that may be in place. This can be harder to justify to people outside of the security realm who may want to rule out results that include source code, arguing that an attacker wouldn’t have that level of access. To counter this we believe the proof is in the results and if the overarching objective for the pen test is to improve the security of the in scope systems then white box is the only way to go.

That being said, the security provider should have a clear process around how the data is handled. Ideally the tester would be given temporary access to the source code via GitHub or similar. If the source code needs to be transferred, this should be done securely with encryption and the copy of source code should be deleted upon completion of the engagement.

About the Author

Marco Cantarella is one of the co-founders and Commercial Director at Tanto Security. He has worked in cyber security for over 10 years on the operations and commercial side of penetration tests, compliance, incident response and red teaming. You can connect with him on LinkedIn or get in touch at marco@tantosec.com or 1300 182 686.