Penetration testing is a critical part of the cybersecurity industry. It’s been around for a long time and pentesting fatigue is a real thing. However, there are some who feel that pentesting itself is dead. Are they right? How can we tell if penetration testing (alias include PenTesting, Pen Testing) is still relevant or not.

Penetration Testing can conjure up different things for different people. But that shouldn’t necessarily be the case. This article talks about pentesting and offensive security more broadly and includes what you should be looking for when engaging a provider. It touches on the differences between offerings and will give you an idea on what types of things you should be requesting from your providers.

Not everyone thinks all day every day thinking about penetration testing like we do. This means that on paper and from brief discussion with different providers it can be difficult to determine what you are getting or even where to start.

Black, White and Inbetween

Varying definitions and different interpretations of terms at this point is almost a feature of the cyber security market. But let’s define what we classify as white, grey and black box security testing here at Tanto Security:

  • White box involves the consultant doing the pentest having the same access to information that a developer working on the app has. This typically includes technical documentation, user accounts for every level of privilege, guides on usage and importantly the source code.
  • Grey box testing is when we are provided with some information about the app. For example the technologies it uses, user accounts and other general information that most users would have access to. This is information that given enough time we or an attacker could find.
  • Black box testing usually involves very limited information such as only the target scope. This is generally unauthenticated and while it may be a good starting point if you have never done security testing before or budget is very limited we typically do not recommend this.

In our previous blog post we covered the benefits of and why you should be doing white box pentesting. White Box pentesting is a way to ensure you are getting the most coverage and if you’ve hired experienced consultants it allows them to apply their knowledge to uncover vulnerabilities that aren’t as obvious from the outside in.

Notes on Automated Pentesting

The below twitter thread from Mick Douglas lays out nicely the arguments for and against automated pentesting.

Read “Mick Douglas on automated pentesting tools” on X

As this thread highlights for the most part automated pentest != pentest. But every pentest does involve an automated component. There are some great tools out there so it makes sense to utilise these. The experienced consultant will use the tools to perform certain tasks, but there will always be a manual process that isn’t possible to automate. To analogise, even with the best tool box in the world you won’t be able to make a beautiful chest of draws without knowing how to use them.

Going In Depth

Methodology and Approach: From Scanning to Beyond

If you spend any money on offensive security you want to make sure you get value. This applies from a one off app pentest through to an end to end Red Team. As the defensive side lifts their game we as offensive security consultants have had to lift our game. At a point in the past running a few scans might uncover some interesting SQL injections or XSS. In what should be seen as a massive win for the offensive security community a lot of modern frameworks and technologies have addressed these issues out of the box. That coupled with more security conscious developers and tech teams mean this low hanging fruit isn’t as easy to pick off. This is where the offensive side’s methodologies have had to move forward and become more robust.

To address this methodologies from ASVS , CREST Defensible Penetration Test and PTES to name few have been increasing coverage. No one methodology covers everything perfectly and the best pentest firms leverage these and couple it with their own experience to modify and adjust based on what is being tested.

Part of the reason for this is enterprises have adopted more technology solutions to solve business problems. This has led to diversity and complexity in environments. For example the web applications being developed today are increasingly complicated across the code and tech stack. Developers can write applications in different languages from ruby to python to .net to php to golang to javascript and many more or a mix of these. On top of this are the different frameworks that developers can use in these languages, each adding a layer of complexity. The diversity also affects the infrastructure of applications with web apps hosted on prem or in the cloud, virtualised or serverless. The choices that a developer has to make are endless.

In turn this affects how you pentest these apps. The tester needs to be aware of what types of technologies are predisposed to what types of bugs. They also need to know which tools and techniques to use in order to get effective results and be able to identify which technologies are used (this is where whitebox helps a lot). In this kind of environment the idea of a single methodology or tool that effectively assesses the technology in scope is not practical.

Beyond penetration testing more mature companies have started to focus on the bigger picture of their overall security. This has led to a big uptick in Adversary Emulation and Simulation exercises across the industry.

  • Adversary Emulation is based on attacks that have been shown to already exist in the real world. In this type of exercise we would take a known attack sequence and copy the tactics, techniques, and procedures (TTPs). Think replicating a known ransomware crew’s modus operandi.

  • Adversary Simulation would include attacks that are tailored to an environment. Think more advanced Red Team’s where we are mapping capabilities to understand where weaknesses are and performing attacks based on that. The type of attack used might be unique to that environment.

This move in the industry in now being enforced by a number of regulators around the world. Here in Australia it’s the Council of Financial Regulators via CORIE and in other juristictions includes CBEST in the UK, Singapore ABS Red Team Guidelines and the TIBER-EU Framework from the European Central Bank.

The common theme across this is that a good chunk of the industry is actively trying to move away from providing generic engagements. Part out of wanting to continue to offer value and part out of necessity. It is trending in the right direction with more education about what good offensive security should involve it will keep moving that way.

Art + Science

We have always thought that true Pentesting involves not just science but a little bit of art. The best pentesters are creative thinkers - generally one of the reasons they ended up as pentesters is because they look at things differently. If every pentester was to deliver the same result given an application then the case could be made that every pentest should be fully automated. But our own experience and industry veterans know this is not the case.

This point can be hard to quantify but is worth keeping in mind next time you speak to your providers.

Common Mistakes

Before we wrap just a few quick points on where things might be going wrong in the current world of offensive security. The things listed below are issues that would be pointed out by anyone who’s been involved in the industry and are worth reiterating:

  1. Unrealistic timing and budget
  2. Not scoping correctly
  3. Not doing enough due diligence on who to hire
  4. Poor prep work and not providing all the info
  5. Failing to address the issues identified

These are coming from the vendor side and we fully appreciate the realities of being within the constraints of an organisation. This can mean despite best intentions budgets get squeezed or things don’t get fixed. In this instance speak to your providers and come up with a plan on how you can still get what you need out of the engagement while operating within the constraints of the organisation.

Conclusion

Is pentesting dead? Well, we don’t think so but maybe some forms of pentesting should be. For example the idea of an all encompassing pentest and a one size fits all approach is moving to a much more specialised and targeted test in order to deal with the increasing choice and complexity of IT environments. The good news is that the core concepts of pentesting will still be relevant even as things change around us.The challenge is to continue to evolve at the same pace as everything around us.