Extending Your Test Automation Pack Into Security

DevSecOps is the current buzz word for many people and security organisations are receiving numerous queries about implementation from their clients.

There is however an obvious flaw for a lot of people, which is that most organisations are not yet fully-fledged DevOps, if at all. Trying therefore to switch straight into DevSecOps could prove problematic for obvious reasons!

Aligned with this aim of reaching the Valhalla of DevSecOps, is the need to automate elements of Security Testing.

However, you do not need to be “DevSecOps” to add Security automation into what you do.

THE REQUIREMENT

“Our requirement is to add Security into our earlier test phases and not just at the point we are ready to Go Live?”

This requirement started life as “can you help us move to DevSecOps”, but then following more detailed discussions with the client, and through understanding their current capabilities & change roadmap for the next 12 months, it quickly evolved into what was actually a requirement to test, from a security perspective, earlier in the lifecycle. Whilst the client had an objective to move to DevOps, they agreed this was unlikely to happen within the next year.

This client was quite representative of many out there. In addition to a large internal network, they have several Web Applications which are complex in nature and they are developing, testing and promoting code to the live estate on an almost weekly basis.  They are aware of the shortcomings of a single Pen Test, which they have scheduled and have performed by a third party on a annual basis.

Their scheduled Pen test happens in January, but beyond the remediations from this Pen test, they have little or no idea as to whether during the subsequent 11 months of the year (or 44 code releases), they have actually introduced vulnerabilities as a result of bad coding practices , bad designs, insecure configuration and so on.

EXPANDING UPON CURRENT CAPABILITIES

So, we had established 2 important elements.  

1. The client is not currently using solid DevOps practices, and is actually unlikely to move to that model any time soon.
2. The client acknowledges the shortcomings of the once per year Pen Test and wants to move to something much more continuous and automated in the interim.

Many organisations have strong test capabilities in place already. Not necessarily from a security perspective, but certainly from a Functional and Non-Functional perspective, where they hold automated test sets which are being executed overnight, numbering hundreds of tests and giving really good insights as to the quality of a particular build.

The organisation we are looking at here also had a strong test automation capability, with some highly skilled people building and maintaining a robust automated test pack. They were routinely running 400 automated functional tests overnight giving a good coverage of the application. They had a mature test process with a skilled resource pool.  

What was required, was to expand upon this test automation capability and add security to the automation.

ADDING SECURITY TO EXISITING TEST AUTOMATION

We engaged with the client’s automation team with the objective of adding an automated Security element to what the client was already doing on a day to day basis. As mentioned, the client was already relatively mature from an automated test perspective, using Jenkins, JMeter and functional automated tests using Selenium.  

The Precursor automation team added another element to this automation, by engineering a call to a Vulnerability Scanning Solution at the point when a build was proven as a good candidate. The order of execution through the clients existing test packs, was to run the Functional Automation tests, followed then by the initiation of a mini Performance check, again in an automated manner.

We then added a conditional call to a Vulnerability scanner.

It became clear we should only logically start a security scan, when the results from the previous automated tests held a high enough % pass rate to consider it a good candidate build, and therefore worthy of a security scan.  The reason for this, is that some Security scans can be quite heavy on processing and time, therefore we wanted it only to be performed against a build we had some degree of confidence in.  If for example, we only had a 10% pass rate from our functional automation, there seems little point in expending time & resources scanning for vulnerabilities on an application build, which was looking like it would be sent back to dev and taking significant code changes the next day.

Note- We did eventually reduce the security scan timing by configuring the scanner against the technologies being used by the application under test. Many scanners will attempt, by default, to scan applications/infrastructure for everything, as they have no knowledge of the technology stack.

Once the automated decision making had taken place following Functional and Performance test and a positive decision to proceed was established, the security scan was invoked via an API call to the scanner which involved amongst other things, creating a customised class wrapping on a Json Object and passing target IP’s and URL parameters to the scanner.

When the scan is completed, a report is pulled from the scanner, detailing any findings, with Vulnerabilities listed with an associated CVSS score. Due to the fact that all this was automated, it quickly formed part of the test process, providing a clear position as to whether any particular build is relatively better or worse than its predecessors. This assessment can also be automated, by comparing results from previous scans and associated CVSS, to the results from the latest scan and highlighting any Regression which may have taken place.

We always recommend Penetration Testing as the best initial mechanism to establish the security position of an application. However, Pen testing can be expensive and time consuming, and if done infrequently, then you may find you have large amounts of vulnerabilities to remediate all at once. This is not a good position to be in and it is far more desirable to establish the baseline and then test almost continuously.

It is important, as with all testing, that you try and have a test environment as representative of Live as possible. Ideally this level of automation for security resides in a Pre-Production replica live instance. If not scaled as per Live, then at least representative enough to allow for extrapolation of the Performance test results and hardening from a security perspective.

SUMMARY

In the end the client did not want or need DevSecOps. What they wanted and needed was an ability to test in an automated manner, adding security testing into their existing test processes by expanding upon their existing automation capabilities.

There will always be vulnerabilities automated tooling cannot find, however the ability to establish a baseline security position via a Pen test and from there on, test and assess quickly and on demand, highlighting any deviation from that baseline is critical to any organisation who need to promote change frequently.

Precursor Security can help with Test Automation for any stage of the SDLC and can also assist with Vulnerability scanning along with traditional penetration testing. Please get in touch to discuss.