What is DevSecOps and its role in CD?
DevSecOps stresses the importance of incorporating security into the software development lifecycle (SDLC). Building security into your team’s culture, process and tools avoids siloes and ensures that rapid delivery is not at the expense – or mercy – of security vulnerabilities.
Understanding DevOps
Before we dive into the security side of things, let’s take a brief moment to talk about DevOps.
Closely allied to agile software development, DevOps is about creating a culture of collaboration between software development and operations with everyone sharing a common goal of delivering valuable, working software to users efficiently. To put this into practice, DevOps advocates for robust, automated processes that shorten delivery times and promote rapid feedback, creating a cycle of continuous improvement.
So far, so good, but just like when you’re running to catch a train, when speeding up the software delivery process it’s worth checking you’ve not left anything important behind. Unfortunately this has often been the case with security, and that’s what DevSecOps aims to address.
DevOps Security
The potential impact of security vulnerabilities in your software cannot be overstated.
We’re not just talking about financial and reputational damage. Software security breaches have resulted in vast amounts of sensitive data being leaked, from internal documents to unreleased products and customers’ personal information, including passwords. Depending on jurisdiction, leaking user data can result in severe financial penalties as well as legal liability.
Despite all this, security is often thought of by development teams as an encumbrance rather than an asset. Security audits and reports slow down progress and stop you getting your latest shiny features out of the door and into the hands of your users. But that mentality is playing right into the bad guys’ hands; if we choose to ignore the importance of security in the SDLC, we run the risk of enabling the next Wannacry or Heartbleed.
While those exploits became headline news, their origins were far more pedestrian. Vulnerabilities exist because humans write code and humans make mistakes. Sometimes those mistakes have been made by others before and are easier to recognize (if you know where to look) and sometimes they are new, perhaps resulting from a combination of factors that have not previously come together in quite that way.
To add to the complexity, almost all software incorporates dependencies – whether open-source or proprietary – and there is no guarantee that that third-party code is free of vulnerabilities either.
So how should security be brought into the SDLC? The DevSecOps approach is to apply the same principles of collaboration, shared responsibility, and automating whatever can be automated that have enabled faster, more stable releases, to security practices.
Shifting security to the left
The waterfall development process takes a linear approach, starting with requirements gathering, followed by design, build, integration and test phases, and finally – usually months or years after the start – release.
The test phase typically involved lengthy, manual processes of quality assurance, security auditing and acceptance testing. In a process derisively known as “throwing things over the wall”, the product would be handed off from one function to the next, with each team writing a lengthy report.
One of those stages belonged to the information security, or infosec, team, who would carry out a security report and produce a detailed list of findings and recommendations. These often required changes to the codebase, after which the software would move through the stages again to confirm everything had been implemented as expected (or not). With such a drawn-out process it’s little wonder that releases were a cause for major celebration!
Agile software development, the DevOps movement, and the rise of cloud computing have disrupted all of that. Competition is stiff and speed is of the essence. If you’re not addressing your users’ needs and fixing bugs promptly, they’ll soon find another provider who does. Delivering value regularly and being able to roll out fixes quickly has become the norm for many organizations, with a CI/CD pipeline a cornerstone of their operation.
Although the term DevSecOps might imply otherwise, DevOps was never intended to exclude security.
Unfortunately, however, the reality was that as boundaries were broken down between development and operations teams, the information security teams typically remained siloed. In an effort to address this, DevSecOps – together with several other versions, such as SecDevOps, DevOpsSec and Rugged DevOps – was coined.
DevSecOps emphasizes the importance of building security in from the start, extending the culture of shared understanding and responsibility to security concerns, and building security checks into the automated testing regime of a CI/CD pipeline. As with operations, the idea is that by shifting security to the left, it’s easier to incorporate security requirements and best practices than if they are retrofitted after the product has been built.
Putting the Sec in DevSecOps: Security as Code
For the uninitiated, it can be tempting to think that shifting security to the left just requires making your software available for testing by the security team as a stage in your CI/CD pipeline.
While there is a place for manual security testing, as we’ll discuss below, limiting your engagement with security requirements to a step at the end of the pipeline is little different to throwing the software over the wall and waiting for the report to come back.
The likely result is either that recommendations are ignored because they will take too long to implement, or that the whole process is brought to a crashing halt and the release delayed indefinitely while issues are addressed and the risks are assessed. Either way, it engenders an us-and-them mentality between security and the rest of the organization which doesn’t help either side get what they want, namely the release of useful, secure software.
So what does shift left methodology mean in practice? While there’s no one-size-fits-all solution, the following tools and techniques can help to integrate security throughout the SDLC.
Appoint security champions
While a culture of shared responsibility is important, it’s not enough to tell your development teams that they’re all responsible for software security. Keeping up with the latest attack vectors and malware developments is a full-time job, and you can’t expect everyone to maintain the same level of expertise. Bringing a security champion into a multi-disciplinary team to coach others and share best practices can help build understanding and promote a culture of collaboration with the security professionals.
Make security a design constraint
In order for developers to share responsibility for the security of the software they are building, security needs to be considered before any code is written. It should be woven into user stories, raised during backlog review meetings, and discussed when planning each sprint. When working out how to tackle a new feature, take the time to discuss the risks it might present and how to mitigate them.
Adopting a threat modelling strategy, such as STRIDE, or using a tool such as the OWASP cheat sheet, can help you stay on track while you learn these new skills. While considering security at the design stage won’t catch everything, it’s a good start and helps promote a DevSecOps culture.
Add automated security tests to the pipeline
Automation is a central tenet of DevOps. Not only does it speed up tasks and ensure they are performed consistently, but it also frees up humans to do more creative work. If you want to deliver software to users regularly and frequently, adding automated security tests to your pipeline is essential.
When writing automated unit, integration and end-to-end tests, security features should be considered just as relevant as any other functionality. If your team has been incorporating security requirements into user stories and discussing threat models as part of the design process, adding tests that cover security functions is a natural extension to that work.
Get expert assistance
In addition to the tests you write yourself, there are various security testing tools that can help to build confidence in the quality of your code. While traditional security scanning tools were not well suited to an automated CI/CD pipeline, CI/CD security testing tools are now available that have been designed for automation and integrate into the pipeline with results surfaced via the dashboard or fed straight into bug tracking tools. As with all automated tests, it pays to structure your testing to provide feedback as quickly as possible.
Static application security testing (SAST) tools perform static code analysis to check your source code for known security flaws, such as buffer overflows and SQL injection opportunities. Because static analysis is run on the source code, it can be run early in the CI/CD pipeline, as soon as changes have been committed.
SAST tools are language specific, and some integrate to the IDE to provide continuous feedback as you write or the option to run tests on demand. By directing developers to the specific line of code that contains the vulnerability, SAST tools offer targeted guidance which makes fixing the issue quicker. However, false positives can be an issue, so being able to mute or dismiss particular alerts is essential if you want to avoid the risk of all SAST results becoming background noise.
The corollary to SAST is dynamic application security testing (DAST). This takes a black box testing approach on the running application, checking for known vulnerabilities such as cross-site scripting, command and SQL injection, and insecure server configuration. Because DAST tools require the application to be built and deployed to a test environment, they are typically used later in the CI/CD pipeline.
Check your dependencies
Software composition or component analysis (SCA) is another type of automated security test that can be run early on in the process. As we said above, practically every codebase incorporates open source libraries and other components, and these can introduce vulnerabilities.
SCA tools should crawl not just the dependencies you’ve dropped in, but also their dependencies recursively up the supply chain – a perfect example of a job best left to computers, particularly when you consider how often new dependencies are added to a project.
As well as being run automatically as part of the CI/CD pipeline, some SCA tools are available as IDE plugins to provide feedback on the fly. As with static and dynamic security tests, SCA testing is limited by the vulnerabilities that the tools are aware of, so ensure the tools you choose are regularly updated with new exploits, and cover areas that are most relevant to your product and organization. However, when it comes to finding new and unexpected exploits that requires the human touch.
Bring in the Red Team
The concept of a red team originates from war games, where members of your own side are asked to play the role of the enemy in a simulated attack in order to identify weaknesses in your own defenses and strategies. The same approach has been used to great effect in software development, sometimes in the guise of penetration testing or ethical hacking.
For a red team to work effectively, they cannot be involved in the development of the system under test. Similar to exploratory testing, red teaming requires testers to think outside the box and use the software in ways other than was intended.
A red team can work either on a staging environment (ideally as similar to production as possible) or on a live, production system. If you’re going for the latter option, you’ll need a high degree of confidence in your product’s failure modes or very tolerant users (and C-level execs!).
Treat all vulnerabilities as bugs
A DevSecOps approach involves everyone taking responsibility for the security of your software. It’s therefore time to stop thinking of security issues as somehow separate from “regular” bugs. Any failures or vulnerabilities discovered in your system belong in the same backlog as all the other issues and should be prioritized alongside them.
Responsibility for fixing them resides with the whole team, not just the security champion. Taking this approach helps to build knowledge and expertise within the team, and you can take those skills to the next project you work on.
Keep watching production
Despite all your efforts in the previous stages of the SDLC, there is still a risk that a hacker will find a weakness in your live system. Firewalls and monitoring tools are still worth investing in to protect both your organization and your users. Runtime application self protection (RASP) tools are the latest addition to these defenses, automatically detecting and blocking suspicious behavior.
Wrapping up
Shifting security to the left means making it a consideration for the whole team at every stage of the SDLC. When you’re applying DevOps practices, that lifecycle iterates frequently, providing you and your team with regular feedback on how your software behaves and is used in the real world. Baking security into your CI/CD pipeline means you get regular feedback on the security of your application and can improve it in the same way as the rest of your functionality.
Checking for and protecting against known vulnerabilities will at least ensure your software keeps pace with attackers, rather than leaving you undefended from known exploits. Treating every new vulnerability as a bug to be triaged, fixed and tested for in future will make your software more robust over time.
Ultimately DevOps – and therefore DevSecOps – are as much about culture as they are about the tools and processes that enable rapid, frequent delivery of software. After all, it’s the people that make it possible. If you want to embed security into the SDLC, creating a culture of shared responsibility rather than targeted blame is key. Anyone should be able to raise a security concern and be listened to – whether that’s during sprint planning, code review, manual testing or on the live system – and everyone has a role in recognizing the importance of security and implementing it.