However, setting up a relevant and effective CI/CD pipeline for each project context can be complex. Technologies vary, security requirements can differ, and target environments are not always identical. Given the ambitions and challenges posed by creating a unified CI/CD pipeline, it may not always be prudent to leverage IaaS or on-premise services, which also require infrastructure team investments. Cloud (PaaS) solutions offer a good middle ground between customizing the CI/CD pipeline and ease of implementation. Cloud solutions also allow for on-demand resource provisioning to better adapt to business needs.

There are numerous cloud-based CI/CD solutions that can potentially meet both security and efficiency requirements for the development pipeline. In this article, we aim to present our perspective on Amazon Web Services (AWS) solutions, which remain one of the market leaders.

What can AWS CI/CD services offer in terms of features and added value?

If you are not familiar with AWS CodeCommit, CodePipeline, CodeBuild, or CodeDeploy, we offer an introduction to better understand the workings of the AWS DevSecOps environment. To provide an overview of the tools offered by AWS, we describe the functionality of these different services in the following paragraphs.

Let’s start from the beginning: From DevOps to DevSecOps

DevOps is a key element in the software development lifecycle of companies. DevOps relies on CI/CD tooling and is  pipeline on which the evolution of source code into a production-ready application depends. CI/CD accelerates the phases of build, test, and deployment to increase the delivery frequency of applications. This acceleration is made possible by automating many tasks within a CI/CD pipeline, which is a series of actions leading to production deployment.

DevSecOps adds security aspects to DevOps and relies on certain internal tools within the CI/CD pipeline. These tools integrate at every level of the CI/CD pipeline to scan the source code (SAST – Static Application Security Testing), dependencies (SCA – Software Composition Analysis), and more. The goal, as discussed in our previous article, is to integrate security as early as possible. The CI/CD pipeline is a significant component in ensuring the security of developments. One could even say that the CI/CD pipeline plays as important a role in secure development as Identity and Access Management (IAM) does in identity and access management.

CI/CD in AWS

AWS offers a multitude of services that not only provide classic infrastructure services but also allow the establishment of continuous development pipelines (from source code to deployment), while ensuring proper security testing.

The orchestrator CodePipeline organises and links the different stages of the CI/CD pipeline. This tool coordinates the progression within the CI/CD pipeline based on the results of other tools and services. If one of the tools returns a failure code, the pipeline can be blocked if necessary. The reasons for a pipeline failure can vary, such as insufficient code security score or tool deployment failure.

Code Management: SCM and AWS CodeCommit

Code version control systems (or SCM: Source Code Manager) are essential tools for collaborative code editing during  development and serve as the starting point for continuous integration pipelines. Currently, only three SCMs offer native integration: GitHub, BitBucket, and AWS CodeCommit. For any other integration with a non-natively supported SCM, you can create a serverless Lambda function-based routine and a webhook (HTTP notification) to download source code to AWS S3 with each developer commit.

AWS CodeCommit is the SCM service offered by AWS. It’s a code hosting service that supports version control and collaboration, similar to GitHub or GitLab, with Git commands. The advantage of AWS CodeCommit is its full integration with the AWS environment, making it easier to interconnect with other AWS services. Using AWS CodeCommit also allows for the use of AWS Identity and Access Management (IAM), avoiding the duplication of identity repositories and role management within a third-party SCM. All of this makes AWS CodeCommit a suitable solution when used within an entirely AWS environment due to its close integration with other AWS services. However, AWS CodeCommit offers relatively limited features compared to GitHub such as user experience and interface, and has a smaller community than GitHub or GitLab. If the CI/CD pipeline includes multiple solutions external to AWS, other solutions such as GitHub or GitLab will likely provide more flexibility.

Build Phase: AWS CodeBuild

Once development is complete, AWS CodeBuild takes over. This tool can be used for both compiling/building an application and running tests via CI runners. The service executes the instructions provided in an input file called buildspec.yml. It is a versatile tool, similar to classic CI tools like GitLab CI or GitHub Actions.

AWS CodeBuild also allows for running security tests (SAST, SCA, etc.) by installing and using applications on its runners. Take SonarQube, for example, a code quality tool with a SAST module for scanning source code to identify vulnerabilities. The execution works as follows:

1. When the source code is modified, a webhook notification (HTTP POST request from the SCM) is sent to AWS (in practice, this event is managed by AWS EventBridge or AWS CodePipeline), triggering the test.
2. The source code is duplicated on the CI runner, which scans it and produces a report.
3. This report is then sent to a SonarQube server (on-premise or on an EC2).
4. After analysis, SonarQube produces a final report indicating the code’s security level.
5. These results are sent to CodeBuild, which interprets, based on the conditions in the buildspec.yml file, whether the test was successful or not.

Again, the key advantage of CodeBuild is its integration with the environment, allowing close collaboration with other AWS services. For example, it’s easier to assign specific roles to CodeBuild projects, use AWS Secrets Manager (for secret management), or enable deployment with AWS CodeDeploy.

Deployment: AWS CodeDeploy

The deployment of an application marks the end of its development cycle. Within AWS, deployment is achieved through AWS CodeDeploy. Its role is to retrieve the artifacts and necessary configuration files from dedicated S3 buckets and deploy them on the chosen server (EC2, etc.). AWS CodeDeploy differs from AWS Elastic Beanstalk, which deploys an application solely based on its code (usually not supporting compiled languages like C/C++).

CodeDeploy operates by deploying code to any type of server, whether hosted by AWS or not. Its operation is simple: an agent (CodeDeploy agent) is installed on the target server. This agent is responsible for downloading the artifacts, installing them, and launching the application.

It is necessary to define in advance the instances involved in the deployment and assign them an arbitrary AWS tag for identification. All these instances then constitute a “deployment group.” When deployment is initiated, CodeDeploy selects the relevant instances and publishes its instructions. However, communication is initiated by the target instance; the CodeDeploy agent contacts the CodeDeploy service by polling for new instructions (polling mode). This communication method avoids opening ports, enhancing the security posture of the instance.

AWS CodeDeploy is an effective tool for deploying code to any type of infrastructure. However, it requires the installation of an agent managed by AWS on the instance where the code is deployed, which may not always be desirable depending on the client’s context. Polling by EC2 instances may impact the performance of a critical application or be detected as malicious by Endpoint Detection and Response (EDR) or Network Detection & Response (NDR) systems.

Securing the AWS CI/CD Pipeline

Given the critical role of the CI/CD pipeline in application development, it is essential to secure this infrastructure, including tooling, integration, and pipeline configuration. Below, we summarise some areas to consider when implementing an AWS CI/CD pipeline, which can be managed through the creation of AWS policies to alert or enforce their application.

Flow Management

By default, flows to AWS managed services (CodeBuild, CodeDeploy, etc.) transit over the internet before returning to the client instance of the resource. To avoid sending all flows to AWS services over the internet, we recommend setting up VPC endpoints. These network access points allow instances within a VPC to contact AWS services as if they were deployed within the VPC.

Secret Management

Secrets required to access services or other APIs should not be stored in plaintext in SCMs or pipeline configuration files. To avoid any leakage of confidential information during legitimate or unauthorised access to these directories, we recommend implementing an AWS Secret Manager to store secrets (e.g., SonarQube API keys) and distribute them to services only when necessary. Retrieving a secret is done through an API call to this vault, with privilege verification.

Supervision/Monitoring

Like any infrastructure, the CI/CD pipeline requires monitoring. Native AWS solutions for service monitoring include AWS CloudWatch for log collection, AWS EventBridge for creating alerts, and AWS SNS/SQS for sending notifications to predefined groups (email, SMS, push notifications, etc.). Monitoring the CI/CD pipeline allows for alerting against potentially dangerous production releases, for example, if a project attempts to bypass implemented security policies.

Identity and Access Management

Privilege management within AWS is based on Role-Based Access Control (RBAC) whereby each user action requires specific permissions. For example, if a user wants access to an S3 bucket, they must first obtain read permission associated with the corresponding S3 resource. It is essential to adhere to the principle of least privilege, which involves assigning clients (users and services) only the rights they need. AWS permissions allow for complete configuration of client access to each service/resource. However, the granularity of rights can be cumbersome to configure in a large-scale CI/CD infrastructure. AWS offers predefined roles that allow for quick application of sets of permissions. Still, these predefined roles often do not adhere to the principle of least privilege. Therefore, it is important to create roles that apply the principle of least privilege without delving into micromanagement of rights.

Our Beliefs on AWS CI/CD

The CI/CD solutions available in AWS cloud are interesting and natively integrated with other AWS services. Native integration is particularly useful in the case of a pipeline hosted entirely by AWS. When most of a company’s infrastructure is already migrated to AWS, you can take advantage of interconnections between services and powerful access management and monitoring solutions with minimal additional configuration. However, for a simple and isolated use case, AWS CodeCommit or AWS CodeBuild might not be the preferred choice. Solutions such as GitHub and GitLab offer more comprehensive solutions, better integration with other vendors, and a more user-friendly interface. Similarly, regarding security, AWS does not offer native CI/CD security services for code validation (SAST, DAST, etc.). AWS does not provide native integration, but third-party services can still be integrated relatively easily.

*Example of BitBucket Integration in AWS CodeBuild – Source

Cet article CI/CD in AWS: The Solution to All Your Problems? What You Need to Know. est apparu en premier sur RiskInsight.

How to ensure the security of your applications despite outsourcing their development?

Integrating security into projects is an important process for companies to define and integrate security aspects into products as early as possible. This avoids increasing the cost of remediation if it has not been planned and is implemented at the end of the project.

In the context of developments, Agile Security and DevSecOps define the processes and tools to be put in place to integrate security as early as possible, as presented in our previous article giving examples.

These methods are often defined on internal developments. However, it is often the case that companies call on external service providers to develop a particular application or functionality. In this case, it is important to ensure that these providers follow rigorous security practices and that they integrate security into their development processes to the same standards as the requester. This leads to the following question:

External developments: how to maintain confidence in externally developed code?  

In the remainder of this article, external code is defined as all code elements that have not been developed through an internalised CI/CD chain. For example, a freelance developer using the internal CI/CD chain or an enterprise workstation is not considered external code.

In addition, we will consider two models of application delivery depending on the development model used by the provider:

• delivery of the source code itself
• delivery of the executable, i.e. the already precompiled code

It is important to note that these two application delivery models have different implications in terms of cyber security and DevSecOps.

Code delivery

In the case of code delivery, external providers hand over the code they have written, usually in the form of source files (e.g. .java files for Java code), to the company. The company can then audit, compile and deploy the code on its own servers.

Code delivery has several advantages. The first advantage is flexibility: by delivering the source code, the company can easily make changes and customisations to the code. It can also integrate the code into its existing development and deployment environment (CI/CD) containing all the pre-configured security tools.

The company then does not have to place its trust in the security of the provider’s CI chain over which it has no control. In addition, the company with access to the source code can also audit it and thus verify that it is secure. These audits tend to be more comprehensive as the auditor has access to much more detail about the operation of the code and can perform both static and dynamic analysis of the code.

On the other hand, code delivery has some disadvantages. The company must have the skills to adapt the build and deployment stages to the production context. If these skills are not available in-house, this can lead to additional costs.  

Here are some good practices to maximise confidence in the delivered code:

• Share as early as possible (contract, kick-off meeting) the expected requirements on security in development, software versions, internal tooling used for deployment, confidentiality of source code, etc. Some clients require external developers to have a certain level of certification or training (for example, a level of training on Secure Code Warrior, in a certain programming language).
• Define and contractualise commitments on the remediation processes for identified vulnerabilities after code delivery and the associated monitoring (monitoring tools, SLAs, etc.)
• Implement a hash or signature type control on the code sent to ensure its integrity and define the methods for secure transfer of the source code with the service provider
• Integrate the code received into the existing CI/CD chain, including the Infrastructure as Code (IaC) files
• Carry out the functional security tests initially defined during the threat modelling: Evil User Stories and Security Stories

Some organisations may be faced with a situation where the notion of external developers corresponds to developers from other entities within the same group. These entities may have their own CI chains but depend on the CD or CI/CD chain of the central production team.

In these cases, an interconnection of the different CI chains to the central CI/CD chain can be considered. This solution allows the different teams to develop with the tools that best suit them.

The level of security provided by the project CI/CD chain is ideally equivalent to that of production but this is not necessarily the case. The production CI/CD chain controls the code to be deployed.

However, security control is often carried out too late in the development process. To ensure effective security in developments, it is crucial to ensure that security is integrated from the beginning of the development cycle (shift-left). To address this, it is recommended to provide self-service security tools for project teams to identify vulnerabilities early in their development using the appropriate target tools.

Otherwise, the security tools in the production CI/CD chain will ensure compliance with the group’s rules without slowing down the production release if automated security controls have been put in place within the project chain.

This solution also allows production to ensure the use of images (systems, docker, etc.) or artefacts (libraries) validated by the company.

These interconnections between the different pipelines can, for example, clone the branch to be deployed by the product team in order to push them into the CD chain. However, the production teams must have the appropriate rights. Technically, the model for managing the rights granted (ideally temporarily) must meet both the need to facilitate execution and the need for rights provisioning (manual vs. automatic), while limiting access to all branches or projects in order to respect the principle of least privilege.

Most of the good practices mentioned above also apply to reduce the time to production.

Although the methods described above appear to be the most effective for gaining control over applications developed by third parties, companies sometimes find themselves receiving executables without access to the source code. This may be due to licensing restrictions, for example. In this case, some of the good practices outlined above do not apply, and it is necessary to rethink how to integrate changes into production so as not to neglect certain security aspects.

Executable delivery

In the case of executable delivery, external providers hand over an executable file (e.g., an .exe file for Windows servers) that can be directly executed by the company without compilation. This delivery method is often used for commercial software that still requires some configuration adjustments.

In this context, the integration in the deployment chain is much more limited and only a few classical CD steps can be performed without the security steps of the CI chain being verified:

• Performing an artefact scan
• Performing a DAST scan to detect the most common vulnerabilities
• Performing penetration tests

Reports from the security tools of the development provider’s chain can also be requested. This must be included in the service contract, along with the security requirements for the level of security of the code.

Finally, a signature of the code to ensure its integrity is necessary at the time of the exchange and the executable. For this purpose, it is better to use signatures via certificates rather than hash prints, since the latter make it possible to verify the origin (non-repudiation) in addition to the integrity of the executable.

In conclusion, it is important for companies to ensure the quality and security of the code delivered by external providers, especially when the latter are developing code on external CI chains. There are several ways to convince yourself of the security of the delivered code:

• Clear and precise contractual clauses can help define the expectations and responsibilities of each party with regard to the quality and security of the code.
• Sharing specifications and security expectations with external providers can also help ensure that the delivered code meets the company’s requirements.
• Integration with internal development chain tools can facilitate verification of code quality and security, as well as the implementation of automated testing. These integrations raise both technical and process challenges that must be anticipated to facilitate the deployment of external developments.

By implementing these different approaches, companies can increase their confidence in the code delivered by external providers and ensure the security of their application.

Cet article Stay in control of your external developments est apparu en premier sur RiskInsight.