Vulnerability Assessment and your CI/CD Pipeline: Enabling Compliance using Chef Habitat and Security Scanning Solutions

Agentless and Agent-based Scanning Solutions

A Compliance Agent is software provided by a Security Vendor in order to supplement traditional network scans and authenticated scans conducted by a centralized software installation dedicated for such scans. These agents run on a Linux host as a daemon, or on a Windows host as a service, and report to the control plane supplied by the Security Vendor. Some examples of such agents are:

• Nessus Agent by Tenable
• Insight Agent by Rapid7
• Cloud Agent by Qualys

Because an agentless scanning solution needs to perform outbound network connections to all of its targets, including SSH connections for authenticated scans, the solution works best in a centralized-hub-and-spoke Virtual Private Cloud (VPC) architecture. In this architecture, one centralized ‘hub’ VPC is peered with multiple ‘spoke’ VPCs, with each of these spoke VPCs typically representing an environment for a user-facing service. Often, the centralized hub VPC contains services used by the organization for internal uses. For example: scanning software, a VPN server, an artifact repository, a CI/CD system, etc.

However, not all organizations make use of a centralized-hub-and-spoke architecture. For example, picture the following organization’s overall infrastructure:

• Four unpeered VPCs, each representing an environment for an AWS Elastic Kubernetes Service (EKS) cluster with four self-managed EKS worker nodes, which will run the organization’s user-facing services.
• CircleCI as the CI/CD system to build Docker images to run on the EKS clusters.
• AWS Elastic Container Registry as the Docker Registry for the aforementioned images.
• AWS Systems Manager (SSM) Agent running on the EKS worker nodes, for remote access directly from the AWS console (no need for a VPN server).
• GitHub as the VCS provider for the organization.

In this case, a centralized VPC would have nothing to host! However, this does not change the fact that the organization may need to undergo a SOC2 audit. In this case, a couple of their audit deliverables may be to create organizational procedures for periodic scans of their assets (the self-managed EKS worker nodes), and also a report of their assets run against a CIS level 2 benchmark and the resulting list of security objectives that need to be rectified.

In order to create these deliverables, the organization needs a scanning software, especially one featuring predefined controls, such as the CIS level 2 benchmark for Amazon Linux 2 – which corresponds to the base Linux distribution of the EKS worker nodes. Security Vendors such as Tenable, Rapid7, and Qualys supply products for this exact use case.

Thus, the organization described above is able to leverage the Compliance Agents provided by one of these three vendors in an architecture without a centralized VPC, and without VPC peering.

Here, rather than having a centralized, self-managed scanning software installation, each agent on the organization’s self-managed EKS worker nodes has an outbound connection to a Security Vendor’s managed control plane. The organization can perform scans, generate reports, and pursue outstanding security objectives as needed, without needing to change their mostly vendor-managed, hub-free architecture.

Using a Chef Effortless Package for the Compliance Agent

We can leverage a pattern of Chef Habitat Packages called Chef Effortless. This pattern builds Chef Infra Cookbooks and packages them inside the Chef Habitat package. The Chef Infra Cookbooks are pieces of Configuration Management code, similar to Ansible Playbooks or SaltStack States. However, the benefit of the Chef Effortless model is that we can leverage the Chef Infra Cookbooks without the need for a Chef Server: Using automated release orchestration tools, we can create Build Pipelines and Deployment Pipelines for a Chef Effortless package, link the two pipelines together, and thus every time we push to the codebase’s master branch, the tool will then rebuild the package and redeploy it.

Using a Configuration Management solution such as Chef Infra Cookbooks, we can perform a vendor-supported installation of the Compliance Agent:

rpm_package 'qualys-cloud-agent' do
source "#{node['qualys']['assets_dir']}/#{node['qualys']['pkg_name']}"
end

(link to code)

execute 'link-agent' do
command "/usr/local/qualys/cloud-agent/bin/qualys-cloud-agent.sh ActivationId=#{node['qualys']['activation_id']} CustomerId=#{node['qualys']['customer_id']}"
end

(link to code)

Deploying the Qualys Cloud Agent

Start by forking the following GitHub repository: https://github.com/ShuttleOps/soc2-habitat-best-practices. Use your preferred release orchestration tool to create a build and deploy pipeline.

Seeing the Compliance Agent in Action

Once you have deployed, the Chef Effortless package will have fully completed its Chef Infra run, and the Qualys Cloud Agent will have connected to the Qualys Cloud Platform and configured itself.

Now we can even begin generating reports, for example, we can generate the following CIS level 2 benchmark report for Ubuntu 18.x.

We hope that this article gave some more insight into Compliance Agents, why organizations use them, and how to leverage them in the context of Chef Habitat.

---

Are You Ready for a DevOps Transformation?

While software continues to eat the world at an ever-increasing pace with DevOps, the challenges and struggles of companies implementing DevOps is very real. We all can overcome these challenges by working together, improving our tools, processes, knowledge, and training our workforce.

Indellient is a Software Development Company that specializes in Data Analytics, Cloud Services, Managed IT Solutions, DevOps Services, and Document Process Automation.

---