Why use Chef for automation and orchestration

Chef has been a leading open source tool for automating the provisioning and configuration of servers for the better part of a decade. In recent years the company added InSpec and Habitat to the portfolio, open source projects that automate policy compliance testing and the deployment and configuration of applications, respectively. The company’s flagship commercial offering, Chef Automate, brings all of these pieces together.  

Chef Automate provides a suite of enterprise capabilities for workflow, node visibility, and compliance, and integrates with the open source products Chef, InSpec, and Habitat. Chef Automate comes with support services for the entire platform, including the open source components. In addition to providing views into operational, compliance, and workflow events, it includes a pipeline for continuous delivery of infrastructure and applications.

Chef components and workflow

The Chef DK (development kit) workstation is where users interact with Chef. On the workstation users author and test cookbooks using tools such as Test Kitchen (to generate test VMs) and interact with the Chef server using the command line tools. For instance, Knife is a command-line tool that provides an interface between a local Chef repo and the Chef server. Knife helps users to manage nodes, cookbooks, data bags, and the installation (bootstrap) of the Chef client onto nodes, among other tasks. Most files in a Chef cookbook are written in Ruby, although some configurations are written in YAML.

Chef

The open source Chef server acts as a hub for configuration data. The Chef server stores cookbooks, the policies that are applied to nodes, and metadata that describes each registered node that is being managed by Chef. Nodes use the Chef client to ask the Chef server for configuration details, such as recipes, templates, and file distributions. In other words, Chef is by default a pull-based system; it also has push capabilities.

Chef Supermarket is the location in which community cookbooks are shared and managed. The Chef management console, chef-client (agent) run reporting, high availability configurations, and Chef server replication are available as part of Chef Automate.

InSpec is a free and open-source framework for testing and auditing your applications and infrastructure. It is the foundation of the Compliance portion of Chef Automate. It integrates with Puppet and Ansible as well as Chef.

Habitat is an open source, cloud native application automation and application lifecycle management platform, designed from the application’s point of view rather than the enterprise’s or the platform’s point of view.

Chef

Chef for devops, compliance, and cloud

Chef Automate helps to provision and deploy apps faster, more frequently and more reliably—in other words, it supports devops. It also automates compliance by reducing server drift, identifying compliance violations, and automatically remediating any issues. Chef Compliance, based on the open source InSpec, used to be a separate product, but is now part of Chef Automate.

Cloud migration is one of the interesting use cases for Chef. That includes AWS, Microsoft Azure, Google Cloud Platform, mixed deployments, and hybrid clouds. Another major set of use cases is ensuring compliance to PCI, HIPAA, and other security and privacy regulations.

As shown in the figure below, one of Chef’s selling points is that it works with what you have. That includes the major Git-based repositories, CI/CD systems, operating systems, clouds, and container orchestration systems.

Chef

Chef installation and setup

In general, a Chef Automate installation consists of a minimum of two servers: a Chef server (at least four vCPUs and 8 GB RAM), which contains the cookbooks and data used to build, test, and deploy your components within Chef Automate and your infrastructure, and a Chef Automate server (at least four vCPUs and 16 GB RAM), which coordinates the process of moving a change through the workflow pipeline as well as providing insights and visualizations about your Chef Automate cluster.

There are two optional servers, a push jobs server, which is used to create infrastructure nodes for deployment testing and is also needed if you use push jobs-based build nodes as part of your testing and deployment process, and runners or build nodes (at least two vCPUs and 4 GB RAM), which perform the work of running builds, tests, and deployments out of Chef Automate, and are only required when using the workflow capabilities of Chef Automate.

You start by installing the Chef server, either stand-alone or in a high availability configuration. Then create a user and organization for use with Chef Automate, using chef-server-ctl commands. Optionally download the code for and create a push jobs server, and then reconfigure the Chef server, again using chef-server-ctl commands.

At this point you can install and configure Chef Automate using rpm or dpkg. Install your license, and use automate-ctl commands to run a pre-flight check and the setup process. The setup will prompt you to create a runner for workflow. Finally, you can configure your nodes for data collection.

AWS OpsWorks for Chef Automate simplifies the installation process greatly, assuming that you want to have your Automate and Chef servers on AWS—you can deploy in 10 minutes or less. You can still manage your on-premises nodes from OpsWorks, although OpsWorks shines when most of your nodes are on AWS, as it can automatically enroll nodes into auto-scaling groups.

There is a decent tutorial on AWS to teach you about Chef, Chef Automate, and OpsWorks, in which you’ll set everything up and perform automation tasks step by step. The tutorial takes a little longer than a basic deployment, but is worth doing if you’re new to Chef.

You can also install Chef Automate into VMs from the AWS Marketplace. In addition, Chef has integrations with Google Cloud Platform, the Microsoft Azure Marketplace, and VMware.

Strong on devops and compliance, with broad platform support and a large collection of modules, Chef Automate provides a full suite of enterprise capabilities to automate the delivery and ongoing operation of hybrid infrastructure. It is likely to fill most or all of your needs for IT automation.

—

Cost: Open source projects (Chef, InSpec, Habitat, etc.), free. Chef Automate, $137/node/year with standard (12x5) support. AWS OpsWorks with Chef Automate, $0.0155/node/hour. Hosted Chef, $72/node/year.

Platform: Chef Automate Server requires a RHEL, SUSE, or Ubuntu OS. In addition to these, Chef Automate Job Runner is supported on MacOS.

Chef Automate is compatible with the VMware, CoreOS, Docker, Windows, and Linux operating systems; the Google, AWS, Azure, OpenStack, and VMware clouds; the Kubernetes, Docker Swarm, and Mesosphere container orchestration systems. A cloud-based Chef Automate service is available as AWS OpsWorks for Chef Automate. You should browse to the Automate console with Google Chrome; IE is specifically not supported.

Chef DK (development kit) is commercially supported on MacOS 10.11, RHEL 6, SUSE 11, Ubuntu LTS, Windows 10 or Windows Server 2012, and later versions of those operating systems. Community support is available for Debian 7 and Scientific Linux 6 and later.