About Handlers

[edit on GitHub]

Use a handler to identify situations that arise during a Chef Infra Client run, and then tell Chef Infra Client how to handle these situations when they occur.

There are three types of handlers:

Exception/Report Handlers

Exception and report handlers are used to trigger certain behaviors in response to specific situations, typically identified during a Chef Infra Client run.

• An exception handler is used to trigger behaviors when a defined aspect of a Chef Infra Client run fails.
• A report handler is used to trigger behaviors when a defined aspect of a Chef Infra Client run is successful.

Both types of handlers can be used to gather data about a Chef Infra Client run and can provide rich levels of data about all types of usage, which can be used later for trending and analysis across the entire organization.

Exception and report handlers are made available to a Chef Infra Client run in one of the following ways:

• By adding the chef_handler resource to a recipe, and then adding that recipe to the run-list for a node. (The chef_handler resource is available from the chef_handler cookbook.)
• By adding the handler to one of the following settings in the node’s client.rb file: exception_handlers and/or report_handlers

Run from Recipes

The chef_handler resource allows exception and report handlers to be enabled from within recipes, which can then added to the run-list for any node on which the exception or report handler should run. The chef_handler resource is available from the chef_handler cookbook.

To use the chef_handler resource in a recipe, add code similar to the following:

chef_handler 'name_of_handler' do
  source '/path/to/handler/handler_name'
  action :enable
end

For example, a handler for Growl needs to be enabled at the beginning of a Chef Infra Client run:

chef_gem 'chef-handler-growl'

and then is activated in a recipe by using the chef_handler resource:

chef_handler 'Chef::Handler::Growl' do
  source 'chef/handler/growl'
  action :enable
end

Run from client.rb

A simple exception or report handler may be installed and configured at run-time. This requires editing of a node’s client.rb file to add the appropriate setting and information about that handler to the client.rb or solo.rb files. Depending on the handler type, one (or more) of the following settings must be added:

When this approach is used, the client.rb file must also tell Chef Infra Client how to install and run the handler. There is no default install location for handlers. The simplest way to distribute and install them is via RubyGems, though other methods such as GitHub or HTTP will also work. Once the handler is installed on the system, enable it in the client.rb file by requiring it. After the handler is installed, it may require additional configuration. This will vary from handler to handler. If a handler is a very simple handler, it may only require the creation of a new instance. For example, if a handler named MyOrg::EmailMe is hardcoded for all of the values required to send email, a new instance is required. And then the custom handler must be associated with each of the handler types for which it will run.

For example:

require '/var/chef/handlers/email_me'         # the installation path

email_handler = MyOrg::EmailMe.new            # a simple handler

start_handlers << email_handler               # run at the start of the run
report_handlers << email_handler              # run at the end of a successful run
exception_handlers << email_handler           # run at the end of a failed run

Start Handlers

A start handler is not loaded into a Chef Infra Client run from a recipe, but is instead listed in the client.rb file using the start_handlers attribute. The start handler must be installed on the node and be available to Chef Infra Client prior to the start of a Chef Infra Client run. Use the chef-client cookbook to install the start handler.

Start handlers are made available to a Chef Infra Client run in one of the following ways:

• By adding a start handler to the chef-client cookbook, which installs the handler on the node so that it is available to Chef Infra Client at the start of a Chef Infra Client run
• By adding the handler to one of the following settings in the node’s client.rb file: start_handlers

Run from Recipes

The chef-client cookbook can be configured to automatically install and configure gems that are required by a start handler. For example:

node.override['chef_client']['load_gems']['chef-reporting'] = {
  :require_name => 'chef_reporting',
  :action => :install
}

node.override['chef_client']['config']['start_handlers'] = [
  {
    :class => 'Chef::Reporting::StartHandler',
    :arguments => []
  }
]

include_recipe 'chef-client::config'

Run from client.rb

A start handler can be configured in the client.rb file by adding the following setting:

For example, the Reporting start handler adds the following code to the top of the client.rb file:

begin
  require 'chef_reporting'
  start_handlers << Chef::Reporting::StartHandler.new()
rescue LoadError
  Chef::Log.warn 'Failed to load #{lib}. This should be resolved after a chef run.'
end

This ensures that when a Chef Infra Client run begins the chef_reporting event handler is enabled. The chef_reporting event handler is part of a gem named chef-reporting. The chef_gem resource is used to install this gem:

chef_gem 'chef-reporting' do
  action :install
end

Event Handlers

Use the Handler DSL to attach a callback to an event. If the event occurs during a Chef Infra Client run, the associated callback is executed. For example:

• Sending email if a Chef Infra Client run fails
• Aggregating statistics about resources updated during a Chef Infra Client runs to StatsD

on Method

Use the on method to associate an event type with a callback. The callback defines what steps are taken if the event occurs during a Chef Infra Client run and is defined using arbitrary Ruby code. The syntax is as follows:

Chef.event_handler do
  on :event_type do
    # some Ruby
  end
end

where

• Chef.event_handler declares a block of code within a recipe that is processed when the named event occurs during a Chef Infra Client run
• on defines the block of code that will tell Chef Infra Client how to handle the event
• :event_type is a valid exception event type, such as :run_start, :run_failed, :converge_failed, :resource_failed, or :recipe_not_found

For example:

Chef.event_handler do
  on :converge_start do
    puts "Ohai! I have started a converge."
  end
end

Event Types

The following table describes the events that may occur during a Chef Infra Client run. Each of these events may be referenced in an on method block by declaring it as the event type.

Examples

The following examples show ways to use the Handler DSL.

Send Email

Use the on method to create an event handler that sends email when a Chef Infra Client run fails. This will require:

• A way to tell Chef Infra Client how to send email
• An event handler that describes what to do when the :run_failed event is triggered
• A way to trigger the exception and test the behavior of the event handler

Define How Email is Sent

Use a library to define the code that sends email when a Chef Infra Client run fails. Name the file helper.rb and add it to a cookbook’s /libraries directory:

require 'net/smtp'

module HandlerSendEmail
  class Helper

    def send_email_on_run_failure(node_name)

      message = "From: Chef <chef@chef.io>\n"
      message << "To: Grant <grantmc@chef.io>\n"
      message << "Subject: Chef run failed\n"
      message << "Date: #{Time.now.rfc2822}\n\n"
      message << "Chef run failed on #{node_name}\n"
      Net::SMTP.start('localhost', 25) do |smtp|
        smtp.send_message message, 'chef@chef.io', 'grantmc@chef.io'
      end
    end
  end
end

Add the Handler

Invoke the library helper in a recipe:

Chef.event_handler do
  on :run_failed do
    HandlerSendEmail::Helper.new.send_email_on_run_failure(
      Chef.run_context.node.name
    )
  end
end

• Use Chef.event_handler to define the event handler
• Use the on method to specify the event type

Within the on block, tell Chef Infra Client how to handle the event when it’s triggered.

Test the Handler

Use the following code block to trigger the exception and have the Chef Infra Client send email to the specified email address:

ruby_block 'fail the run' do
  block do
    fail 'deliberately fail the run'
  end
end

etcd Locks

The following example shows how to prevent concurrent Chef Infra Client runs from both holding a lock on etcd:

lock_key = "#{node.chef_environment}/#{node.name}"

Chef.event_handler do
  on :converge_start do |run_context|
    Etcd.lock_acquire(lock_key)
  end
end

Chef.event_handler do
  on :converge_complete do
    Etcd.lock_release(lock_key)
  end
end

HipChat Notifications

Event messages can be sent to a team communication tool like HipChat. For example, if a Chef Infra Client run fails:

Chef.event_handler do
  on :run_failed do |exception|
    hipchat_notify exception.message
  end
end

or send an alert on a configuration change:

Chef.event_handler do
  on :resource_updated do |resource, action|
    if resource.to_s == 'template[/etc/nginx/nginx.conf]'
      Helper.hipchat_message("#{resource} was updated by chef")
    end
  end
end

Handlers and Cookbooks

The following cookbooks can be used to load handlers during a Chef Infra Client run.

chef_handler

Exception and report handlers can be distributed using the chef_handler cookbook. This cookbook is authored and maintained by Chef and exposes a custom resource that can be used to enable custom handlers from within recipes and to include product-specific handlers from cookbooks. The chef_handler cookbook can be accessed here: https://github.com/chef-cookbooks/chef_handler. See the README.md for additional information.

Chef Infra Client

Start handlers can be distributed using the chef-client cookbook, which will install the handler on the target node during the initial configuration of the node. This ensures that the start handler is always present on the node so that it is available to Chef Infra Client at the start of every run.

Custom Handlers

A custom handler can be created to support any situation. The easiest way to build a custom handler:

1. Download the chef_handler cookbook
2. Create a custom handler
3. Write a recipe using the chef_handler resource
4. Add that recipe to a node’s run-list, often as the first recipe in that run-list

Syntax

The syntax for a handler can vary, depending on what the the situations the handler is being asked to track, the type of handler being used, and so on. All custom exception and report handlers are defined using Ruby and must be a subclass of the Chef::Handler class.

require 'chef/log'

module ModuleName
  class HandlerName < Chef::Handler
    def report
      # Ruby code goes here
    end
  end
end

where:

• require ensures that the logging functionality of Chef Infra Client is available to the handler
• ModuleName is the name of the module as it exists within the Chef library
• HandlerName is the name of the handler as it is used in a recipe
• report is an interface that is used to define the custom handler

For example, the following shows a custom handler that sends an email that contains the exception data when a Chef Infra Client run fails:

require 'net/smtp'

module OrgName
  class SendEmail < Chef::Handler
    def report
      if run_status.failed? then
        message  = "From: sender_name <sender@example.com>\n"
        message << "To: recipient_address <recipient@example.com>\n"
        message << "Subject: chef-client Run Failed\n"
        message << "Date: #{Time.now.rfc2822}\n\n"
        message << "Chef run failed on #{node.name}\n"
        message << "#{run_status.formatted_exception}\n"
        message << Array(backtrace).join('\n')
        Net::SMTP.start('your.smtp.server', 25) do |smtp|
          smtp.send_message message, 'sender@example', 'recipient@example'
        end
      end
    end
  end
end

and then is used in a recipe like:

send_email 'blah' do
  # recipe code
end

report Interface

The report interface is used to define how a handler will behave and is a required part of any custom handler. The syntax for the report interface is as follows:

def report
  # Ruby code
end

The Ruby code used to define a custom handler will vary significantly from handler to handler. Chef Infra Client includes two default handlers: error_report and json_file. Their use of the report interface is shown below.

The error_report handler:

require 'chef/handler'
require 'chef/resource/directory'

class Chef
  class Handler
    class ErrorReport < ::Chef::Handler
      def report
        Chef::FileCache.store('failed-run-data.json', Chef::JSONCompat.to_json_pretty(data), 0640)
        Chef::Log.fatal("Saving node information to #{Chef::FileCache.load('failed-run-data.json', false)}")
      end
    end
 end
end

The json_file handler:

require 'chef/handler'
require 'chef/resource/directory'

class Chef
  class Handler
    class JsonFile < ::Chef::Handler
      attr_reader :config
      def initialize(config={})
        @config = config
        @config[:path] ||= '/var/chef/reports'
        @config
      end
      def report
        if exception
          Chef::Log.error('Creating JSON exception report')
        else
          Chef::Log.info('Creating JSON run report')
        end
        build_report_dir
        savetime = Time.now.strftime('%Y%m%d%H%M%S')
        File.open(File.join(config[:path], 'chef-run-report-#{savetime}.json'), 'w') do |file|
          run_data = data
          run_data[:start_time] = run_data[:start_time].to_s
          run_data[:end_time] = run_data[:end_time].to_s
          file.puts Chef::JSONCompat.to_json_pretty(run_data)
        end
      end
      def build_report_dir
        unless File.exist?(config[:path])
          FileUtils.mkdir_p(config[:path])
          File.chmod(00700, config[:path])
        end
      end
    end
  end
end

Optional Interfaces

The following interfaces may be used in a handler in the same way as the report interface to override the default handler behavior in Chef Infra Client. That said, the following interfaces are not typically used in a handler and, for the most part, are completely unnecessary for a handler to work properly and/or as desired.

data

The data method is used to return the Hash representation of the run_status object. For example:

def data
  @run_status.to_hash
end

run_report_safely

The run_report_safely method is used to run the report handler, rescuing and logging errors that may arise as the handler runs and ensuring that all handlers get a chance to run during a Chef Infra Client run (even if some handlers fail during that run). In general, this method should never be used as an interface in a custom handler unless this default behavior simply must be overridden.

def run_report_safely(run_status)
  run_report_unsafe(run_status)
rescue Exception => e
  Chef::Log.error('Report handler #{self.class.name} raised #{e.inspect}')
  Array(e.backtrace).each { |line| Chef::Log.error(line) }
ensure
  @run_status = nil
end

run_report_unsafe

The run_report_unsafe method is used to run the report handler without any error handling. This method should never be used directly in any handler, except during testing of that handler. For example:

def run_report_unsafe(run_status)
  @run_status = run_status
  report
end

run_status Object

The run_status object is initialized by Chef Infra Client before the report interface is run for any handler. The run_status object keeps track of the status of a Chef Infra Client run and will contain some (or all) of the following properties:

Examples

The following sections show examples of handlers.

Cookbook Versions

Community member juliandunn created a custom report handler that logs all of the cookbooks and cookbook versions that were used during a Chef Infra Client run, and then reports after the run is complete. This handler requires the chef_handler resource (which is available from the chef_handler cookbook).

cookbook_versions.rb

The following custom handler defines how cookbooks and cookbook versions that are used during a Chef Infra Client run will be compiled into a report using the Chef::Log class in Chef Infra Client:

require 'chef/log'

module Opscode
  class CookbookVersionsHandler < Chef::Handler

    def report
      cookbooks = run_context.cookbook_collection
      Chef::Log.info('Cookbooks and versions run: #{cookbooks.keys.map {|x| cookbooks[x].name.to_s + ' ' + cookbooks[x].version} }')
    end
  end
end

default.rb

The following recipe is added to the run-list for every node on which a list of cookbooks and versions will be generated as report output after every Chef Infra Client run.

include_recipe 'chef_handler'

cookbook_file "#{node['chef_handler']['handler_path']}/cookbook_versions.rb" do
  source 'cookbook_versions.rb'
  owner 'root'
  group 'root'
  mode '0755'
  action :create
end

chef_handler 'Opscode::CookbookVersionsHandler' do
  source "#{node['chef_handler']['handler_path']}/cookbook_versions.rb"
  supports :report => true
  action :enable
end

This recipe will generate report output similar to the following:

[2013-11-26T03:11:06+00:00] INFO: Chef Run complete in 0.300029878 seconds
[2013-11-26T03:11:06+00:00] INFO: Running report handlers
[2013-11-26T03:11:06+00:00] INFO: Cookbooks and versions run: ["chef_handler 1.1.4", "cookbook_versions_handler 1.0.0"]
[2013-11-26T03:11:06+00:00] INFO: Report handlers complete

Reporting

Start handler functionality was added when Chef started building add-ons for the Chef Infra Server. The Reporting add-on is designed to create reporting data based on a Chef Infra Client run. And since Reporting needs to be able to collect data for the entire Chef Infra Client run, Reporting needs to be enabled before anything else happens at the start of a Chef Infra Client run.

start_handler.rb

The following code shows the start handler used by the Reporting add-in for the Chef Infra Server:

require 'chef/handler'
require 'chef/rest'
require 'chef/version_constraint'

class Chef
  class Reporting
    class StartHandler < ::Chef::Handler

      attr_reader :config

      def initialize(config={})
        @config = config
      end

      def report
        version_checker = Chef::VersionConstraint.new('< 11.6.0')
        if version_checker.include?(Chef::VERSION)
          Chef::Log.info('Enabling backported resource reporting Handler')
          rest = Chef::REST.new(Chef::Config[:chef_server_url], @run_status.node.name, Chef::Config[:client_key])
          resource_reporter = Chef::Reporting::ResourceReporter.new(rest)
          @run_status.events.register(resource_reporter)

          resource_reporter.run_started(@run_status)
        else
         Chef::Log.debug('Chef Version already has new Resource Reporter - skipping startup of backport version')
        end
      end
    end
  end
end

json_file Handler

The json_file handler is available from the chef_handler cookbook and can be used with exceptions and reports. It serializes run status data to a JSON file. This handler may be enabled in one of the following ways.

By adding the following lines of Ruby code to either the client.rb file or the solo.rb file, depending on how Chef Infra Client is being run:

require 'chef/handler/json_file'
report_handlers << Chef::Handler::JsonFile.new(:path => '/var/chef/reports')
exception_handlers << Chef::Handler::JsonFile.new(:path => '/var/chef/reports')

By using the chef_handler resource in a recipe, similar to the following:

chef_handler 'Chef::Handler::JsonFile' do
  source 'chef/handler/json_file'
  arguments :path => '/var/chef/reports'
  action :enable
end

After it has run, the run status data can be loaded and inspected via Interactive Ruby (IRb):

irb(main):002:0> require 'json' => true
irb(main):003:0> require 'chef' => true
irb(main):004:0> r = JSON.parse(IO.read('/var/chef/reports/chef-run-report-20110322060731.json')) => ... output truncated
irb(main):005:0> r.keys => ['end_time', 'node', 'updated_resources', 'exception', 'all_resources', 'success', 'elapsed_time', 'start_time', 'backtrace']
irb(main):006:0> r['elapsed_time'] => 0.00246

error_report Handler

The error_report handler is built into Chef Infra Client and can be used for both exceptions and reports. It serializes error report data to a JSON file. This handler may be enabled in one of the following ways.

By adding the following lines of Ruby code to either the client.rb file or the solo.rb file, depending on how Chef Infra Client is being run:

require 'chef/handler/error_report'
report_handlers << Chef::Handler::ErrorReport.new()
exception_handlers << Chef::Handler::ErrorReport.new()

By using the chef_handler resource in a recipe, similar to the following:

chef_handler 'Chef::Handler::ErrorReport' do
  source 'chef/handler/error_report'
  action :enable
end

Community Handlers

The following open source handlers are available from the Chef community: