chef-serverdensity

This cookbook provides an easy way to install the Server Density agent and manage server specific alerts.

Requirements

Cookbooks

This cookbook has dependencies on the following cookbooks:

• apt
• yum::epel

Platforms:

• Ubuntu
• Debian
• RHEL
• CentOS
• Fedora

Usage

Basic

1. Include recipe[serverdensity::alerts] in a run list

2. Then:

   • Override the node['serverdensity']['agent_key'] attribute on a higher level. recommended
   • or use the API to query for devices matching the node's hostname or create a new one if not found, by setting the config options.

Advanced (recommended)

1. Add serverdensity as a dependency to another cookbook

2. Use include_recipe 'serverdensity' to install sd-agent

3. Call the LWRP as described below to dynamically configure sd-agent

4. Call the serverdensity_plugin LWRP to configure plugins

5. Call the serverdensity_alert LWRP to configure custom alerts

Attributes

Basic Config

• node['serverdensity']['account'] required
  Your Server Density subdomain, i.e. companyname.serverdensity.io
• node['serverdensity']['agent_key']
  Your Server Density agent key (don't set this if you want to use the API to handle querying nodes/creating nodes)
• node['serverdensity']['enabled']
  Should sd-agent be running, default true

Optional API Config

If you don't set agent_key then set these parameters and new servers will be automatically created in your account.

If your account URL ends in .com you are using v1

• node['serverdensity']['username']
  Username for authenticating with the v1 API (if agent_key isn't set)
• node['serverdensity']['password']
  Password for authenticating with the v1 API (if agent_key isn't set)

If your account URL ends in .io you are using v2

• node['serverdensity']['token']
  Your API token from Preferences > Security in Server Density.

Optional Advanced Config

• node['serverdensity']['sd_url']
  
  By default this will be "https://#{node.serverdensity.account}", avoid setting manually where possible.

• node['serverdensity']['device_group']
  
  Sets the group for the device to be created in, to inherit alerts automatically.

• node['serverdensity']['apache_status_url']
  
  URL to get the Apache2 status page from (e.g. mod_status), disabled if not set

• node['serverdensity']['apache_status_user']
  
  Username to authenticate to the Apache2 status page, required if apache_status_url is set

• node['serverdensity']['apache_status_pass']
  
  Password to authenticate to the Apache2 status page, required if apache_status_url is set

• node['serverdensity']['mongodb_server']
  
  Server to get MongoDB status monitoring from, this takes a full MongoDB connection URI so you can set username/password etc. details here if needed, disabled if not set

• node['serverdensity']['mongodb_dbstats']
  
  Enables MongoDB stats if true and mongodb_server is set, default: false

• node['serverdensity']['mongodb_replset']
  
  Enables MongoDB replset stats if true and mongodb_server is set, default: false

• node['serverdensity']['mysql_server']
  
  Server to get MySQL status monitoring from, disabled if not set

• node['serverdensity']['mysql_user']
  
  Username to authenticate to MySQL, required if mysql_server is set

• node['serverdensity']['mysql_pass']
  
  Password to authenticate to MySQL, required if mysql_server is set

• node['serverdensity']['nginx_status_url']
  
  URL to get th Nginx status page from, disabled if not set

• node['serverdensity']['rabbitmq_status_url']
  
  URL to get the RabbitMQ status from via HTTP management API, disabled if not set

• node['serverdensity']['rabbitmq_user']
  
  Username to authenticate to the RabbitMQ management API, required if rabbitmq_status_url is set

• node['serverdensity']['rabbitmq_pass']
  
  Password to authenticate to the RabbitMQ management API, required if rabbitmq_status_url is set

• node['serverdensity']['tmp_directory']
  
  Override where the agent stores temporary files, system default tmp will be used if not set

• node['serverdensity']['pidfile_directory']
  
  Override where the agent stores it's PID file, temp dir (above or system default) is used if not set

• node['serverdensity']['logging_level']
  
  Set the logging level for the agent

• node['serverdensity']['alerts']
  
  This should be an array of hashes, each of which defines an alert, keys should match those exposed by the alert LWRP and will be created by the alerts recipe.

• node['serverdensity']['plugin_dir']
  
  Sets the directory the agent looks for plugins, if left blank it is ignored

• node['serverdensity']['plugin_options'] deprecated
  
  Use the plugin LWRP where possible, this exists for legacy reasons only.
  
  A hash of optional named plugin options if you have agent plugins you want to configure, simple key-values will be added to the [Main] section of the config while sub-hashes will be generated into sections e.g. {'Beanstalk'=>{'host'=>'localhost'}} becomes:

[Beanstalk]
host = localhost

Recipes

default

Installs the sd-agent, ready for the LWRP based setup.

alerts

Configures sd-agent from attributes rather than LWRPs, and creates attribute based alerts defined in the ['serverdensity']['alerts'] hash. Here is an example of an alert (for API v1):

node['serverdensity']['alerts']['high-load'] = { 
  'metadata' => {
    :userId => ['group'],
    :notificationType => ['email'],
    :checkType => 'loadAvrg',
    :comparison => :>,
    :triggerThreshold => 3,
    :notificationFixed => true,
    :notificationDelay => 5,
    :notificationFrequencyOnce => true
  }
}

LWRP

serverdensity

Actions

• clear
  remove all alerts from device
• configure
  write agent config, get token (see below)
• disable
  stop agent if running
• enable
  start agent if not running
• setup
  initialize API for future calls
• sync
  synchronize device metadata
• update (default)
  setup api, either configure and enable or disable agent, sync metadata if API is available

Getting Device Token

The configure action of this LWRP facilitates the dynamic configuration the sd-agent. The agent_key for the device can be acquired by various methods, in order attempts are made to:

1. use the agent_key passed into LWRP
2. use agent_key defined in attributes
3. read the agent_key from /etc/sd-agent-key on the server
4. extract agent_key from EC2's internal metadata API
5. find the device in Server Density and request the agent_key
6. create the device in Server Density and request the agent_key

Which of these steps take place depends on the various parameters passed in (see below), and when the agent_key is found. As soon as it is acquired no further steps are run.

Default

The default recipe will use steps 2-4 to find an agent_key

Manual

# step 1 only
serverdensity node.name do
  agent_key '00000000000000000000000000000000'
end

API v1

# steps 2-6
serverdensity node.name do
  username 'foo'
  password 'bar'
end

API v2

# steps 2-6
serverdensity node.name do
  token '00000000000000000000000000000000'
end

Other settings

Device

By default, step 5 will use the hostname of the device to match against those stored in Server Density, however occasionally it makes more sense to match on something else, for example when using EC2:

# v2 only (v1 only supports name and hostname keys)
serverdensity node.name do
  token '00000000000000000000000000000000'
  device :providerId => node.ec2.instance_id
end

Metadata

The LWRP also supports writing metadata (via the sync action) to devices during creation via the API. Updating metadata is also supported by API v2.

serverdensity node.name do
  token '00000000000000000000000000000000'
  metadata :group => 'chef-lwrp'
end

Settings

Settings is a hash used to override configuration options set in the attributes before the agent config file is written. Take a look at this template to see which settings can be defined.

serverdensity_alert

This is used to create alerts for your newly minted device, it currently just acts as a wrapper for API calls and as such, v1 and v2 usage is significantly different, the hope is to give them a shared DSL in the future.

This LWRP requires that API credentials (v1 or v2) have been provided, if they have not, it will throw an error.

API v1

# create v1 alert (https://github.com/serverdensity/sd-api-docs/blob/master/sections/alerts.md#add)
serverdensity_alert 'high-cpu' do
  metadata(
    :userId => ['group'],
    :notificationType => ['email'],
    :checkType => 'loadAvrg',
    :comparison => :>,
    :triggerThreshold => 3,
    :notificationFixed => true,
    :notificationDelay => 5,
    :notificationFrequencyOnce => true
  )
end

API v2

# create v2 alert
serverdensity_alert 'high-cpu' do
  metadata(
    # params as described here https://apidocs.serverdensity.com/Alerts/Alert_Configs/Creating
  )
end

serverdensity_plugin

This is used to install and configure plugins, it creates a new plugin config file inside /etc/sd-agent/conf.d and symlinks the file specified in path to the plugin_dir. Currently plugins must be placed on the server outside of this LWRP and then referenced (with path). A future goal is to allow plugin installation directly via
plugins.serverdensity.com.

API (both versions)

# install Supervisord plugin, configure it, and restart the sd-agent
serverdensity_plugin 'Supervisord' do
  path '/opt/my-company/sd-plugins/Supervisord.py'
  config(
    rpc_url: 'unix:///var/run/supervisor.sock'
  )
  notifies :restart, 'service[sd-agent]'
end

The config file will namespace configurations so that they do not clash between plugins. The example above would produce the following config file:

[Supervisord]
rpc_url = unix:///var/run/supervisor.sock

IMPORTANT: Some older plugins may require global settings, if this is the case it is possible to use the plugin_options attribute to set these, however it is recommended to upgrade the plugin where possible.

Notes

As this cookbook depends on a few other cookbooks it's recommended you use a library like Berkshelf, librarian-chef or knife-github-cookbooks to automatically manage and install them.

Publishing on the Chef Supermarket

https://supermarket.getchef.com/cookbooks/serverdensity

1. Clone the repo to ~/Desktop/serverdensity
2. From CWD ~/Desktop run knife cookbook site share serverdensity "Monitoring & Trending" --cookbook-path ~/Desktop

References

• Server Density home page
• akatz/chef-serverdensity
• Jonty/chef-serverdensity
• serverdensity/chef-serverdensity

Authors

• Original Author: Avrohom Katz iambpentameter@gmail.com
• Modified by: Jonty Wareing jonty@jonty.co.uk
• Modified by: Server Density hello@serverdensity.com
• Rewritten by: Mal Graty mal.graty@googlemail.com

License

MIT