cookbook 'ceph-chef', '= 0.9.20'
ceph-chef
(79) Versions
0.9.20
-
-
1.1.28
-
1.1.27
-
1.1.26
-
1.1.25
-
1.1.24
-
1.1.22
-
1.1.21
-
1.1.20
-
1.1.19
-
1.1.18
-
1.1.17
-
1.1.16
-
1.1.15
-
1.1.14
-
1.1.13
-
1.1.12
-
1.1.11
-
1.1.10
-
1.1.9
-
1.1.8
-
1.1.7
-
1.1.6
-
1.1.5
-
1.1.4
-
1.1.3
-
1.1.2
-
1.1.1
-
1.1.0
-
1.0.23
-
1.0.22
-
1.0.21
-
1.0.20
-
1.0.19
-
1.0.18
-
1.0.17
-
1.0.16
-
1.0.14
-
1.0.13
-
1.0.12
-
1.0.11
-
1.0.10
-
1.0.9
-
1.0.8
-
1.0.7
-
1.0.6
-
1.0.5
-
1.0.4
-
1.0.3
-
1.0.2
-
1.0.1
-
0.9.29
-
0.9.28
-
0.9.27
-
0.9.26
-
0.9.25
-
0.9.24
-
0.9.23
-
0.9.22
-
0.9.21
-
0.9.20
-
0.9.19
-
0.9.18
-
0.9.17
-
0.9.16
-
0.9.15
-
0.9.14
-
0.9.13
-
0.9.12
-
0.9.11
-
0.9.10
-
0.9.9
-
0.9.8
-
0.9.7
-
0.9.6
-
0.9.5
-
0.9.4
-
0.9.3
-
0.9.1
-
0.9.0
Follow6
- 1.1.28
- 1.1.27
- 1.1.26
- 1.1.25
- 1.1.24
- 1.1.22
- 1.1.21
- 1.1.20
- 1.1.19
- 1.1.18
- 1.1.17
- 1.1.16
- 1.1.15
- 1.1.14
- 1.1.13
- 1.1.12
- 1.1.11
- 1.1.10
- 1.1.9
- 1.1.8
- 1.1.7
- 1.1.6
- 1.1.5
- 1.1.4
- 1.1.3
- 1.1.2
- 1.1.1
- 1.1.0
- 1.0.23
- 1.0.22
- 1.0.21
- 1.0.20
- 1.0.19
- 1.0.18
- 1.0.17
- 1.0.16
- 1.0.14
- 1.0.13
- 1.0.12
- 1.0.11
- 1.0.10
- 1.0.9
- 1.0.8
- 1.0.7
- 1.0.6
- 1.0.5
- 1.0.4
- 1.0.3
- 1.0.2
- 1.0.1
- 0.9.29
- 0.9.28
- 0.9.27
- 0.9.26
- 0.9.25
- 0.9.24
- 0.9.23
- 0.9.22
- 0.9.21
- 0.9.20
- 0.9.19
- 0.9.18
- 0.9.17
- 0.9.16
- 0.9.15
- 0.9.14
- 0.9.13
- 0.9.12
- 0.9.11
- 0.9.10
- 0.9.9
- 0.9.8
- 0.9.7
- 0.9.6
- 0.9.5
- 0.9.4
- 0.9.3
- 0.9.1
- 0.9.0
Installs/Configures Ceph (Jewel and above)
cookbook 'ceph-chef', '= 0.9.20', :supermarket
knife supermarket install ceph-chef
knife supermarket download ceph-chef
Ceph-Chef Cookbook
DESCRIPTION
Installs and configures Ceph, a distributed network storage and filesystem designed to provide excellent performance, reliability, and scalability. Supports Hammer and higher releases (nothing below Hammer is supported in this repo).
The current version is focused on installing and configuring Ceph for Ubuntu, CentOS and RHEL.
For documentation on how to use this cookbook, refer to the USAGE section.
Recommendation
Also, you can look at https://github.com/bloomberg/chef-bcs. That Chef App (repo) uses this repo for Bloomberg's large clusters. The chef-bcs repo is an S3 Object Store Cluster used in multiple data centers.
Note: The documentation is a WIP along with a few other features. This repo is actively managed.
For help, use Gitter chat, mailing-list or issues here in this repo.
NOTE: Users of ceph-cookbook
The original ceph-cookbook will remain and may continue to be updated (see that repo for specifics). We tried to use some of the interesting features of ceph-cookbook but we added a lot of enhancements and simplifications. Simply replacing ceph-cookbook with ceph-chef will not work without a few modifications. Also, ceph-chef only works with Chef 12.5+ and Hammer and higher. Nothing below the Hammer release of Ceph is supported in this repo. In addition, only civitweb is used going forward (not Apache).
NOTE: The current LWRP are using the style prior to Chef version 12.5. There will a new release shortly that will support the now recommended way of handling custom resources. To make that change easier we will be using a helper cookbook called Poise. Using Poise makes creating custom resources and common services very simple.
Chef
>= 12.5+
Platform
Tested as working:
- Ubuntu Trusty (14.04) [Still verifying updates work]
- CentOS (7.1)
- RHEL (7.1)
Cookbooks
The ceph cookbook requires the following cookbooks from Chef:
USAGE
Ceph cluster design and Ceph support are beyond the scope of this README, please turn to the:
public wiki, mailing lists, visit our IRC channel, or contact Red Hat:
http://ceph.com/docs/master
http://ceph.com/resources/mailing-list-irc/
NOTE: The recommendation above on the Bloomberg Cluster repo will give you a solid design and/or provide ideas and guidance on laying out your Ceph cluster at scale.
This cookbook can be used to implement a chosen cluster design. Most of the configuration is retrieved from node attributes, which can be set by an environment json file or by a wrapper cookbook that updates the attributes directly. A basic cluster configuration will need most of the following attributes:
-
node['ceph']['config']['fsid']
- the cluster UUID -
node['ceph']['config]'['global']['public network']
- a CIDR specification of the public network -
node['ceph']['config]'['global']['cluster network']
- a CIDR specification of a separate cluster replication network -
node['ceph']['config]'['global']['rgw dns name']
- the main domain of the radosgw daemon
Most notably, the configuration does NOT need to set the mon initial members
, because the cookbook does a node search based on TAGS to find other mons in the same environment. However, you can add them to node['ceph']['config']['global']['mon initial members'] = <whatever mon ip list you want>
The other set of attributes that this recipe needs is node['ceph']['osd']['devices']
, which is an array of OSD definitions, similar to the following:
- {'device' => '/dev/sdb'} - Use a full disk for the OSD, with a small partition for the journal
- {'type' => 'directory', 'device' => '/src/node/sdb1/ceph'} - Use a directory, and have a small file for the journal
- {'device' => '/dev/sde', 'dmcrypt' => true} - Store the data encrypted by passing --dmcrypt to
ceph-disk-prepare
- {'device' => '/dev/sdc', 'journal' => '/dev/sdd2'} - use a full disk for the OSD with a custom partition for the journal on another device such as an SSD or NMVe
Using a Policy Wrapper Cookbook
To automate setting several of these node attributes, it is recommended to use a policy wrapper cookbook. This allows the ability to use Chef Server cookbook versions along with environment version restrictions to roll out configuration changes in an ordered fashion.
It also can help with automating some settings. For example, a wrapper cookbook could peek at the list of hard drives that ohai
has found and populate node['ceph']['osd_devices'] accordingly, instead of manually typing them all in:
node.override['ceph']['osd_devices'] = node['block_device'].each.reject{ |name, data| name !~ /^sd[b-z]/}.sort.map { |name, data| {'journal' => "/dev/#{name}"} }
For best results, the wrapper cookbook's recipe should be placed before the Ceph cookbook in the node's runlist. This will ensure that any attributes are in place before the Ceph cookbook runs and consumes those attributes.
Ceph Monitor
Ceph monitor nodes should use the ceph-mon role.
Includes:
- ceph-chef::default
Ceph Metadata Server
Ceph metadata server nodes should use the ceph-mds role.
Includes:
- ceph-chef::default
Ceph OSD
Ceph OSD nodes should use the ceph-osd role
Includes:
- ceph-chef::default
Ceph RADOS Gateway (RGW)
Ceph RGW nodes should use the ceph-radosgw role
ATTRIBUTES
General
-
node['ceph']['search_environment']
- a custom Chef environment to search when looking for mon nodes. The cookbook defaults to searching the current environment -
node['ceph']['branch']
- selects whether to install the stable, testing, or dev version of Ceph -
node['ceph']['version']
- install a version of Ceph that is different than the cookbook default. If this is changed in a wrapper cookbook, some repository urls may also need to be replaced, and they are found in attributes/repo.rb. If the branch attribute is set to dev, this selects the gitbuilder branch to install node['ceph']['extras_repo']
- whether to install the ceph extras repo. The tgt recipe requires thisnode['ceph']['config']['fsid']
- the cluster UUIDnode['ceph']['config']['global']['public network']
- a CIDR specification of the public networknode['ceph']['config']['global']['cluster network']
- a CIDR specification of a separate cluster replication networknode['ceph']['config']['config-sections']
- add to this hash to add extra config sections to the ceph.confnode['ceph']['user_pools']
- an array of pool definitions, with attributesname
,pg_num
andcreate_options
(optional), that are automatically created when a monitor is deployed
Ceph MON
-
node['ceph']['config']['mon']
- a hash of settings to save in ceph.conf in the [mon] section, such as'mon osd nearfull ratio' => '0.75'
Ceph OSD
-
node['ceph']['osd_devices']
- an array of OSD definitions for the current node -
node['ceph']['config']['osd']
- a hash of settings to save in ceph.conf in the [osd] section, such as'osd max backfills' => 2
-
node['ceph']['config']['osd']['osd crush location']
- this attribute can be set on a per-node basis to maintain Crush map locations
Ceph MDS
-
node['ceph']['config']['mds']
- a hash of settings to save in ceph.conf in the [mds] section, such as'mds cache size' => '100000'
-
node['ceph']['cephfs_mount']
- where the cephfs recipe should mount CephFS -
node['ceph']['cephfs_use_fuse']
- whether the cephfs recipe should use the fuse cephfs client. It will default to heuristics based on the kernel version
Ceph RADOS Gateway (RGW)
Note: Only supports the newer 'civetweb' version of RGW (not Apache)
-
node['ceph']['radosgw']['port']
- Port of the rgw. Defaults to 80 -
node['ceph']['radosgw']['rgw_dns_name']
- the main domain of the radosgw daemon, to calculate the bucket name from a subdomain
Resources/Providers
ceph_client
The ceph_client LWRP provides an easy way to construct a Ceph client key. These keys are needed by anything that needs to talk to the Ceph cluster, including RGW, CephFS, and RBD access.
Actions
- :add - creates a client key with the given parameters
Parameters
- :name - name attribute. The name of the client key to create. This is used to provide a default for the other parameters
- :caps - A hash of capabilities that should be granted to the client key. Defaults to
{ 'mon' => 'allow r', 'osd' => 'allow r' }
- :as_keyring - Whether the key should be saved in a keyring format or a simple secret key. Defaults to true, meaning it is saved as a keyring
- :keyname - The key name to register in Ceph. Defaults to
client.#{name}.#{hostname}
- :filename - Where to save the key. Defaults to
/etc/ceph/ceph.client.#{name}.#{hostname}.keyring
ifas_keyring
and/etc/ceph/ceph.client.#{name}.#{hostname}.secret
if notas_keyring
- :owner - Which owner should own the saved key file. Defaults to root
- :group - Which group should own the saved key file. Defaults to root
- :mode - What file mode should be applied. Defaults to '00640'
ceph_cephfs
The ceph_cephfs LWRP provides an easy way to mount CephFS. It will automatically create a Ceph client key for the machine and mount CephFS to the specified location. If the kernel client is used, instead of the fuse client, a pre-existing subdirectory of CephFS can be mounted instead of the root.
Actions
- :mount - mount CephFS
- :umount - unmount CephFS
- :remount - remount CephFS
- :enable - adds an fstab entry to mount CephFS
- :disable - removes an fstab entry to mount CephFS
Parameters
- :directory - name attribute. Where to mount CephFS in the local filesystem
- :use_fuse - whether to use ceph-fuse or the kernel client to mount the filesystem. ceph-fuse is updated more often, but the kernel client allows for subdirectory mounting. Defaults to true
- :cephfs_subdir - which CephFS subdirectory to mount. Defaults to '/'. An exception will be thrown if this option is set to anything other than '/' if use_fuse is also true
ceph_pool
The ceph_pool LWRP provides an easy way to create and delete Ceph pools.
It assumes that connectivity to the cluster is setup and that admin credentials are available from default locations, e.g. /etc/ceph/ceph.client.admin.keyring.
Actions
- :add - creates a pool with the given number of placement groups
- :delete - deletes an existing pool
Parameters
- :name - the name of the pool to create or delete
- :pg_num - number of placement groups, when creating a new pool
- :create_options - arguments for pool creation (optional)
- :force - force the deletion of an exiting pool along with any data that is stored in it
DEVELOPING
Style Guide
This cookbook requires a style guide for all contributions. Travis will automatically verify that every Pull Request follows the style guide.
- Install ChefDK
- Activate ChefDK's copy of ruby:
eval "$(chef shell-init bash)"
bundle install
bundle exec rake style
Testing
This cookbook uses Test Kitchen to verify functionality. A Pull Request can't be merged if it causes any of the test configurations to fail.
- Install ChefDK
- Activate ChefDK's copy of ruby:
eval "$(chef shell-init bash)"
bundle install
bundle exec kitchen test aio-debian-74
bundle exec kitchen test aio-ubuntu-1204
bundle exec kitchen test aio-ubuntu-1404
LICENSE AND AUTHORS
Author: Chris Jones cjones303@bloomberg.net
Copyright 2016, Bloomberg Finance L.P.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Dependent cookbooks
apache2 >= 1.1.12 |
apt >= 0.0.0 |
yum >= 3.8.1 |
yum-epel >= 0.0.0 |
poise >= 2.5.0 |
Contingent cookbooks
There are no cookbooks that are contingent upon this one.
ceph-chef
NOTE: You can find out how to use this cookbook and see how Bloomberg uses it at:
https://github.com/bloomberg/chef-bcs
v0.9.0 (2015-11-17)
BETA: Documentation needs more clarity and substance. There are still true OS agnostic testing going on. One major
change coming is an update to the newer Chef style of moving provider/resource into classes in the libraries folder.
This will simplify the cookbook structure going forward.
- Support for Chef 12.5+
- Complete Ceph functionality
- Only supports Ceph Hammer and above
- RADOS Gateway (RGW) only supports civetweb style (no Apache)
Foodcritic Metric
0.9.20 failed this metric
FC002: Avoid string interpolation where not required: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/erasure_profiles_set.rb:28
FC002: Avoid string interpolation where not required: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/pools_remove.rb:28
FC002: Avoid string interpolation where not required: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_federated.rb:119
FC002: Avoid string interpolation where not required: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_federated.rb:120
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:565
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:574
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:583
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:592
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:601
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:610
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:619
FC004: Use a service resource to start and stop services: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/mon_stop.rb:35
FC004: Use a service resource to start and stop services: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/osd_stop_all.rb:34
FC022: Resource condition within loop may not behave as expected: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:138
FC022: Resource condition within loop may not behave as expected: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_federated.rb:49
FC022: Resource condition within loop may not behave as expected: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_federated.rb:58
FC023: Prefer conditional attributes: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/maint_osd_remove_zap.rb:70
FC023: Prefer conditional attributes: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/maint_osd_remove_zap.rb:85
FC023: Prefer conditional attributes: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/maint_osd_remove_zap.rb:94
FC023: Prefer conditional attributes: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/rpm.rb:37
FC034: Unused template variables: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/templates/default/ceph.conf.erb:1
FC048: Prefer Mixlib::ShellOut: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_users.rb:26
FC048: Prefer Mixlib::ShellOut: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_users.rb:27
FC048: Prefer Mixlib::ShellOut: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_users.rb:37
FC048: Prefer Mixlib::ShellOut: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_users.rb:38
0.9.20 failed this metric
FC002: Avoid string interpolation where not required: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/pools_remove.rb:28
FC002: Avoid string interpolation where not required: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_federated.rb:119
FC002: Avoid string interpolation where not required: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_federated.rb:120
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:565
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:574
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:583
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:592
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:601
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:610
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:619
FC004: Use a service resource to start and stop services: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/mon_stop.rb:35
FC004: Use a service resource to start and stop services: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/osd_stop_all.rb:34
FC022: Resource condition within loop may not behave as expected: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/libraries/ceph_chef_helper.rb:138
FC022: Resource condition within loop may not behave as expected: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_federated.rb:49
FC022: Resource condition within loop may not behave as expected: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_federated.rb:58
FC023: Prefer conditional attributes: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/maint_osd_remove_zap.rb:70
FC023: Prefer conditional attributes: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/maint_osd_remove_zap.rb:85
FC023: Prefer conditional attributes: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/maint_osd_remove_zap.rb:94
FC023: Prefer conditional attributes: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/rpm.rb:37
FC034: Unused template variables: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/templates/default/ceph.conf.erb:1
FC048: Prefer Mixlib::ShellOut: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_users.rb:26
FC048: Prefer Mixlib::ShellOut: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_users.rb:27
FC048: Prefer Mixlib::ShellOut: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_users.rb:37
FC048: Prefer Mixlib::ShellOut: /tmp/cook/65cc2de329e569590b5acca3/ceph-chef/recipes/radosgw_users.rb:38