Menu

Chef Supermarket

Adoptable Cookbooks List

Looking for a cookbook to adopt? You can now see a list of cookbooks available for adoption!
List of Adoptable Cookbooks

Supermarket Belongs to the Community

Supermarket belongs to the community. While Chef has the responsibility to keep it running and be stewards of its functionality, what it does and how it works is driven by the community. The chef/supermarket repository will continue to be where development of the Supermarket application takes place. Come be part of shaping the direction of Supermarket by opening issues and pull requests or by joining us on the Chef Mailing List.

View our collection of guides, documentation, and articles

Cookbooks
Advanced Options

Select Badges

Select Supported Platforms

Select Status

RSS

logstash (16) Versions 1.0.0

Installs/Configures logstash

Policyfile
Berkshelf
Knife 
cookbook 'logstash', '~> 1.0.0', :supermarket
cookbook 'logstash', '~> 1.0.0'
knife supermarket install logstash
knife supermarket download logstash
README
Dependencies
Changelog
Quality 33%

<a name="title"></a> chef-logstash Build Status

Description

This is the semi-official 'all-in-one' Logstash cookbook.

This cookbook is primarily a library cookbook.

While you can still use the agent and server recipes, they are not recommended as they are very limited in what they do.

If you are using logstash < 1.2 you might want to use the 0.6.x branch.
If you are using logstash < 1.4 you might want to use the 0.7.x branch.

Requirements

All of the requirements are explicitly defined in the recipes. Every
effort has been made to utilize Community Cookbooks.

However if you wish to use an external ElasticSearch cluster, you will
need to install that yourself and change the relevant attributes for
discovery. The same applies to integration with Graphite.

This cookbook has been tested together with the following cookbooks,
see the Berksfile for more details

Attributes

Default

see [attributes/default.rb](attributes/default.rb)

Beaver (alternative to Logstash Agent)

no longer used. see Community Beaver cookbook

Source

no longer supports installing from source.

Lightweight Resource Providers

These now do all the heavy lifting.

logstash_instance

This will install a logstash instance. It will take defaults from attributes for most attributes.

see [resources/instance.rb](resources/instance.rb)

logstash_service

This will create system init scripts for managing logstash instance. It will take defaults from attributes for most attributes.

see [resources/service.rb](resources/service.rb)

experimental support for pleaserun has been added. Only native for Ubuntu 12.04 has been thoroughly tested.

logstash_config

This will create logstash config files. It will take defaults from attributes for most attributes.

see [resources/config.rb](resources/config.rb)

logstash_pattern

This will install custom grok patterns for logstash. It will take defaults from attributes for most attributes:

see [resources/pattern.rb](resources/pattern.rb)

logstash_plugins

This will install the logstash community plugins:

see [resources/plugins.rb](resources/plugins.rb)

logstash_curator

This will install the ElasticSearch Curator and setup a cron job. This replaces the deprecated index_cleaner:

see [resources/curator.rb](resources/curator.rb)

attribute precidence in logstash LWRPs

We've done our best to make this intuitive and easy to use.

  1. the value directly in the resource block.
  2. the value from the hash node['logstash']['instance'][name]
  3. the value from the hash node['logstash']['instance_default']

You should be able to override settings in any of the above places. It is recommended for readability that you set non-default options in the LWRP resource block. But do whichever makes sense to you.

Searching

There is a search helper library libraries/search.rb which will help you search for values such as elasticsearch_ip. see the server recipe for an example of its usage.

Testing

Rubocop, FoodCritic, Rspec, Test-Kitchen

$ bundle exec rake

Test Kitchen

$ kitchen converge server_ubuntu

Contributing

Any and all contributions are welcome. We do ask that you test your contributions with the testing framework before you send a PR. All contributions should be made against the master branch.

Please update tests and changelist with your contributions.

Documentation contributions will earn you lots of hugs and kisses.

Usage

A proper readme is forthcoming but in the interim....

These two recipes show how to install and configure logstash instances via the provided LWRPs

  • [recipes/server.rb](recipes/server.rb) - This would be your indexer node
  • [recipes/agent.rb](recipes/agent.rb) - This would be a local host's agent for collection

See the elkstack community cookbook for a great example of using the LWRPs provided by this cookbook.

License and Author

  • Author: John E. Vincent
  • Author: Bryan W. Berry (bryan.berry@gmail.com)
  • Author: Richard Clamp (@richardc)
  • Author: Juanje Ojeda (@juanje)
  • Author: @benattar
  • Author: Paul Czarkowski (@pczarkowski)
  • Copyright: 2012, John E. Vincent
  • Copyright: 2012, Bryan W. Berry
  • Copyright: 2012, Richard Clamp
  • Copyright: 2012, Juanje Ojeda
  • Copyright: 2012, @benattar
  • Copyright: 2014, Paul Czarkowski

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Dependent cookbooks

build-essential >= 0.0.0
runit >= 0.0.0
git >= 0.0.0
ant >= 0.0.0
logrotate >= 0.0.0
ark >= 0.0.0
poise-python >= 0.0.0
curl >= 0.0.0
beaver >= 0.0.0

Contingent cookbooks

apache-log-analysis Applicable Versions
apache-log-analysis 0.1.2
apache-log-analysis 0.1.3
apache-log-analysis 0.1.4
apache-log-analysis 0.1.5
elk Applicable Versions
elk 0.0.1
elk 0.0.2
elk 0.0.4
elk 0.0.5
elk 0.0.6
elk 0.0.7
elk 0.0.8
elk 0.0.9
elk 0.1.1
elk 0.1.2
elk 0.1.3
elk 0.1.4
elk 0.1.5
elk 0.1.6
elk 0.1.7
elk 0.1.8
elk 0.1.9
elk 0.2.0
elk 0.2.1
elk 0.2.2
elk 0.2.3
elk 0.2.4
elk 0.2.5
elk 0.2.6
elk 0.2.7
elk 0.2.8
elk 0.2.9
elk 0.3.0
elk 0.3.1
elkstack Applicable Versions
elkstack 0.1.1
elkstack 0.3.0
elkstack 1.1.0
elkstack 2.0.0
elkstack 2.1.4
elkstack 3.0.1
elkstack 3.1.3
elkstack 3.1.4
elkstack 3.2.0
elkstack 3.2.1
elkstack 3.2.2
elkstack 3.2.3
elkstack 3.2.4
elkstack 3.2.6
elkstack 3.2.7
elkstack 4.0.0
elkstack 4.1.0
elkstack 4.2.0
elkstack 4.2.1
elkstack 4.2.2
elkstack 4.2.3
elkstack 5.0.0
elkstack 5.0.1
elkstack 5.0.2
elkstack 6.0.0
elkstack 6.0.1
elkstack 6.0.2
elkstack 6.0.3
elkstack 6.0.4
et_elk Applicable Versions
et_elk 3.0.0
intel_hadoop_metrics Applicable Versions
intel_hadoop_metrics 1.0.0
intel_hadoop_metrics 1.1.0

CHANGELOG for chef-logstash

1.0.0

  • MAJOR - Get tests passing on Chef 12.7
  • MAJOR - Use a versioned directory structure
  • MAJOR - Use systemd for Debian 8 (Jessie)
  • MINOR - Add Max Heap to systemd init parameters
  • MINOR - Add udp listening port 514
  • MINOR - Remove Vagrant support

0.12.0

  • MAJOR - logstash version 1.5.4
  • MINOR - systemd improvements
  • MINOR - implement join_groups attribute

0.11.2

  • MINOR - default logstash version 1.4.2
  • MINOR - correct status code for initv scripts
  • MINOR - parameterize open file limits for upstart method
  • MINOR - late night bad resource in server recipe.

0.11.0

  • MAJOR - depreciate non runit service types.
  • MINOR - fix bug where node['logstash'][instance_name] must exist.
  • MAJOR - remove pyshipper in favor of beaver community cookbook.
  • MAJOR - remove beaver in favor of community cookbook.
  • MAJOR - assumes ChefDK for Development/Testing
  • MAJOR - use keys from config_template hash to make templates reusable.

0.10.0:

  • major rework of service LWRP
  • rework of attribute precidence
  • node[logstash][default] changed to node[logstash][instance_default]

This file is used to list changes made in each version of chef-logstash.

0.9.2:

  • update to fix PAX header issue on community site

0.9.1:

  • curator LWRP

0.9.0:

this will almost certainly break backwards compatibility

  • support for Logstash 1.4
  • major refactor towards being a library cookbook
    • instance LWRP
    • service LWRP
    • pattern LWP
    • config LWP

0.7.7:

  • Support for new beaver config #239
  • Support for multiline codec #240
  • Parameterize /var/lib/logstash #242
  • Fix parameter spacing option #244

0.7.6:

  • introduced more testing
    • Strainer: rubocop, knife test, foodcritic, chefspec
    • lots of style fixes for rubocop
    • skeleton spec files for each recipe
    • testkitchen + server spec

0.7.5:

  • added fedora systemd support
  • moved zeromq repos to own recipe

0.7.4:

  • bump logstash version to 1.3.2

0.7.3:

  • support for sudo with upstart for agent and server

0.7.2:

  • embedded kibana support

0.7.1:

  • various bugfixes
  • support for multiple logstash workers via attribute.

0.7.0:

New features

  • settable gid when using runit or upstart as supervisor
  • default logstash version 1.2.2
  • attributes to specify: config_dir, home, config_file for both agent and server.
  • don't install rabbit by default
  • allow for conditionals to be set in the filters and outputs hashes.
  • allow for disabling data driven templates.
  • attributes to enable regular(ish) style chef templates.

Bug fixes

  • Vagrantfile cleanup, support more OS
  • Cookbook Dependency cleanup

0.2.1 (June 26, 2012)

New features
* Use ruby hashes supplied by roles to populate inputs, filters,
and outputs
* redhat-family support
* change default version of logstash to 1.1.1preview
* add in Travis-CI support

Bug fixes
* keep apache default site from obscuring kibana

Collaborator Number Metric 
            
1.0.0 passed this metric

Contributing File Metric 
            
1.0.0 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a CONTRIBUTING.md file

Foodcritic Metric 
            
1.0.0 failed this metric
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/config.rb:46
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/curator.rb:41
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/curator.rb:46
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/curator.rb:57
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/curator.rb:73
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/curator.rb:78
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/curator.rb:89
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:50
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:56
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:62
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:75
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:86
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:94
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:103
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:119
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:128
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:138
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:146
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:155
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:165
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:171
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:180
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:188
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:197
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:206
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:215
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:227
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:232
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/instance.rb:239
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/pattern.rb:39
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/plugins.rb:51
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/plugins.rb:68
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/plugins.rb:97
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/service.rb:45
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/service.rb:49
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/service.rb:53
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/service.rb:57
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/service.rb:88
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/service.rb:126
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/service.rb:132
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/service.rb:139
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/service.rb:158
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/service.rb:163
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/service.rb:193
FC085: Resource using new_resource.updated_by_last_action to converge resource: logstash/providers/service.rb:198
FC120: Do not set the name property directly on a resource: logstash/providers/plugins.rb:55
FC120: Do not set the name property directly on a resource: logstash/recipes/server.rb:32
FC122: Use the build_essential resource instead of the recipe: logstash/recipes/source.rb:2
Run with Foodcritic Version 16.3.0 with tags metadata,correctness ~FC031 ~FC045 and failure tags any

No Binaries Metric 
            
1.0.0 passed this metric

Testing File Metric 
            
1.0.0 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file

Version Tag Metric 
            
1.0.0 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must include a tag that matches this cookbook version number

John Vincent John Vincent
Bryan Berry
Chris Lundquist
Uknown
Seth Thomas
Martin Smith

Details

View Source View Issues

Updated June 12, 2017 Created on

Supported Platforms

  • centos >= 0.0.0
  • fedora >= 0.0.0
  • debian >= 0.0.0
  • ubuntu >= 0.0.0
  • redhat >= 0.0.0
  • scientific >= 0.0.0
  • amazon >= 0.0.0

License

Apache-2.0

Download Cookbook