pig chef cookbook

Installs pig, a data analysis program for hadoop. It's like SQL but awesome and infinitely scalable.

Overview

This cookbook

• installs the package, from either the release (tarball available directly from the Apache Pig project) or the Cloudera apt repo.
• builds the piggybank of helper functions, if desired
• links to external jars and .confs (hbase, zookeeper, etc)

Attributes

• [:pig][:home_dir] - Location of pig code (default: "/usr/lib/pig")
• [:pig][:release_url] - URL of pig release tarball (default: ":apache_mirror:/pig/pig-:version:/pig-:version:.tar.gz")
• [:pig][:combine_splits] - tunable: combine small files to reduce the number of map tasks (default: "true")
  • Processing input (either user input or intermediate input) from multiple small files can be inefficient because a separate map has to be created for each file. Pig can now combined small files so that they are processed as a single map. combine_splits turns this on or off.
• [:pig][:version] - (default: "0.9.1")
• [:java][:java_home] - JAVA_HOME environment variable to set for compilation (default: "/usr/lib/jvm/java-6-sun/jre")
  • JAVA_HOME environment variable to set for compilation. This should be the path to the 'jre' subdirectory of your Sun Java install (not OpenJDK).

Recipes

• default - Base configuration for pig
• install_from_package - Installs pig from the cloudera package -- verified compatible, but on a slow update schedule.
• install_from_release - Install From the release tarball.
• integration - Link in jars from hbase and zookeeper
• piggybank - Compiles the Piggybank, a library of useful functions for pig

Integration

Supports platforms: debian and ubuntu

Cookbook dependencies:
* java
* apt
* install_from

License and Author

Author:: Philip (flip) Kromer - Infochimps, Inc (coders@infochimps.com)
Copyright:: 2011, Philip (flip) Kromer - Infochimps, Inc

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

readme generated by cluster_chef's cookbook_munger