cookbook 'airflow', '~> 1.2.2'
airflow (6) Versions 1.2.2 Follow4
Installs and configures Airflow workflow management platform.
cookbook 'airflow', '~> 1.2.2', :supermarket
knife supermarket install airflow
knife supermarket download airflow
Airflow Chef Cookbook
Installs and configures Airflow workflow management platform. More information about Airflow can be found here: https://github.com/airbnb/airflow
Supported Platforms
Ubuntu (Tested on 14.04, 16.04).
CentOS (Tested on 7.2).
Limitations
The Airflow all and oracle packages are not supported, this is due the Oracle package having dependencies which cannot be automatically installed. I will look how to solve this and add support for those packages at later stage.
Contributing
Please follow instructions in the [contributing doc](CONTRIBUTING.md).
Usage
- Use the relevant cookbooks to install and configure Airflow.
- Use environment variable in /etc/default/airflow (for Ubuntu) or /etc/sysconfig/airflow (for CentOS) to configure Airflow during the startup process. (More info about Airflow environment variables at: Setting Configuration Options)
- Make sure to run airflow initdb as part of your startup script.
Recipes
- default - Executes other recipes.
- directories - Creates required directories.
- user - Creates OS user and group.
- packages - Installs OS and pip packages.
- config - Handles airflow.cfg
- services - Creates services env file.
- webserver - Configures service for webserver.
- scheduler - Configures service for scheduler.
- worker - Configures service for worker.
- flower - Configures service for flower.
- kerberos - Configures service for kerberos.
- packages - Installs Airflow and supporting packages.
Attributes
User config
- ["airflow"]["airflow_package"] - Airflow package name, defaults to 'apache-airflow'. Use 'airflow' for installing version 1.8.0 or lower.
- ["airflow"]["version"] - The version of airflow to install, defaults to latest (nil).
- ["airflow"]["user"] - The user Airflow is executed with and owner of all related folders.
- ["airflow"]["group"] - Airflow user group.
- ["airflow"]["user_uid"] - Airflow user uid
- ["airflow"]["group_gid"] - Airflow group gid
- ["airflow"]["user_home_directory"] - Airflow user home directory.
- ["airflow"]["shell"] - Airflow user shell.
General config
- ["airflow"]["directories_mode"] - The permissions airflow and user directories are created.
- ["airflow"]["config_file_mode"] - The permissions airflow.cfg is created.
- ["airflow"]["bin_path"] - Path to the bin folder, default is based on platform.
- ["airflow"]["run_path"] - Pid files base directory
- ["airflow"]["is_upstart"] - Should upstart be used for services, determined automatiaclly.
- ["airflow"]["init_system"] - The init system to use when configuring services, only upstart or systemd are supported and defaults based on ["airflow"]["is_upstart"] value.
- ["airflow"]["env_path"] - The path to services env file, determined automatiaclly.
Python config
- ["airflow"]["python_runtime"] = Python runtime as used by poise-python cookbook.
- ["airflow"]["python_version"] = Python version to install as used by poise-python cookbook.
- ["airflow"]["pip_version"] = Pip version to install (true - installs latest) as used by poise-python cookbook.
Package config
- default['airflow']['packages'] - The Python packages to install for Airflow.
- default['airflow']['dependencies'] - The dependencies of the packages listed in default['airflow']['packages']. These are OS packages, not Python packages.
airflow.cfg
This cookbook enables to configure any airflow.cfg paramters dynamically by using attributes structure like (see the attributes file for [airflow.cfg examples](attributes/default.rb)):
["airflow"]["config"]["CONFIG_SECTION"]["CONFIG_ENTRY"]
License
Apache 2.0 (http://www.apache.org/licenses/LICENSE-2.0)
Author
Dependent cookbooks
apt >= 0.0.0 |
poise-python >= 0.0.0 |
Contingent cookbooks
There are no cookbooks that are contingent upon this one.
Airflow cookbook changelog
1.2.1
- Airflow package name support (default: apache-airflow)
- Lazy config loading
1.2.0
- Added ubuntu 16.04 support
- Packages version pinning
- Fixed python and pip installation
- Bug fixes
1.1.0
- Added CentOS support
- Flower, kerberos and worker services
- New airflow packages support
1.0.1
- Some foodcritic fixes
- README updated
- Fixed some minor attributes issues and cosmetics
1.0.0
- Initial release of airflow-cookbook
Collaborator Number Metric
1.2.2 failed this metric
Failure: Cookbook has 1 collaborators. A cookbook must have at least 2 collaborators to pass this metric.
Contributing File Metric
1.2.2 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a CONTRIBUTING.md file
Foodcritic Metric
1.2.2 failed this metric
FC069: Ensure standardized license defined in metadata: airflow/metadata.rb:1
Run with Foodcritic Version 16.3.0 with tags metadata,correctness ~FC031 ~FC045 and failure tags any
No Binaries Metric
1.2.2 passed this metric
Testing File Metric
1.2.2 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file
Version Tag Metric
1.2.2 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must include a tag that matches this cookbook version number
1.2.2 failed this metric
1.2.2 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a CONTRIBUTING.md file
Foodcritic Metric
1.2.2 failed this metric
FC069: Ensure standardized license defined in metadata: airflow/metadata.rb:1
Run with Foodcritic Version 16.3.0 with tags metadata,correctness ~FC031 ~FC045 and failure tags any
No Binaries Metric
1.2.2 passed this metric
Testing File Metric
1.2.2 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file
Version Tag Metric
1.2.2 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must include a tag that matches this cookbook version number
1.2.2 failed this metric
Run with Foodcritic Version 16.3.0 with tags metadata,correctness ~FC031 ~FC045 and failure tags any
1.2.2 passed this metric
Testing File Metric
1.2.2 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file
Version Tag Metric
1.2.2 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must include a tag that matches this cookbook version number
1.2.2 failed this metric
1.2.2 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must include a tag that matches this cookbook version number