cookbook 'btm-test', '~> 0.1.6'
btm-test (7) Versions 0.1.6 Follow1
Installs/Configures anaconda
cookbook 'btm-test', '~> 0.1.6', :supermarket
knife supermarket install btm-test
knife supermarket download btm-test
anaconda cookbook
Chef cookbook for installing Continuum Analytic's
Anaconda: "completely free Python
distribution for large-scale data processing, predictive analytics, and
scientific computing". Specifically:
- Anaconda 2.2 or 2.3
- python2 or python3
- x86 or x86_64
- Miniconda
- python2 or python3
- x86 or x86_64
- Usage tested on Ubuntu, unittested on Debian, CentOS, and RedHat. See [rspec tests](spec/default_spec.rb#L100) and [kitchen tests](.kitchen.yml#L16) for the full list.
This also serves as an example for developing and testing Chef cookbooks. It
uses:
-
ChefDK; 0.8.1
- chef-client 12.4.4
- Berkshelf for dependency resolution; 3.3.0
- Test Kitchen for comprehensive testing across multiple platforms, with tests written in serverspec; 1.4.2
- Docker, with kitchen-docker integration
- Foodcritic for style checking; 5.0.0
- RSpec/Chefspec for rapid testing; 3.3.2
Note that the release process uses Chef 11 because of https://github.com/chef/chef/issues/3888
In addition:
- Vagrant to provide an out-of-the-box working example; 1.7.4
Requirements
If you want to just have a working Anaconda VM, install:
- Vagrant
For the full experience (e.g. running the test suite), also install:
- ChefDK
- Docker
- Don't forget Docker Machine if you're on OSX; installing this via homebrew is highly recommended.
Quickstart
The sample [Vagrantfile](Vagrantfile) will build you an Anaconda VM with no
changes necessary; note it might take a few minutes to download the Anaconda
installer itself.
$> vagrant up --provision ... # the sample image includes `recipe[anaconda::shell_conveniences]`, which # means conda is already in PATH via /etc/profile.d $> vagrant ssh $vagrant> conda --version conda 3.14.1 # or you add it to PATH manually $> vagrant ssh $vagrant> export PATH=/opt/anaconda/2.3.0/bin:${PATH} $vagrant> conda --version conda 3.14.1
It includes a Jupyter (IPython) notebook server accessible at http://33.33.33.123:8888
Lastly, to use it in a cookbook:
include_recipe 'anaconda::default'
Warning! If you're also using the python cookbook...
You MUST include recipe[anaconda::python_workaround]
, otherwise subsequent
chef runs will fail. See the
issue for
details.
Usage, recipes, attributes, and resources
The main recipe is anaconda::default
. Include it in your runlist, and it will
install the package as well as any necessary dependencies.
The following are user-configurable attributes. Check
[attributes/default.rb](attributes/default.rb) for more details.
-
anaconda
-
version
: the Anaconda version to install. Valid values are: - 2.2.0
- 2.3.0
- latest (for miniconda only)
-
python
: which version of Python to install for. Valid values are: - python2
- python3
-
flavor
: what architecture the instance is. Valid values are: - nil (will autodetect)
- x86 (32-bit)
- x86_64 (64-bit)
-
install_type
: which Anaconda distribution to install. Valid values are: - anaconda
- miniconda
-
install_root
: the parent directory of all anaconda installs. note that individual installs go into#{install_root}/#{version}
-
accept_license
: must be explicitly set to the stringyes
(there are no defaults); any other value will reject the license. -
owner
: the user who owns the install -
group
: the group who owns the install
-
recipe[anaconda::shell_conveniences]
Include this to have the environment set for all users (login shells) via
/etc/profile.d
. Useful for development.
resource anaconda_package
You can use the anaconda_package
resource to install new packages into the
Anaconda environment:
# I don't know what 'astroid' is, just using it as a sample package anaconda_package 'astroid' do # the other supported action is `:remove` action :install end
See the [resource definition](resources/package.rb) for additional options; in
general, all it does is present the same options as conda install
/conda
.
remove
resource anaconda_nbservice
This only works with a full Anaconda installation! I.e. the notebook service
will not work out-of-the-box if installed with miniconda.
The anaconda_nbservice
will run a Jupyter notebook server as a runit service:
anaconda_nbservice 'notebook-server' do # listen on all interfaces; there will be a warning since security is # disabled ip '*' port '8888' install_dir '/opt/ipython/server' service_action [ :enable, :start ] end
The standard configuration should be good enough, but you might need to write
your own run service template:
anaconda_nbservice 'server-with-custom-template' do user ipython_user group ipython_group install_dir install_dir template_cookbook 'your_cookbook' # note that if your template name is TEMPLATE, then this value should be # 'TEMPLATE", but the file should be 'sv-TEMPLATE-run.erb' run_template_name 'your_template_name' run_template_opts({ ... }) service_action [ :enable, :start ] end
Tests
To run the full test suite:
# this will take a while, especially the first time $> script/cibuild ... # check the final result; bash return codes: 0 is good, anything else is not $> echo $?
- to run just the [chefspecs](spec/default_spec.rb):
$> rspec
- to run just the test kitchen serverspec [integration tests](test/integration/default/serverspec/default_spec.rb):
# this is done via docker/kitchen-docker # the list of OSes is defined in .kitchen.yml $> kitchen verify # test a specific OS; `kitchen list` $> kitchen verify default-ubuntu-1204
- check for style issues with Foodcritic
$> foodcritic
Releases and issues
Standard stuff:
- master is the active version in development
- releases are made with
Github, and
git tag
'ed - issues should be opened in the Github issue tracker
TODO
- add a pre-provision for kitchen tests to avoid redownloading the installer on every test (really slows down the tests)
-
https://github.com/poise/python is now deprecated, in favor of
https://github.com/poise/poise-python; see if the python workaround is still
necessary
- the supermarket version (https://supermarket.chef.io/cookbooks/python) looks like it also points to the deprecated one
Author
Author:: Matt Chu (matt.chu@gmail.com)
Dependent cookbooks
apt >= 0.0.0 |
runit >= 0.0.0 |
bzip2 >= 0.0.0 |
tar >= 0.0.0 |
Contingent cookbooks
There are no cookbooks that are contingent upon this one.
CHANGELOG
0.6.2
First official 0.6.x release; see 0.6.0 changelog for details.
Note that this reverts the release process back to using Chef 11.x/ChefDK 0.3.6
because of https://github.com/chef/chef/issues/3888.
0.6.1 (broken, untagged release thanks to ChefDK)
Do not use.
0.6.0 (unreleased thanks to ChefDK)
This is a backwards-INCOMPATIBLE release
API, cookbook changes:
- Refactored attributes to be cleaner and more readable
- The user and group are now created if they don't exist, and by default are installed under anaconda/anaconda
- New attributes for configuring notebook server
- The OS architecture (x86 versus x86_64) is now autodetected by default
Platform, support changes:
- Added support for Anaconda 2.3 (all pythons and all architectures)
- Removed support for all Anacondas < 2.2.0
- updated metadata.rb 'supports' tags (hopefully this doesn't break anything)
Tools changes:
- Migrated to chefdk 0.8.1
- upgraded gems
- updated chefspec
- updated documentation
- removed rbenv/rvm files
Testing changes:
- Moved to kitchen-docker for Docker-based testing, which is much much much faster
- Updated supported (tested) platforms:
- ubuntu:
- removed: 12.10, 13.04, 13.10
- added: 15.04
- debian:
- removed: 6.0.5
- added: 7.8, 8.1 (7.9 and 8.2 not in fauxhai yet)
- centos:
- removed: 5.8, 6.0, 6.3
- added: 5.11, 6.6, 7.1.1503
- redhat (chefspec only, no kitchen testing):
- removed: 5.8, 6.3
- added: 5.9, 6.6, 7.1.1503
0.5.1
- Added support for Anaconda 2.2.0 with Python 3
- Courtesy: https://github.com/nareynolds
- Minor cleanup: remove unused file
0.5.0
- Upgraded to Anaconda 2.2.0 from 2.0.1
- Includes support for Anaconda 2.1.0
- Added support for Miniconda
- Upgraded Vagrantfile:
- Upgraded Chef from 11.16+ to 12+ to match Kitchen tests
- Upgraded OS from 12.04 to 14.04
0.4.4
Backwards-compatible fix for issue when using both the python cookbook and this
one. Issue: https://github.com/thmttch/chef-continuum-anaconda/issues/12
0.4.3
Unintended release, thanks to Chef Supermarket's crappiness.
0.4.2
New resource 'anaconda_nbservice', for running an IPython notebook server.
Somewhat experimental, it's fairly basic but usable.
0.4.1
Hotfix release:
- bad attribute name broke conda package installs and removes. Identified by https://github.com/thmttch/chef-continuum-anaconda/issues/8, fixed by https://github.com/thmttch/chef-continuum-anaconda/pull/9. Cheers to @touchdown for the report.
0.4.0
Major cleanup and basic usability fixes (read: almost not sure why it worked
before, if it even did):
- (core, breaking) Renamed
anaconda::shell-conveniences
toanaconda::shell_conveniences
- (core, breaking) No longer uses the
python
cookbook; this caused problems after anaconda was installed - (core)
shell_conveniences
now installs into/etc/profile.d
, so it is automatically sourced by login shells - (core) Ubuntu 14.04 is now a supported (and fully tested) platform
- (core) Ubuntu 14.04 is now the default platform used in the sample Vagrantfile
- (core) Vagrantfile now correctly installs Anaconda, like the README said it was supposed to
- (docs) various readme updates
- (testing) Complete coverage of all resources in chefspec
- (testing) Removed Ubuntu 12.10 and 13.04 from kitchen testing; there's
something wrong with
apt-get
on these images. It appears to be something about the apt keys, but they're just removed from testing for now.
0.3.3
Bugfix release:
- Fixes related to install script and permissions (#5). Courtesy @mwalton236
0.3.1
New resource: anaconda_package
, for installing and removing packages via
conda
0.3.0
Rewrote all project history
0.2.1
Fix incorrect checksums for Anaconda 2.0.1
0.2.0
Feature improvments:
- updated the default installation to Anaconda 2.0.1, from 1.9.2
Miscellaneous:
- minor
cibuild
improvement to ensure clean tests - minor code changes to support updated toolchain
- removed Chef Development Kit recommendation after experimenting with it
Collaborator Number Metric
0.1.6 failed this metric
Failure: Cookbook has 0 collaborators. A cookbook must have at least 2 collaborators to pass this metric.
Contributing File Metric
0.1.6 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a CONTRIBUTING.md file
Foodcritic Metric
0.1.6 failed this metric
FC064: Ensure issues_url is set in metadata: btm-test/metadata.rb:1
FC065: Ensure source_url is set in metadata: btm-test/metadata.rb:1
FC066: Ensure chef_version is set in metadata: btm-test/metadata.rb:1
Run with Foodcritic Version 16.3.0 with tags metadata,correctness ~FC031 ~FC045 and failure tags any
No Binaries Metric
0.1.6 passed this metric
Testing File Metric
0.1.6 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file
Version Tag Metric
0.1.6 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must include a tag that matches this cookbook version number
0.1.6 failed this metric
0.1.6 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a CONTRIBUTING.md file
Foodcritic Metric
0.1.6 failed this metric
FC064: Ensure issues_url is set in metadata: btm-test/metadata.rb:1
FC065: Ensure source_url is set in metadata: btm-test/metadata.rb:1
FC066: Ensure chef_version is set in metadata: btm-test/metadata.rb:1
Run with Foodcritic Version 16.3.0 with tags metadata,correctness ~FC031 ~FC045 and failure tags any
No Binaries Metric
0.1.6 passed this metric
Testing File Metric
0.1.6 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file
Version Tag Metric
0.1.6 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must include a tag that matches this cookbook version number
0.1.6 failed this metric
FC065: Ensure source_url is set in metadata: btm-test/metadata.rb:1
FC066: Ensure chef_version is set in metadata: btm-test/metadata.rb:1
Run with Foodcritic Version 16.3.0 with tags metadata,correctness ~FC031 ~FC045 and failure tags any
0.1.6 passed this metric
Testing File Metric
0.1.6 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file
Version Tag Metric
0.1.6 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must include a tag that matches this cookbook version number
0.1.6 failed this metric
0.1.6 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must include a tag that matches this cookbook version number