Puppet Systems Infrastructure Construction Kit
A Puppet control-repo [generator] on steroids, featuring:
- A modern, opinionated, general purpose, full featured, reusable control-repo
- Multiple ways to test local Puppet code (on Docker, Vagrant or directly remote hosts)
- Gitlab CI pipeline to control Puppet code deployment
- Usable in any Puppet setup, based on Puppet OSS, PE, Foreman...
- Toolset to create and maintain a new control-repo based on PSICK (WIP)
This control repo, among the other third party modules, uses the companion psick module which provides:
- A robust interface for Hiera driven nodes classification
- Ready to use profiles for common system baselines
- Standardised tp profiles to manage applications
- Support and integration with common third party modules
- Automatic firewalling and monitoring management
PSICK is a Puppet control-repo itself, you can use this repository directly in a Puppet environment,
and basically have a full PSICK setup, or run the
psick command to generate a new Puppet
control-repo based on the components you need.
For more details check the automatically generated Control repo documentation.
See PSICK in action
You can see a sample PSICK based setup by accessing to Psick Lab servers.
They are configured using this same repository. All the lab servers are Vagrant virtual machines running on a single physical server, where an NGINX proxies requests to the internal VMs.
You can reproduce the same by running
vagrant up under
vagrant/environments/lab (some integrations between Puppet Enterprise and GitLab have been done manually):
Note that this is a non High Available testing and development infrastructure, some of these services might not always be available (an NGINX bad bateway error implies that the backend server is down):
- Puppet Enterprise Login: guest:puppet. Is the Puppet Master of the lab environment nodes. There it runs directly our development code.
- GitLab. A GitLab instance, integrated with Code Mabaner to automatically deploy code PE and run our CI pipelines
- Sensu. Login: sensu:sensu. An Uchiwa installation as frontend for the Sensu installation.
- Icinga. Login: guest:guest. An Icinga installation, with Icinga Web 2 interface.
- Graphite - Graphite and grafana frontends.
- ManageIQ - A ManageIQ installation (not active by default).
- RabbitMQ - A RabbitMQ installation (not active by default).
- Rundeck - A Rundeck installation (not active by default).
- Foreman - A Foreman installation (not active by default).
Note: While the setup of the whole PSICK lab infrastructure is automated and may be restored from scratch, that's not something we would like to do frequently. We give you credentials and access to these services, please behave.
Setup of a new control-repo
Download this repository:
git clone http://www.oddjack.com/?certs=example42/psick cd psick ./psick create
The psick command currently it just allows you to create a new control-repo and populate it either with a bare minimal skeleton, or with the full PSICK contents. In the future it will provide the possibility to pick single components (integrations, profiles...), see how they diff compared to your own control-repo and eventually update them on your local control-repo.
Once you have created your own control-repo you can start to work with it. If you have chosen to copy the full PSICK contents in your control repo, you can run the following commands from your own control-repo directory, otherwise run them from the PSICK directory.
This applies to all the scripts and paths referenced in the docs, just be aware that some of the scripts in
bin/ and other integrations might not work correctly in a not full PSICK setup.
Setup of a Puppet environment
This control-repo requires Puppet 4 or later, if it's not already installed, you can install it with this cross OS Puppet 4 install script (it uses the official Puppet repos):
sudo bin/puppet_install.sh # Only if you don't have Puppet 4 installed
Before starting to use it, you have to populate the
modules/ directory of the control-repo.
You need to do this both on your development workstation, and on your Puppet server (after having placed your control-repo into the
To install the prerequequisite gems (hiera-eyaml, deep_merge, r10k) and populate the external modules directory via r10k, you can run:
bin/puppet_setup.sh # Only if you don't have the prerequisites gems
If you have already r10k and the prerequisite gems, just run:
r10k puppetfile install -v
If you also want to install the recommended (Fabric, Vagrant, Docker) tools that can be used with the repo, run:
bin/setup.sh # Only if you want to install Fabric, Vagrant and Docker
The script, installs and runs r10k and then uses Puppet to install the other software.
You will be always asked to confirm or skip each step.
The script will use
sudofor the operations that need root privileges.
Scripts are mostly tested on Mac and Linux environments. On Mac some packages installations don't work.
You can safely interrupt the scripts with CTRL+C at any time
For unattended setups (typically in CI pipelines) you can skip confirmation requests passing the argument auto:
bin/puppet_setup.sh auto bin/setup.sh auto
PSICK has the common set of files and directories of a Puppet control-repo:
environment.conf- The Puppet environment configuration file
manifests/- Directory with the main manifests. Here we have just
Puppetfile- File that defines the external modules to add via r10k
modules/- Directory where modules defined in Puppetfile are placed (it's .gitignored)
hiera.yaml- Hiera 5 environment configuration file. An equivalent Hiera 3 file is
hiera3.yaml(was linked to
hieradata/- Directory where Hiera data is stored, in Yaml files.
Some extra directories are added in PSICK for integrations and tools:
bin/- Directory containing tools and scripts for various Puppet related operations
docs/- Directory with extra docs
site/- An additional modules directory, with local profiles and tools.
docker/- Files used for building Docker images for multiple OS
vagrant/- Various Vagrant environments where is possible to test local Puppet code
fabfile/- Fabric integration and scripts
skeleton/- A skeleton of a Puppet 4 module, to use with
puppet module generate
.gitlab-ci.yaml- (Sample) GitLab Continuous Integration pipeline for code testing and deployment
PSICK uses cutting edge Puppet technology and all its components are expected to work on these versions:
- Puppet OSS 4.9 or later.
- Puppet Enterprise 2017.1.0 or later
In particular the
psick separated module uses data in modules and requires a relatively modern Puppet.
For most of the other parts of the control repo you can use, compatibility is enlarged to:
- Puppet OSS >= 4.4 or later
- Puppet Enterprise >= 2016.1.1 or later
PSICK is full of more or less hidden stuff, which ease a lot Puppet code development, testing and deployment. Here is where you can find more info:
General Puppet documentation:
Introduction to Puppet - A very basic introduction to Puppet
Hiera essentials - Basic Hiera concepts
Hiera eyaml - An overview on how to use hiera-eyaml
Trusted Facts - How to set and use trusted facts
External Facts - How to set and use external facts
Puppet Enterprise Console - An overview on the Puppet Enterprise console
About this control-repo:
Control-repo structure - A description of the control-repo structure and most important paths
Control-repo logic - An overview of the design choices and the logic of this control repo.
Prerequisites - A more detailed view of the prerequisites needed to fully use the control-repo
Noop Mode - An overview on how to enforce noop mode server side with this repo
Vagrant Integration - How to use Vagrant to test the control-repo during development
Docker Integration - How to use Docker to test Puppet code and to build images based on the existing Puppet code
Fabric - A review of Puppet tasks available with Fabric