diff --git a/README.md b/README.md index f817b1a..7fc7d9c 100644 --- a/README.md +++ b/README.md @@ -10,25 +10,22 @@ Pet](https://blog.engineyard.com/2014/pets-vs-cattle). If you are looking for a role to install Jenkins and you want to configure everything through the web interface and you don't care about being able to -repeatably deploy this same fully-configured Jenkins then you don't need this -role, have a look at the +repeatably deploy this same fully-configured Jenkins, then you don't need this +role. Instead, have a look at the [geerlingguy/ansible-role-jenkins](https://github.com/geerlingguy/ansible-role-jenkins) role instead. Requirements ------------ -Requires curl to be installed on the server. - -If deploying using Docker then you need Docker installed on the server. - -(Docker, apt-get and yum are the only supported ways at the moment although more -ways can easily be added, PRs welcome). +If deploying using Docker, then you need Docker installed on the server. Docker, +apt-get, and yum are the only supported ways at the moment although more ways +can easily be added, PRs welcome. Installation ------------ -Install using ansible galaxy: +Install using Ansible Galaxy: ``` $ ansible-galaxy install emmetog.jenkins @@ -44,7 +41,7 @@ The following variables influence how Jenkins is installed: - `docker`: Install in a Docker container - `apt`: Install Jenkins directly on Ubuntu/Debian Linux systems - `yum`: Install Jenkins directly on RedHat/CentOS Linux systems -- `jenkins_version`: The exact version of jenkins to install +- `jenkins_version`: The exact version of Jenkins to install The following variables influence how Jenkins is configured: @@ -56,6 +53,8 @@ The following variables influence how Jenkins is configured: - `jenkins_java_opts`: Options passed to the Java executable - `jenkins_config_owner`: Owner of Jenkins configuration files - `jenkins_config_group`: Group of Jenkins configuration files +- `jenkins_auth`: How Ansible should authenticate itself with Jenkins, (see the + "Authentication and Security" section below) The following list variables influence the jobs/plugins that will be installed in Jenkins: @@ -76,53 +75,145 @@ Example Playbook vars: jenkins_version: "2.73.1" jenkins_hostname: "jenkins.example.com" - jenkins_port: 80 + jenkins_port: 8080 jenkins_install_via: "docker" jenkins_jobs: - - "my-first-job" - - "another-awesome-job" + - "my-first-job" + - "another-awesome-job" jenkins_include_secrets: true jenkins_include_custom_files: true jenkins_custom_files: - src: "jenkins.plugins.openstack.compute.UserDataConfig.xml" dest: "jenkins.plugins.openstack.compute.UserDataConfig.xml" + jenkins_plugins: + - git + - blueocean jenkins_custom_plugins: - - "openstack-cloud-plugin/openstack-cloud.jpi" + - "openstack-cloud-plugin/openstack-cloud.jpi" roles: - emmetog.jenkins ``` -HTTPS ------ +Managing Configuration Files +---------------------------- -If you want to enable HTTPS on Jenkins, this can be done as follows: +The example above will look for job configuration files in +`{{ playbook_dir }}/jenkins-configs/jobs/my-first-job/config.xml` and +`{{ playbook_dir }}/jenkins-configs/jobs/another-awesome-job/config.xml`. -- Define `jenkins_port_https` to the port that Jenkins should listen on -- Define variables *either* for the JKS keystore or the CA signed certificate: - * For JKS keystore, you'll need to define: - - `jenkins_https_keystore`: Path to the keystore file on the control host, - which will be copied to the Jenkins server by this role. - - `jenkins_https_keystore_password`: Password for said JKS keystore. Use of - the Ansible vault is recommended for this. - * For a CA signed certificate file, you'll need to define: - - `jenkins_https_certificate`: Path to the certificate file, which will be - copied to the Jenkins server by this role. - - `jenkins_https_private_key`: Private key for said CA signed certificate. - Use of the Ansible vault is recommended for this. -- Optionally, `jenkins_https_validate_certs` should be defined to `false` if - you are using a self-signed certificate. +**NOTE**: These directories are customizable, see the +`jenkins_source_dir_configs` and `jenkins_source_dir_jobs` role variables. -If you are deploying Jenkins with Docker, then using a reverse proxy such as -[jwilder/nginx-proxy](https://github.com/jwilder/nginx-proxy) or -[traefik](https://github.com/containous/traefik) is recommended instead of -configuring Jenkins itself. This gives a bit more flexibility and allows for -separation of responsibilities. See the documentation in those projects for -more details on how to deploy the proxies and configure HTTPS. +The role will also look for `{{ playbook_dir }}/jenkins-configs/config.xml` +This `config.xml` file will be copied to the server and used as the job +configuration template. -If using a reverse proxy in front of the Jenkins instance and deploying using -Docker you probably want to set the `jenkins_docker_expose_port` variable to -false so that the port is not exposed on the host, only to the reverse proxy. +The above example will also upload the entire secrets directory under +`{{ playbook_dir }}/jenkins-configs/secrets`, and also copy custom files +defined in the `{{ jenkins_custom_files }}` variable. Note that +`{{ jenkins_include_secrets }}` and `{{ jenkins_include_custom_files }}` +variables should be set to `true` for features these to work. Additionally, +the role can install custom plugins by providing the .jpi or .hpi files in the +`{{ jenkins_custom_plugins }}` list variable. + +The `config.xml` and the custom files are treated as templates so you can put +variables in them, including sensitive data from the Ansible vault. + +When you want to make a change in a configuration file, or you want to add a +new item (such as a job, plugin, etc) the normal workflow is: + +1. Make the change in the Jenkins UI +2. Copy the resulting XML files back into your VCS +3. For newly-created files, don't forget to add them to the respective list: + - For new jobs, these must be added to `jenkins_jobs` + - For custom files, these must be added to `jenkins_include_custom_files` + - For custom plugins, these must be added to `jenkins_custom_plugins` + +Example Jenkins Configuration File +---------------------------------- + +In `{{ jenkins_source_dir_configs }}/config.xml` you put your global Jenkins +configuration, for example: + +```xml + + + + 2.176.1 + RESTART + 1 + EXCLUSIVE + true + + + false + false + + false + + ${JENKINS_HOME}/workspace/${ITEM_FULLNAME} + ${ITEM_ROOTDIR}/builds + + + + + + 0 + 0 + + + + all + false + false + + + + all + 0 + + JNLP-connect + JNLP2-connect + + + + false + + + + +``` + +Example Job Configuration File +------------------------------ + +Here's an example of what you could put in +`{{ playbook_dir }}/jenkins-configs/jobs/my-first-job/config.xml`: + +```xml + + + + My first job, it says "hello world" + false + + + true + false + false + false + + false + + + echo "Hello World!" + + + + + +``` Authentication and Security --------------------------- @@ -163,7 +254,7 @@ To create an API token, you'll need to do the following: ``` jenkins_custom_files: - src: "users/the_username/config.xml" - dest: "users/ci/config.xml" + dest: "users/the_username/config.xml" ``` Note that you may need to change the `src` value, depending on where you save @@ -190,157 +281,40 @@ Likewise, for the above to work, you'll need at least Ansible 2.9.0pre5 or 2.10 (which are, at the time of this writing, both in development. See [this Ansible issue](https://github.com/ansible/ansible/issues/61672) for more details). -Jenkins Configs ---------------- - -The example above will look for the job configs in -`{{ playbook_dir }}/jenkins-configs/jobs/my-first-job/config.xml` and -`{{ playbook_dir }}/jenkins-configs/jobs/another-awesome-job/config.xml`. - -***NOTE***: These directories are customizable, see the -`jenkins_source_dir_configs` and `jenkins_source_dir_jobs` role variables. - -The role will also look for `{{ playbook_dir }}/jenkins-configs/config.xml` -These config.xml will be templated over to the server to be used as the job -configuration. It will upload the whole secrets directory under -`{{ playbook_dir }}/jenkins-configs/secrets` and configure custom files provided -under `{{ jenkins_custom_files }}` variable. Note that -`{{ jenkins_include_secrets }}` and `{{ jenkins_include_custom_files }}` -variables should be set to true for these to work. Additionally the role can -install custom plugins by providing the .jpi or .hpi files as a list under -`{{ jenkins_custom_plugins }}` variable. - -config.xml and custom files are templated so you can put variables in them, for -example it would be a good idea to encrypt sensitive variables in ansible vault. - -Example Job Configs -------------------- - -Here's an example of what you could put in -`{{ playbook_dir }}/jenkins-configs/jobs/my-first-job/config.xml`: - -```xml - - - - My first job, it says "hello world" - false - - - true - false - false - false - - false - - - echo "Hello World!" - - - - - -``` - -Example Jenkins Configs ------------------------ +HTTPS +----- -In `{{ jenkins_source_dir_configs }}/config.xml` you put your global Jenkins -configuration, for example: +If you want to enable HTTPS on Jenkins, this can be done as follows: -```xml - - - - - 2 - NORMAL - true - - - false - - ${ITEM_ROOTDIR}/workspace - ${ITEM_ROOTDIR}/builds - - - - - - ec2-slave-docker-ec2 - false - jenkins-aws-ec2 - - - {{ ssh_jenkins_aws_key }} - - - 1 - - - ami-2654d755 - Docker builder - eu-west-1c - ssh-only - - T2Micro - false - docker - NORMAL - - - - 1 - ubuntu - - - 30 - - false - - 2147483647 - true - false - false - false - - - 22 - - 2147483647 - false - false - - - eu-west-1 - - - 5 - 0 - - - - All - false - false - - - - All - 50000 - - - - -``` +- Define `jenkins_port_https` to the port that Jenkins should listen on +- Define variables *either* for the JKS keystore or the CA signed certificate: + * For JKS keystore, you'll need to define: + - `jenkins_https_keystore`: Path to the keystore file on the control host, + which will be copied to the Jenkins server by this role. + - `jenkins_https_keystore_password`: Password for said JKS keystore. Use of + the Ansible vault is recommended for this. **IMPORTANT**: This string + will be written in plaintext to the Jenkins configuration file on the + server. It will also be visible in the server's process list. Consider + migrating your certificate to a signed certificate file (see below). + * For a CA signed certificate file, you'll need to define: + - `jenkins_https_certificate`: Path to the certificate file, which will be + copied to the Jenkins server by this role. Use of the Ansible vault is + recommended for this file. + - `jenkins_https_private_key`: Private key for said CA signed certificate. + Use of the Ansible vault is recommended for this file. +- Optionally, `jenkins_https_validate_certs` should be defined to `false` if + you are using a self-signed certificate. -Making Changes --------------- +If you are deploying Jenkins with Docker, then using a reverse proxy such as +[jwilder/nginx-proxy](https://github.com/jwilder/nginx-proxy) or +[traefik](https://github.com/containous/traefik) is recommended instead of +configuring Jenkins itself. This gives a bit more flexibility and allows for +separation of responsibilities. See the documentation in those projects for +more details on how to deploy the proxies and configure HTTPS. -When you want to make a big change in a configuration file or you want to add a -new job the normal workflow is to make the change in the Jenkins UI first, then -copy the resulting XML back into your VCS. +If using a reverse proxy in front of the Jenkins instance and deploying using +Docker you probably want to set the `jenkins_docker_expose_port` variable to +false so that the port is not exposed on the host, only to the reverse proxy. License ------- diff --git a/molecule/https/Dockerfile.j2 b/molecule/https/Dockerfile.j2 new file mode 100644 index 0000000..b8ee23f --- /dev/null +++ b/molecule/https/Dockerfile.j2 @@ -0,0 +1,15 @@ +# Molecule managed + +{% if item.registry is defined %} +FROM {{ item.registry.url }}/{{ item.image }} +{% else %} +FROM {{ item.image }} +{% endif %} + +RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 648ACFD622F3D138 && \ + apt-get update && \ + apt-get install -y apt-transport-https aptitude bash ca-certificates sudo python \ + python-apt && \ + apt-get clean + +RUN useradd -G sudo molecule diff --git a/molecule/https/jenkins-configs/config.xml b/molecule/https/jenkins-configs/config.xml new file mode 100644 index 0000000..c6951d2 --- /dev/null +++ b/molecule/https/jenkins-configs/config.xml @@ -0,0 +1,43 @@ + + + + 2.190.2 + RESTART + 1 + EXCLUSIVE + true + + + false + false + + false + + ${JENKINS_HOME}/workspace/${ITEM_FULLNAME} + ${ITEM_ROOTDIR}/builds + + + + + + 0 + 0 + + + + all + false + false + + + + all + 0 + + JNLP-connect + JNLP2-connect + + + + + \ No newline at end of file diff --git a/molecule/https/molecule.yml b/molecule/https/molecule.yml new file mode 100644 index 0000000..3575294 --- /dev/null +++ b/molecule/https/molecule.yml @@ -0,0 +1,35 @@ +--- +dependency: + name: galaxy +driver: + name: docker +lint: + name: yamllint +platforms: + - name: instance + image: ubuntu:16.04 + privileged: true + exposed_ports: + - 8080/tcp + published_ports: + - 0.0.0.0:8080:8080/tcp + env: + JENKINS_HOME: /jenkins +provisioner: + name: ansible + log: true + lint: + name: ansible-lint + options: + # E602: Don't compare to empty string + # All workarounds for this are uglier than just comparing to empty strings. See: + # https://github.com/ansible/ansible-lint/issues/457 + x: ['602'] +verifier: + name: testinfra + env: + # Instruct the python-jenkins library to ignore SSL verification errors, which are + # caused by the self-signed certificate. + PYTHONHTTPSVERIFY: "0" + lint: + name: flake8 diff --git a/molecule/https/playbook.yml b/molecule/https/playbook.yml new file mode 100644 index 0000000..c29d0ff --- /dev/null +++ b/molecule/https/playbook.yml @@ -0,0 +1,19 @@ +--- +- name: Converge + hosts: all + vars: + jenkins_auth: "none" + jenkins_config_owner: "jenkins" + jenkins_config_group: "jenkins" + jenkins_home: "/jenkins" + jenkins_https_keystore: "{{ playbook_dir }}/ssl/test-cert.jks" + # NOTE: For testing purposes, we are using a self-signed certificate with the password + # of "password". You are of course advised to store such data in an Ansible vault. + jenkins_https_keystore_password: "password" + jenkins_https_validate_certs: false + jenkins_install_via: "apt" + jenkins_port: "-1" + jenkins_port_https: "8080" + jenkins_version: "2.190.2" + roles: + - ansible-jenkins diff --git a/molecule/https/ssl/test-cert.jks b/molecule/https/ssl/test-cert.jks new file mode 100644 index 0000000..9fe18e9 Binary files /dev/null and b/molecule/https/ssl/test-cert.jks differ diff --git a/molecule/https/tests/.gitignore b/molecule/https/tests/.gitignore new file mode 100644 index 0000000..c18dd8d --- /dev/null +++ b/molecule/https/tests/.gitignore @@ -0,0 +1 @@ +__pycache__/ diff --git a/molecule/https/tests/test_default.py b/molecule/https/tests/test_default.py new file mode 100644 index 0000000..2649d25 --- /dev/null +++ b/molecule/https/tests/test_default.py @@ -0,0 +1,30 @@ +import os + +import testinfra.utils.ansible_runner + +from jenkins import Jenkins + + +testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner( + os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('all') + + +def test_jenkins_installed(host): + package = host.package('jenkins') + + assert package.is_installed + + +def test_jenkins_version(): + master = Jenkins('https://127.0.0.1:8080') + version = master.get_version() + + assert version == '2.190.2' + + +def test_jenkins_java_process(host): + process = host.process.get(command='/usr/bin/java') + + assert '-Djenkins.install.runSetupWizard=false' in process.args + assert '--httpsKeyStore=/jenkins/secrets/test-cert.jks' in process.args + assert '--httpsKeyStorePassword=password' in process.args diff --git a/tasks/configure-config.yml b/tasks/configure-config.yml deleted file mode 100644 index f3f1da4..0000000 --- a/tasks/configure-config.yml +++ /dev/null @@ -1,57 +0,0 @@ ---- -- name: Set JENKINS_HOME - lineinfile: - create: true - dest: "/etc/default/jenkins" - line: "JENKINS_HOME={{ jenkins_home }}" - regexp: '^JENKINS_HOME=' - state: present - -- name: Set Jenkins port - lineinfile: - dest: /etc/default/jenkins - regexp: '^HTTP_PORT=' - line: "HTTP_PORT={{ jenkins_port }}" - -- name: Set Jenkins Java command line options - lineinfile: - dest: /etc/default/jenkins - regexp: '^JAVA_ARGS=' - line: "JAVA_ARGS=\"{{ jenkins_java_opts }}\"" - -- name: Copy JKS keystore credentials - copy: - src: "{{ jenkins_https_keystore }}" - dest: "{{ jenkins_home }}" - when: jenkins_https_keystore and jenkins_https_keystore_password - -- name: Copy CA signed certificate - copy: - src: "{{ jenkins_https_certificate }}" - dest: "{{ jenkins_home }}" - when: jenkins_https_certificate and jenkins_https_private_key - -- name: Initialize HTTPS credentials fact - set_fact: - jenkins_https_creds: "" - -- name: Set JKS keystore credentials - set_fact: - jenkins_https_creds: > - --httpsKeyStore='{{ jenkins_home }}/{{ jenkins_https_keystore }}' \ - --httpsKeyStorePassword='{{ jenkins_https_keystore_password }}' - when: jenkins_https_keystore and jenkins_https_keystore_password - -- name: Set CA signed certificate credentials - set_fact: - jenkins_https_creds: > - --httpsCertificate='{{ jenkins_home }}/{{ jenkins_https_certificate }}' \ - --httpsPrivateKey='{{ jenkins_https_private_key }}' - when: jenkins_https_certificate and jenkins_https_private_key - -- name: Set Jenkins command line options - lineinfile: - dest: /etc/default/jenkins - regexp: '^JENKINS_ARGS=' - line: "JENKINS_ARGS=\"--webroot=/var/cache/$NAME/war --httpPort={{ jenkins_port }} \ - --httpsPort={{ jenkins_port_https }} {{ jenkins_https_creds }}\"" diff --git a/tasks/configure-jenkins.yml b/tasks/configure-jenkins.yml index 506dc14..09d3bb1 100644 --- a/tasks/configure-jenkins.yml +++ b/tasks/configure-jenkins.yml @@ -3,6 +3,37 @@ # otherwise some data might be overwritten when it restarts. - include_tasks: "{{ jenkins_install_via }}/stop.yml" +- name: Set JENKINS_HOME + lineinfile: + create: true + dest: "/etc/default/jenkins" + line: "JENKINS_HOME={{ jenkins_home }}" + regexp: '^JENKINS_HOME=' + state: present + when: jenkins_install_via != "docker" + +- name: Set Jenkins port for HTTP + lineinfile: + dest: /etc/default/jenkins + regexp: '^HTTP_PORT=' + line: "HTTP_PORT={{ jenkins_port }}" + when: jenkins_install_via != "docker" + +- name: Set Jenkins port for HTTPS + lineinfile: + dest: /etc/default/jenkins + regexp: '^HTTPS_PORT=' + insertafter: '^HTTP_PORT=' + line: "HTTPS_PORT={{ jenkins_port_https }}" + when: jenkins_install_via != "docker" + +- name: Set Jenkins Java command line options + lineinfile: + dest: /etc/default/jenkins + regexp: '^JAVA_ARGS=' + line: "JAVA_ARGS=\"{{ jenkins_java_opts }}\"" + when: jenkins_install_via != "docker" + - name: Ensure correct ownership of JENKINS_HOME directory file: path: "{{ jenkins_home }}" @@ -11,32 +42,88 @@ mode: 0755 state: directory -- name: Configuration file is up to date (config.xml) +- name: Ensure main configuration file is up to date template: src: "{{ jenkins_source_config_xml }}" dest: "{{ jenkins_home }}/config.xml" - mode: 0644 owner: "{{ jenkins_config_owner }}" group: "{{ jenkins_config_group }}" + mode: 0644 - name: Configure Jenkins location template: src: files/jenkins.model.JenkinsLocationConfiguration.xml.j2 dest: "{{ jenkins_home }}/jenkins.model.JenkinsLocationConfiguration.xml" - mode: 0644 owner: "{{ jenkins_config_owner }}" group: "{{ jenkins_config_group }}" + mode: 0644 + +- name: Initialize Jenkins secrets dir fact + set_fact: + jenkins_secrets: "{{ jenkins_home }}/secrets" -- name: secrets dir is up to date +- name: Copy secrets copy: src: "{{ jenkins_source_secrets }}" - dest: "{{ jenkins_home }}/secrets" + dest: "{{ jenkins_secrets }}" owner: "{{ jenkins_config_owner }}" group: "{{ jenkins_config_group }}" when: jenkins_include_secrets -- name: set permissions on secrets dir +- name: Ensure correct ownership of secrets directory file: - path: "{{ jenkins_home }}/secrets" + path: "{{ jenkins_secrets }}" + owner: "{{ jenkins_config_owner }}" + group: "{{ jenkins_config_group }}" mode: 0700 - when: jenkins_include_secrets + state: directory + +- name: Copy JKS keystore credentials + copy: + src: "{{ jenkins_https_keystore }}" + dest: "{{ jenkins_secrets }}/" + when: jenkins_https_keystore and jenkins_https_keystore_password + +- name: Copy CA signed certificate + copy: + src: "{{ jenkins_https_certificate }}" + dest: "{{ jenkins_secrets }}/" + when: jenkins_https_certificate and jenkins_https_private_key + +- name: Copy CA certificate private key + copy: + src: "{{ jenkins_https_private_key }}" + dest: "{{ jenkins_secrets }}/" + when: jenkins_https_certificate and jenkins_https_private_key + +- name: Initialize HTTPS credentials fact + set_fact: + jenkins_https_creds: "" + +- name: Set JKS keystore credentials + set_fact: + jenkins_https_creds: >- + --httpsKeyStore='{{ jenkins_secrets }}/{{ jenkins_https_keystore | basename }}' + --httpsKeyStorePassword='{{ jenkins_https_keystore_password }}' + when: jenkins_https_keystore and jenkins_https_keystore_password + +# This fact prevents the next task from going over >90 chars +- name: Set fact for HTTPS certificate file + set_fact: + jenkins_https_certificate_file: >- + {{ jenkins_secrets }}/{{ jenkins_https_certificate | basename }} + when: jenkins_https_certificate and jenkins_https_private_key + +- name: Set CA signed certificate credentials + set_fact: + jenkins_https_creds: >- + --httpsCertificate='{{ jenkins_https_certificate_file }}' + --httpsPrivateKey='{{ jenkins_secrets }}/{{ jenkins_https_private_key | basename }}' + when: jenkins_https_certificate and jenkins_https_private_key + +- name: Set Jenkins command line options + lineinfile: + dest: /etc/default/jenkins + regexp: '^JENKINS_ARGS=' + line: "JENKINS_ARGS=\"--webroot=/var/cache/$NAME/war --httpPort=$HTTP_PORT \ + --httpsPort=$HTTPS_PORT {{ jenkins_https_creds }}\"" diff --git a/tasks/main.yml b/tasks/main.yml index 59b6b3a..76b9f93 100644 --- a/tasks/main.yml +++ b/tasks/main.yml @@ -22,9 +22,6 @@ - include: "{{ jenkins_install_via }}/install.yml" -- include: "configure-config.yml" - when: jenkins_install_via != "docker" - - include: "configure-jenkins.yml" - include: "configure-files.yml"