Code development platform for open source projects from the European Union institutions :large_blue_circle: EU Login authentication by SMS has been phased out. To see alternatives please check here

Skip to content
Snippets Groups Projects
Commit 82b7b4b7 authored by Vehbo DZOGOVIC's avatar Vehbo DZOGOVIC
Browse files

add jsonschema

parent af44df23
No related branches found
No related tags found
1 merge request!11Add ansible execution environment
Pipeline #131898 passed
StylesPath = styles
MinAlertLevel = suggestion
[*]
BasedOnStyles = Vale
# Collections
Collections are used in ansible as "packages" which can be shared across multiple projects. Example: community.hashi_vault
Those collections will be installed automatically via awx or the gitlab runner, by adding a requirements.yml file inside the root directory collections in your project.
Example file:
```
# collections/requirements.yml
collections:
- name: community.docker
version: 3.4.8
......@@ -18,3 +27,4 @@ collections:
version: ">=2.11.0"
- name: awx.awx
version: ">=22.7.0"
```
# Linter
For ansible, we have a linter package inside gitlab, which *MUST* be used.
To configure it, put the following content inside your .gitlab-ci.yml
```
default:
tags:
- lab
- shell
stages:
- default_validate
include:
- project: 'digit-c4/digitC4-template-cicd'
file: 'gitlab-ci-ansible.yml'
ref: main
```
The merge requests should be declined if the linter is failling.
# Ansible
## Getting started
Ansible is an automation tool, which will be used to deploy the configuration on all virtual/dedicated machines
##### Principles
- Strive for simplification in what you automate
- If done properly, it can be the documentation of your workflow automation
Ansible is a desired state engine by design, if you are trying to “write code” in your plays and roles, you are setting yourself up for failure. (“Ansible Best Practices: The Essentials”) The YAML-based playbooks were never meant to be for programming.
Version control your content
Start as simple as possible and iterate
Start with a basic playbook and static inventory
Refactor and modularize later
Use our style guide for any content
Be consistent in
- Tagging
- Whitespace
- Naming of tasks, Plays, Variables and Roles
- Directory Layout
- Avoid using commands/shell modules, try to use real modules or if it does not exist, do it yourself
The ansible roles and playbooks structure will be split in multiple parts instead of having multiple big playbooks and roles.
##### Roles
For each software there will be role. For example, there will be a role specific to yum/apt (yum or apt configuration), nginx, iptables/nftables, apache, and so on. There will no longer be a big role which contains anything inside.
Inside a role folder there should be this structure (we take nginx as example):
```
roles/
nginx/
vars/
main.yml <--- this file should contain the package version and so on
templates/
files/
handlers/
main.yml <--- every handler should be here inside
tasks/
main.yml <--- This file should include all other files inside tasks
package.yml <--- Install the package needed
nginx.yml <--- Nginx configuration
....
To create the directory structure, you should use ansible-galaxy.
$ ansible-galaxy init nginx
```
Each of the roles should be developed in a generic way so that all variables and all needed steps a retrieved from the CMDB/Dynamic Inventory instead of adding host exception inside the roles.
##### Playbook
Inside the playbook directory we need to have two different kinds. Once a unique playbook for each role and once an application playbook for each application. This will be separated in two different folders:
- application
- software
For example, in the case of nginx, there will be a playbook nginx.yml which will use the role nginx and run on the hosts inside the group nginx.
For application let's say there is an application with a name testing which need nginx and mysql on the same hosts. Then there will be a playbook named testing, which will call the role nginx and mysql.
You should think about ansible playbooks and roles as a code. You should use a version control system like git or svn and have a linter running on it to verify the syntax code. (ansible-lint is your friend).
##### Variables
Variables should always contain a prefix, which is the role name and also keep your variables in lowercase and use clear names.
Example:
```
nginx_default_port: 80
nginx_keepalive: 25
```
Don't repeat yourself in the same playbook, but try to create a variable which will be used everywhere.
Example:
```
### bad example
- name: Create home directory
file:
path: /home/user
state: directory
- name: Create sub directory
file:
path: /home/user/.ssh
state: directory
### good example
vars:
user_home_dir: "/home/{{ username }}"
user_home_ssh_dir: "{{ user_home_dir }}/.ssh"
tasks:
- name: Create home directory
file:
path: "{{ user_home_dir }}"
state: directory
- name: Create sub directory
file:
path: "{{ user_home_ssh_dir }}"
state: directory
```
##### Generic development of playbooks/roles
In a generic development way, every task should be configurable via variables, and it should be to avoid having a host specific task inside a role or playbook.
To get this done correctly, we will need some dynamic inventories and a CMDB to store every relation and configuration.\
\
How it should look:\
```
# nginx as example
# variable file (dynamic inventory should generate this)
nginx_configuration:
configuration_dir: /etc/nginx/conf.d
www_base_dir: /var/www
applications:
- name: application1
working_dir: "{{ nginx_configuration.base_dir }}/{{ item.name }}" <--- this will be run inside a loop
http_port: 80
https_port: 443
- name: application2
working_dir: "{{ nginx_configuration.base_dir }}/{{ item.name }}"
http_port: 81
https_port: 8443
# The tasks inside the role
- name: Deploy the nginx configuration
template:
src: etc_nginx_conf.d_example.conf.j2
dest: "{{ nginx_configuration.configuration_base_dir }}"
owner: nginx
group: nginx
loop:
- {{ nginx_configuration.applications }}
And then inside the template file, you should prefix item.port and so on.
```
Every Template should contain the following line at the start, so that every deployed file show that it is managed by ansible.
```
{{ansible_managed | comment}}
```
##### Module development
For the module development, we should always use python as the language, and we need to do it like every other module (CRUD principe, Create, read, update, delete).
Also, every module should return a diff version and implement a check option. This will allow us to verify the change before deploying it.
Ansible project used to deploy digit C4 configuration
......@@ -60,6 +60,7 @@ dependencies:
receptorctl
hvac
pynetbox
jsonschema
additional_build_steps:
append_base:
- RUN $PYCMD -m pip install -U pip
......
- name: apt upgrade
hosts: all
tasks:
- name: upgrade machine
ansible.builtin.apt:
update_cache: yes
- name: add dns entry
hosts: cluster_router-internal
gather_facts: true
become: true
roles:
- bind
---
version: 3
build_arg_defaults:
ANSIBLE_GALAXY_CLI_COLLECTION_OPTS: '--pre'
dependencies:
ansible_core:
package_pip: ansible-core==2.14.4
ansible_runner:
package_pip: ansible-runner
galaxy: requirements.yml
python:
- six
- psutil
- hvac
# system: bindep.txt
images:
base_image:
name: docker.io/redhat/ubi9:latest
# Other available base images:
# - quay.io/rockylinux/rockylinux:9
# - quay.io/centos/centos:stream9
# - registry.fedoraproject.org/fedora:38
# - registry.redhat.io/ansible-automation-platform-23/ee-minimal-rhel8:latest
# (needs an account)
# Custom package manager path for the RHEL based images
# options:
# package_manager_path: /usr/bin/microdnf
#additional_build_files:
# - src: files/ansible.cfg
# dest: configs
additional_build_steps:
prepend_base:
- RUN echo This is a prepend base command!
# Enable Non-default stream before packages provided by it can be installed. (optional)
# - RUN $PKGMGR module enable postgresql:15 -y
# - RUN $PKGMGR install -y postgresql
# prepend_galaxy:
# - COPY _build/configs/ansible.cfg /etc/ansible/ansible.cfg
prepend_final: |
RUN whoami
RUN cat /etc/os-release
append_final:
- RUN echo This is a post-install command!
- RUN ls -la /etc
- name: Hello World Sample
hosts: all
tasks:
- name: Hello Message
debug:
msg: "Hello World!"
- name: Ping
hosts: all
tasks:
- name: ping
ansible.builtin.ping:
- name: restart bind9
community.docker.docker_container:
name: bind9
state: started
restart: true
ignore_errors: "{{ ansible_check_mode }}"
- name: read netbox records data
ansible.builtin.uri:
url: "http://{{ lookup('ansible.builtin.env', 'EXTERNAL_GATEWAY') }}/snet/netbox/api/plugins/netbox-dns/records/?zone=ec.local&type__n=NS&type__n=SOA"
method: GET
headers:
Authorization: "Token {{ lookup('ansible.builtin.env', 'NETBOX_TOKEN') }}"
register: netbox_dns_data
- name: deploy bind9 zone file
template:
src: ec.local.j2
dest: /opt/snet/bind9/ec.local
notify: restart bind9
; BIND db file for ec.local
$TTL 86400
@ IN SOA . admin.ec.local. (
{{ ansible_date_time.epoch }} ; serial number YYMMDDNN
28800 ; Refresh
7200 ; Retry
864000 ; Expire
86400 ; Min TTL
)
NS {{ lookup('ansible.builtin.env', 'INTERN_GATEWAY') }}.
$ORIGIN ec.local.
{% for entry in netbox_dns_data.json.results %}
{{ entry.name }} {{ entry.ttl | default(86400) }} {{ entry.type }} {{ entry.value }}
{% endfor %}
- name: get the uptime
hosts: all
gather_facts: false
tasks:
- name: get the output from uptime
ansible.builtin.shell: date
register: uptime
- debug:
var: uptime
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment