Ansible Playbook for setting up a proxmox cluster with some debian and (LXC) container VMs, some tasmota IOT devices, and some Openwrt Access Points and Routers with VLAN separation of the SSIDs on each of the radios
A couple of Ansible playbooks which runs a series of configuration steps to set up an SOE based on Debian, in order to provide a solid foundation for subsequent actions.
It borrows heavily from the work of: Luke Harvey, Bryan Kennedy, Ryan Eschinger, Ashley Rich, and Digital Ocean
It will perform the following:
- Create a new ansible "super user" with with root privileges and public key authentication on your proxmox and debian machines as well as the containers
- Set root user credentials, and a primary user credentials, ssh keys, sudoers to root
- Implement several SSH hardening techniques
- Configure the timezone and enable time synchronization
- Modify the hostname and hosts file
- Install the admin's account with some nice .bashrc settings
- Install a package baseline appropriate for laptops, desktops, virtualisation hosts, containers, as appropriate
- Override some of debian's more annoying defaults
- Manage tasmota settings on your IOT devices
- Manage openwrt settings on your openwrt devices, including installing VLANs 10,30,40,70 and configuring radios on them
For physical boxes, VMs and LXC containers, I take the vanilla debian install where I've just clicked through the install ISO steps in a fairly braindead fashion (our playbooks goes and fixes a bunch of things asked in install anyway), apply bootstrap.yml first, then initial_server_config.yml to bring the configuration up to my SOE and upon any further update to my config or after the machine is patched. I don't automate the entire playbook being played because I still have far too many pets and manual changes that need checking, and the playbook is quite slow to run all the way through, so I'm frequently using --diff, --check and --tags to limit the scope of changes to what I'm currently concerned with. It would be easy to email myself an output of --check --diff initial_server_config.yml every night, but I get enough email as-is.
For OpenWRT APs and tasmota configurations however, we still rely on some manual configuration for a new device from scratch before applying these playbooks - but it does make it easier to wholesale change all of your SSIDs for example. For upgrades however, we can automate all of the tasks needed to bring your device back into compliance with your config without any manual configuration. I apply it routinely after any minor upgrade of the openwrt device, perhaps checking with --diff --check first if I'm a little nervous. I do test on my VMs first (sometimes I'm even sensible enough to test on one of my virtualised APs before I test it on my internet gateway), and take a snapshot manually beforehand. For major upgrades, it hasn't broken majorly for me yet, but my planning tends to be a little more careful around these events. For setting up a new openwrt AP, my manual configuration tends to be limited to setting switch vlan tagging information in /etc/config/network, and assigning radio0 and radio1 consistently with my other devices, before letting uci_config.yml loose on VLAN and network definitions and what SSIDs we've assigned to which radios. This sometimes involves swapping radio0 and radio1 pci/hardware devices in /etc/config/wireless.
- Ansible installed locally on your machine
- You probably want to install
ansible-mitogen(andpython3-mitogen) on your ansible server too, for my ansible.cfg setsstrategy = mitogen_linearto greatly accelerate the playbook (it works with that setting disabled if you can't install migoten, but mitogen has never created any detectable problems for me). I have only tested this from a debian machine (debian 11,12). - Ideally, you'd create a gpg encrypted file in misc/vault-password.gpg, and verify it can be read with: misc/get-vault-pass.sh
- Openwrt plays rely on ansible-openwrt, which is published as a galaxy collection.
- Tasmota plays rely on ansible-tasmota, which is available through ansible galaxy, but which I've modified to allow for and transparently recovers from the tasmota device spontaneously rebooting after certain configurations are applied.
Clone the repo
$ git clone --recurse-submodules https://github.com/spacelama/ansible-initial-server-setup.git
Modify the variables in vars/main.yml according to your needs:
user: the username for your new "super user"
password: a hashed sudo password
my_public_key: the local path to your public SSH key that will be authorized on all remote hosts
domain: your chosen domain
hostname: your chosen hostname
timezone: the most appropriate timezone for your server
ssh_port: your chosen SSH port
Modify hosts.yml with your various host settings
I put a bunch of vault encoded per-host secrets in
host_vars/<hostname>.yml too, encrypted via ansible-vault encrypt_string, eg settings such as ssh_host_rsa_key,
ssh_host_ed25519_key, ssh_host_ecdsa_key, ssh_local_port,
switch_pass, root_id_rsa - host_vars/ being in .gitignore to
further protect their contents (likewise for files/main.*.password
and files/ap.* which should have been vault entries in the first
place). Host settings I want to track in git are in hosts.yml.
Install the ansible_adm account and the sudo permissions for this account to escalate to root with:
$ ansible-playbook bootstrap.yml -u root -k --extra-vars "target=dirac-new" --ask-vault-pass # always run with --check when first starting out!
Just to renew ssh hostkeys etc, without having to first turn on ssh PermitRootLogin:
$ ansible-playbook bootstrap.yml -u ansible --extra-vars "target=dirac-new" --ask-vault-pass --become # always run with --check when first starting out!
Fix up an old installation:
$ ansible-playbook bootstrap.yml -u tconnors -k --extra-vars "target=maxwell" --ask-vault-pass --become --become-method=su -K # always run with --check when first starting out!
Fix up an lxc container:
$ ansible-playbook bootstrap.yml -u root --diff --extra-vars "target=zm" # always run with --check when first starting out!
It's not foolproof, but try --check prior to each real ansible.
--diff is extremely handy, but not foolproof when also running --check.
I frequently --limit to hosts or away from hosts.
$ ansible-playbook --ask-vault-pass initial_server_setup.yml --diff --check --limit='!dirac-new,!fs-new,!hass-debian,!mail'
--limit also useful when you get a new openwrt AP or tasmota device:
$ ansible-playbook -v openwrt_maintenance.yml --diff --check
$ ansible-playbook tasmota_maintenance.yml --diff --check --limit patiofluro-power,loungefrontlight-power --extra-vars "setpsk=true" --extra-vars "setsyslog=true"
Then run the playbooks:
$ ansible-playbook --ask-vault-pass initial_server_setup.yml --diff --limit='!dirac-new,!fs-new,!hass-debian,!mail'
Likewise for tasmota and openwrt:
$ ansible-playbook -v openwrt_maintenance.yml --diff
$ ansible-playbook tasmota_maintenance.yml --diff --limit patiofluro-power,loungefrontlight-power --extra-vars "setpsk=true" --extra-vars "setsyslog=true"
initial_server_setup.yml uses tags on each role.
Limit your changes to only apply webserver and smtp roles with eg:
$ ansible-playbook --diff initial_server_setup.yml --limit met,webserver,iot --tags webserver,smtp --check
or to stop the webserver role from running while still running everything else:
$ ansible-playbook --diff initial_server_setup.yml --limit met,webserver,iot --skip-tags webserver --check
vars/openwrt.yml contains some settings for all your openwrt devices
(routers, APs etc), and sets up a bunch of VLANs for your IOT devices,
windows devices etc, assigned per MAC address (VLAN decided by which
SSID your device joins - my IOT devices from China only know about my
IOT SSID, and some of them get a firewall entry that stops them even
talking to the internet, let alone amongst themselves; sorry, firewall
was done through point-and-click, not yet encoded here). DHCP
reservations set in roles/openwrt/templates/dhcp.* and static
hostnames for serving static RR A records.
You'll need to set up files/ap.mobility_domain files/ap.wpa2.{default_radio0.psk,default_radio1.psk,wifinet{4,5,6,7,10,11}.{psk,ssid}}
to contain values for your PSK etc. hosts.yml knows about some of the
network ssids, so ap.wpa2.wifinet10.ssid and
ap.wpa2.wifinet11.ssid aren't needed or consulted.
My router required a bunch of manual config (upstream VLANs, firewalls, banip etc), but I've been using this to configure fresh APs from scratch. Have a good backup of your APs before you run this for the first time though if you've already set them up in any way. The radio stuff is expected to be quite fragile, and has only received most testing on current openwrt 22.03. and 23.05.*
My inventory is in hosts.yml, and tells us whether the openwrt device
uses DSA switch config or the old definition, via
openwrt_dsa_switch_config. IP address are decided by
inet_addr_suffix in your inventory to assign
192.168.{{interface}}.{{inet_addr_suffix}} (where interface is
decided by uci_config.yml per VLAN). We might set
openwrt_heavy_installation: false for devices with particularly
small flash (but I was able to NFS mount a fileserver to manually
offload the biggest of non-essential packages from even my smallest
wavlink wn575a3 with 8MB of flash). type is 'ap' or 'router' and
decided which packages to install and how to set up DHCP.
Run the playbook to configure all openwrt devices configured in hosts.yml:
$ ansible-playbook openwrt_maintenance.yml --diff # --check to verify changes first
NTP, latitude, longitude, syslog, timezones, SSIDs are encoded in vars/tasmota.yml. You'll need to tweak these.
This will set SSID1 and you might choose fallback SSID2 per host in your hosts.yml inventory - here, we set the first one to be your primary SSID that you mesh or roam between throughout your host, and SSID2 might be the second closest AP to where your device normally sits. That way, tasmota will lock onto (and roam via 80211.r) your closest AP on the primary SSID by default, but if that AP continues to serve valid wifi connections but loses connectivity to the network itself, tasmota's watchdog will notice this loss of packet connectivity, and will failover to the second closest AP that is hopefully still on a working network.
It will configure all mqtt paths to be a single standard (I don't know much about mqtt, but my network and home assistant seem happy with my current settings).
I set disable_default_reset_on_power_reset7=1 on devices that
frequently lose their power, so they don't accidentally get firmware
reset.
The ansible tasmota provider thinks PSK and syslog change every time you try to adjust them, even if actually unchanged, so by default, I don't set them. They only get attempted to be set when you supply setpsk=true and setsyslog=true. But since writing that, I've set a lot more parameters, some of which unconditionally overwrite the setting even when unchanged, and some of these result in the tasmota device rebooting every time. I've had to put a workaround in ansible-tasmota to allow the device to recover and continue setting subsequent parameters.
When making a mass change after testing something well, I'll leave off --limit, but when configuring a new device, I'll use this:
ansible-playbook tasmota_maintenance.yml --diff --extra-vars "setpsk=true" --extra-vars "setsyslog=true" --limit airmon1