After I tried setting up deploy-rs and it (and flakes) is kind of not very good for what I am doing with my computers yesterday, I found that I still want to simplify my deploy tooling to make it easier to ship updates of my Nix systems to their hosts.
I landed on a solution inspired by Xe
Iaso's hosts.toml
setup. See Deploying from my hosts.toml for how this file is created,
structured, and used in the morph commands
themselves.
This page outlines a very simple script which ingests that hosts.toml file and provides a handful of
options to make it easy for me to just specify hostnames and have the
system figure out which manifest they should apply to and what to do
with them.
We use toml and click and some builtins.
import toml
import click
import os
import socket
import subprocessShort help options are good imo, customize that.
CONTEXT_SETTINGS = dict(help_option_names=['-h', '--help'])deploy-targets will print out the
hostnames of all the hosts in the hosts.toml file suitable for using in arroyo-flood or so to interactively
pick hostnames to deploy to.
Usage: deploy-targets [OPTIONS]
print a list of all the hosts in the hosts.toml
Options:
-f FILENAME hosts.toml file path
-h, --help Show this message and exit.
@click.command(context_settings=CONTEXT_SETTINGS)
@click.option('-f', 'hosts_file',
envvar="HOSTS_TOML", type=click.File('r'),
help="hosts.toml file path",
default="/home/rrix/arroyo-nix/networks/hosts.toml")
def list_hosts(hosts_file):
"""
print a list of all the hosts in the hosts.toml
"""
network = toml.load(hosts_file)
for netname, host in get_all_hosts(network):
print(host)deploy does the thing. If you try to
deploy to a host which cannot be pinged, it will be skipped. This
doesn't read targetHost from the Morph
network file, but those are lined up for me. -f will override this behavior, this is useful
to me when I am bootstrapping a node.
Usage: deploy [OPTIONS] [HOSTS]...
build or deploy one or more hosts
Options:
-f FILENAME hosts.toml file path
--all deploy to all hosts in the manifest
--deploy / -b, --build choose whether to deploy or just build.
-a, --action TEXT choose the deploy action
-c, --confirm / -C, --no-confirm
ask before running morph commands
-h, --help Show this message and exit.
@click.command(context_settings=CONTEXT_SETTINGS)
@click.option('-f', 'hosts_file',
envvar="HOSTS_TOML",
help="hosts.toml file path",
type=click.File('r'),
default="/home/rrix/arroyo-nix/networks/hosts.toml")
@click.option('--all', 'deploy_all', is_flag=True, default=False,
help="deploy to all hosts in the manifest")
@click.option(' /-b', '--deploy/--build', 'do_deploy',
help="choose whether to deploy or just build.", is_flag=True, default=True)
@click.option('-a', '--action', 'deploy_action',
help="choose the deploy action", default="switch")
@click.option('-c/-C', '--confirm/--no-confirm', 'confirm',
help="ask before running morph commands", is_flag=True, default=False)
@click.option('-f', '--force/--no-force', 'force',
help="Don't skip unavailable hosts", is_flag=True, default=False)
@click.argument('hosts', nargs=-1)
def wrap(hosts_file, deploy_all, hosts, do_deploy, deploy_action, confirm, force):
"""
build or deploy one or more hosts
"""
network = toml.load(hosts_file)
if deploy_all:
hosts = [h for net,h in get_all_hosts(network)]
elif len(hosts) == 0:
hosts = (socket.gethostname(),)
subnets_to_deploy = get_pairs(network, hosts)
for network, host in subnets_to_deploy:
if do_deploy:
if force or host_available(host):
cmd = f"morph deploy --on={host} --passwd ~/arroyo-nix/networks/{network}.nix {deploy_action} --keep-result"
else:
click.echo(f"{host} is unavailable, skipping...")
continue
else:
cmd = f"morph build --on={host} ~/arroyo-nix/networks/{network}.nix --keep-result"
click.echo(f"Prepared to run '{cmd}'")
if confirm:
input("or hit ctrl-c... ")
os.system(cmd)
def host_available(hostname: str) -> bool:
proc = subprocess.run(f"ping -w 1 -c 1 {hostname}", shell=True, capture_output=True)
return proc.returncode == 0 And these ugly list-comprehensions help to munge the TOML file in to a form the python commands here would like to use.
def get_all_hosts(network):
return [
(name, host)
for name, net in network.items()
for host in net['hosts'].keys()
]
def get_pairs(network, hosts):
return [
(netname, hostname)
for netname, subnet in network.items()
for hostname in hosts
if hostname in subnet['hosts'].keys()
]This is made available to my system builds in my rixpkgs overlay, and added like this:
{ pkgs, ... }:
{
environment.systemPackages = [ pkgs.morph-wrapper ];
}So now when I make changes I can type deploy -b in to my nearest terminal to see my
local system's build come together, then deploy to deploy it to this machine, then deploy $hostnames to deploy it to any number of
my hostnames or deploy --all to deploy it
everywhere.
Shell Environment and Pyproject manifest
This uses poetry and poetry2nix to generate a python application and
nix derivation for use in my systems, a shell.nix is provided as well:
[tool.poetry]
name = "morph-wrapper"
version = "0.1.0"
description = ""
authors = ["Ryan Rix <code@whatthefuck.computer>"]
include = []
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
[tool.poetry.dependencies]
python = "^3.10"
toml = "^0.10.2"
click = "^8.1.3"
[tool.poetry.scripts]
deploy = 'morph_wrapper.wrapper:wrap'
deploy-targets = 'morph_wrapper.wrapper:list_hosts'{ pkgs ? import <nixpkgs> {} }:
let
python-with-my-packages = pkgs.python3.withPackages (p: with p; [
toml
click
]);
in
pkgs.mkShell {
packages = [
python-with-my-packages
pkgs.poetry
];
}{ pkgs ? import <nixpkgs> {},
poetry2nix ? pkgs.poetry2nix,
stdenv ? pkgs.stdenv,
python ? pkgs.python3 }:
poetry2nix.mkPoetryApplication {
inherit python;
projectDir = ./.;
propagatedBuildInputs = [];
}