improve UID management, hopefully

Reviewed-on: #1

README.md

@@ -10,12 +10,12 @@ It *was* the revamped DNS service for `zeke.ecotroph.net`, and thus served zones
 
 ## Overview
 
-In the past, we just ran the version of BIND that came with our distribution (at this moment, that is CentOS 7, which translates to bind 9.11.)  This new configuration runs a very recent version of BIND 9 via a docker image produced by ISC themselves.  We started with 9.18.12 and now are up to 9.18.20.
+In the past, we just ran the version of BIND that came with our distribution (at this moment, that is CentOS 7, which translates to bind 9.11.)  This new configuration runs a very recent version of BIND 9 via a docker image produced by ISC themselves.  We started with 9.18.12 and now are up to 9.18.43.
 
 This docker image imposes a few requirements:
 
-* Internally, the image runs `named` as the `bind` user (104:105).  Since we bind-mount directories, we do need those directories owned by whatever internal UID it is using.
-* We need some way to ensure that our container is run on system reboots, etc.  Here we chose to use `systemd` to do this, although that is not ideal.
+* Internally, the image runs `named` as the `bind` user (53:53).  Since we bind-mount directories, we do need those directories owned by whatever internal UID it is using.
+* We need some way to ensure that our container is run on system reboots, etc.  Here we chose to use `systemd` to do this.
 * Presumably the normal way to do logging for a docker container is to use the standard journal service, although this image is set up to bind-mount `/var/log`.  On the other hand, the standard command uses the `-g` flag, which is "debug" mode, and causes all of the logs to go to stderr.
 * We do want named to stay in the foreground here.  Fortunately, there have always been command line options that do this (`-g` and `-f`).  Thus, in order to log to `/var/log`, we supply a different command: `/usr/sbin/named -f -u bind`.  This will run in the foreground, and run as the internal `bind` user.
 
@@ -82,3 +82,7 @@ We can find the zone files  in `/etc/bind/zones`, although note that your zone m
 ```bash
 named-compilezone -f raw -F text -o - blacka.com /etc/bind/zones/blacka.com.signed
 ```
+
+If using the script that runs the version in our container, note that you may have to use the paths that work *inside the container*.  The current script mounts your current working directory, so you can use `run_named_compilezone.sh ./some.zone`, but not `run_named_compilezone.sh /etc/named/zones/blacka.com`.
+
+That said, we are probably better off just using the version that comes with our OS, and not using the container.

README_TROUBLESHOOTING.md

@@ -0,0 +1,42 @@
+# Troubleshooting docker.named.service
+
+I'm writing this so hopefully the next time I touch this serivce, I can remember how to actually troubleshoot this service, and what the common pitfalls are.
+
+## Docker-based setup
+
+I was in the *very slow* process of switching `diagonal.blacka.com` from using `docker` to using `podman` and `podman-systemd.unit` when I last touched this.  Let's assume we are still on docker.
+
+It took me a while to remember how this service is run.  It is run as simple systemd unit. No docker-compose, or anything.  But all of the docker run option are in a shell script.
+
+The unit file is in /etc/systemd/system/docker.named.service
+
+Commands to stop/start, see the status
+
+```bash
+sudo systemctl status docker.named.service
+sudo systemctl start docker.named.service
+sudo systemctl stop docker.named.service
+sudo journalctl -u docker.named.service
+```
+
+The config install is all in `/etc/bind` -- this is mounted into the container in different spots, but we have it all together, here.
+
+Some pitfalls for working on this:
+
+1. the TSIG keys are encrypted by git-crypt.  There is a filter on the source repo that encrypts the files on update.  This tool exists entirely to have these files unreadable, but checked in, in git.  When you check out or pull from the repo, you will need to unlock the repo -- this will decrypt all of the encrypted files.
+    * The main README talks about how to get the key, but it is in my 1password, although I probably already have it downloaded.  I expect it just be sitting in ~/src/docker_bind on diagonal itself.
+    * The main point is *check to see that files are not encrypted* before copying the updated config into place.
+2. The ISC-provided docker image seems to change the internal UIDs on a periodic basis.  This is a critical detail for running this, so I wonder how other users deal with this.
+    * There is code in `setup.sh` and `setup_docker.sh` to *create* the bind user and group, but it doesn't yet *correct* the UIDs when they change
+    * In any case, there is some code in there to fetch the UID and GID out of the container, which also gives you a hint on how to run the container and get a shell.
+
+In general, if the container doesn't start, we need to just try and run it in debug mode in the container.  The way I did this last:
+
+```bash
+export IMAGE=docker.io/internetsystemsconsortium/bind9:9.18
+docker run -ti  --rm --entrypoint=/bin/sh -v /etc/bind/cfg:/etc/bind -v /etc/bind/cache:/var/cache/bind  -v /etc/bind/zones:/var/lib/bind -v /etc/bind/log:/var/log "$IMAGE"
+# now inside the image
+/usr/sbin/named -u bind -g -c /etc/bind/named.conf
+```
+
+Then look at the logging.  There is a lot of noise (which we should probably investigate), but there should be a real error in there.

cfg/named.keys.conf

@@ -1,4 +1 @@
 include "/etc/bind/keys/zeke-tornado.keys";
-include "/etc/bind/keys/zeke-ogud.keys";
-include "/etc/bind/keys/named_hxr_us.keys";
-include "/etc/bind/keys/named_knitbot_org.keys";

cfg/named.options.conf

@@ -6,8 +6,6 @@ options {
     lock-file "/var/cache/bind/named.lock";
 
     listen-on { any; };    
-    listen-on-v6 { ::1; };
+    listen-on-v6 { any; };
     allow-recursion { 127.0.0.1; ::1; };
-
-	dnssec-validation yes;
 };

cfg/named.primary.conf

@@ -6,17 +6,13 @@ zone "blacka.com" {
 
     notify yes;
     also-notify {
-        45.79.181.51;   // typhoon.kahlerlarson.org
-        104.225.12.28;  // keilir.ogud.com
-        66.92.146.115;  // foss.ogud.com
-        204.109.61.194; // katla.ogud.com
+        45.79.181.51;   // typhoon.kahlerlarson.org, ns1.kahlerlarson.org
+        45.79.191.213;  // ns2.kahlerlarson.org
     };
     allow-transfer {
         127.0.0.1;
         45.79.181.51;  // typhoon.kahlerlarson.lorg
-        104.225.12.28;  // keilir.ogud.com
+        45.79.181.213; // ns2.kahlerlarson.org
         key zeke-tornado.;
-        // key zeke-ogud2.;
-        key zeke-ogud3;
     };
 };

cfg/named.secondary.conf

@@ -1,8 +1,15 @@
-zone "ogud.com" {
-    type secondary;
-    file "/var/lib/bind/secondary/ogud.com";
-    masters {
-        104.225.12.28;
-        #204.109.61.194;
-    };
-};
+
+# talk to Matt about secondarying his zone, since he is double secondarying mine
+# but at the moment, he does not allow me to zone transfer.
+
+# zone "kahlerlarson.org" {
+#     type secondary;
+#     file "/var/lib/bind/secondary/kahlerlarson.org";
+#     masters {
+#         45.79.181.51;
+#         45.79.191.213;
+#     };
+#     allow-transfer {
+#         key zeke-tornado.;
+#     };
+# };

docker.bind.service

@@ -13,4 +13,4 @@ ExecStartPre=-/usr/bin/docker rm bind9
 ExecStart=/etc/bind/run_bind_container.sh
 
 [Install]
-WantedBy=multi-user.target
+WantedBy=multi-user.target

named.container

@@ -0,0 +1,14 @@
+[Container]
+Image=docker.io/internetsystemsconsortium/bind9:9.18
+ContainerName=bind9
+Network=host
+Volume=/etc/bind/cfg:/etc/bind
+Volume=/etc/bind/cache:/var/cache/bind
+Volume=/etc/bind/zones:/var/lib/bind
+Volume=/etc/bind/log:/var/log
+# note that the default command is '-f -c /etc/bind/named.conf -L /var/log/bind/default.log'
+# this is close, but not quite what we want
+Exec=-f -c /etc/bind/named.conf
+
+[Install]
+WantedBy=default.target

run_bind_container.sh

@@ -1,11 +1,25 @@
 #! /bin/bash
-BASE_CONF_DIR=/etc/bind
 
-CMD="/etc/bind/run.sh"
-[ "$1" = "interactive" ] && ARGS="-ti --entrypoint=/bin/bash" && CMD=""
+# This is not normally how this container is launched.  Instead, see
+# named.container, which allows podman-systemd to construct a systemd
+# unit files and run using podman.
+#
+# Instead, this script can be used to launch the container "by hand".
+
+BASE_CONF_DIR=/etc/bind
+CMD="-f -c /etc/bind/named.conf"
+[ "$1" = "interactive" ] && ARGS="-ti --entrypoint=/bin/sh" && CMD=""
+
+[ -x /usr/bin/docker ] && DOCKER=/usr/bin/docker
+[ -x /usr/bin/podman ] && DOCKER=/usr/bin/podman
+
+# Note that as of 2024-09-01, this image is based on Alpine linux and its entrypoint is:
+#    '/usr/sbin/named -u bind'
+# and the default command is:
+#    '-f -c /etc/bind/named.conf -L /var/log/bind/default.log'
 
 # shellcheck disable=SC2086
-docker run $ARGS \
+$DOCKER run $ARGS \
     --rm \
     --name=bind9 \
     --network=host \
@@ -13,20 +27,4 @@ docker run $ARGS \
     -v $BASE_CONF_DIR/cache:/var/cache/bind \
     -v $BASE_CONF_DIR/zones:/var/lib/bind \
     -v $BASE_CONF_DIR/log:/var/log \
-    docker.io/internetsystemsconsortium/bind9:9.18 $CMD
-
-# using bridge networking
-# : "${DNS_PORT:=53}"
-# : "${RNDC_PORT:=953}"
-# docker run $ARGS \
-#     --rm \
-#     --name=bind9 \
-#     --add-host=host.docker.internal:host-gateway \
-#     --publish "$RNDC_PORT:953/tcp" \
-#     --publish "$DNS_PORT:53/udp" \
-#     --publish "$DNS_PORT:53/tcp" \
-#     -v $BASE_CONF_DIR/cfg:/etc/bind \
-#     -v $BASE_CONF_DIR/cache:/var/cache/bind \
-#     -v $BASE_CONF_DIR/zones:/var/lib/bind \
-#     -v $BASE_CONF_DIR/log:/var/log \
-#     docker.io/internetsystemsconsortium/bind9:9.18 $CMD
+    docker.io/internetsystemsconsortium/bind9:9.18 "$CMD"

run_bind_podman_mac.sh

@@ -0,0 +1,19 @@
+#! /bin/bash
+
+BASE_CONF_DIR=/Users/davidb/src/docker_bind
+: "${DNS_PORT:=1053}"
+: "${RNDC_PORT:=1953}"
+CMD="/usr/sbin/named -f -4 -u davidb"
+[ "$1" = "interactive" ] && ARGS="-ti --entrypoint=/bin/bash" && CMD=""
+podman run $ARGS \
+    --rm \
+    --arch=amd64 \
+    --name=bind9 \
+    --publish $RNDC_PORT:953/tcp \
+    --publish $DNS_PORT:53/udp \
+    --publish $DNS_PORT:53/tcp \
+    -v $BASE_CONF_DIR/cfg:/etc/bind \
+    -v $BASE_CONF_DIR/cache:/var/cache/bind \
+    -v $BASE_CONF_DIR/zones:/var/lib/bind \
+    -v $BASE_CONF_DIR/log:/var/log \
+    localhost/blacka/bind9:9.18 $CMD

run_named-compilezone.sh

@@ -1,4 +1,8 @@
 #! /bin/bash
 [ -x "$(which docker 2>/dev/null)" ] && DOCKER=docker
 [ -x "$(which podman 2>/dev/null)" ] && DOCKER=podman
-$DOCKER exec bind9 named-compilezone "$@"
+
+$DOCKER run -ti --rm --name=bind9-tools --network=host \
+    -v "$(pwd):$(pwd)" -w "$(pwd)" \
+    docker.io/internetsystemsconsortium/bind9:9.18 named-compilezone "$@"
+

setup.sh

@@ -1,9 +1,40 @@
 #! /bin/bash
 
-if [ "$EUID" -ne 0 ]; then
-    echo "Must run as root"
+set -e
+
+# NOTE: groupadd and useradd require root.
+[ "$EUID" -ne 0 ] &&  echo "Must be run by root" && exit 1
+
+IMAGE="docker.io/internetsystemsconsortium/bind9:9.18"
+
+# determine current uid and gid
+uidgid=$(podman run --rm --entrypoint=/bin/sh "$IMAGE" -c "/usr/bin/id -u bind; /usr/bin/id -g bind")
+read -d '' -r uid gid <<< "$uidgid" || :
+
+# Create or update the on host user to match the container's 'bind' user and group
+ACTUAL_UID=$(id -u bind 2>/dev/null)
+ACTUAL_GID=$(id -g bind 2>/dev/null)
+
+[ -z "$ACTUAL_GID" ] && groupadd -f -g "$gid" bind
+[ -z "$ACTUAL_UID" ] && useradd -u "$uid" -g "$gid" -M --no-log-init bind
+[ "$ACTUAL_GID" -ne "$gid" ] && groupmod -g "$gid" bind
+[ "$ACTUAL_UID" -ne "$uid" ] && usermod -u "$uid" -g "$gid"
+
+# create/update our main directory setup
+install -d -o bind -g bind -m 0755 /etc/bind/cfg /etc/bind/cache /etc/bind/zones /etc/bind/log/named
+# copy over our config and data without overwriting anything, hopefully.
+rsync -av --chown bind:bind --del ./cfg/  /etc/bind/cfg/
+rsync -av --chown bind:bind ./zones /etc/bind/zones/
+rsync -av --chown bind:bind --ignore-existing ./cache/ /etc/bind/cache/
+
+# install our podman config
+if [ -d /etc/containers/systemd ]; then
+    install -o root -g root -m 0644 named.container /etc/containers/systemd/
+    systemctl daemon-reload
+    systemctl start named
+else
+    echo "containers-common not installed?"
     exit 1
 fi
 
-groupadd -f -g 105 bind
-useradd -u 104 -g 105 -M --no-log-init bind
+exit 0

setup_docker.sh

@@ -0,0 +1,36 @@
+#! /bin/bash
+
+set -e
+
+# NOTE: groupadd and useradd require root.
+[ "$EUID" -ne 0 ] &&  echo "Must be run by root" && exit 1
+
+IMAGE="docker.io/internetsystemsconsortium/bind9:9.18"
+
+# determine current uid and gid
+uidgid=$(docker run --rm --entrypoint=/bin/sh "$IMAGE" -c "/usr/bin/id -u bind; /usr/bin/id -g bind")
+read -d '' -r uid gid <<< "$uidgid" || :
+
+# Create or update the on host user to match the container's 'bind' user and group
+ACTUAL_UID=$(id -u bind 2>/dev/null)
+ACTUAL_GID=$(id -g bind 2>/dev/null)
+
+[ -z "$ACTUAL_GID" ] && groupadd -f -g "$gid" bind
+[ -z "$ACTUAL_UID" ] && useradd -u "$uid" -g "$gid" -M --no-log-init bind
+[ "$ACTUAL_GID" -ne "$gid" ] && groupmod -g "$gid" bind
+[ "$ACTUAL_UID" -ne "$uid" ] && usermod -u "$uid" -g "$gid"
+
+# create/update our main directory setup
+install -d -o bind -g bind -m 0755 /etc/bind/cfg /etc/bind/cache /etc/bind/zones /etc/bind/log/named
+# copy over our config and data without overwriting anything, hopefully.
+rsync -av --chown bind:bind --del ./cfg/  /etc/bind/cfg/
+rsync -av --chown bind:bind ./zones /etc/bind/zones/
+rsync -av --chown bind:bind --ignore-existing ./cache/ /etc/bind/cache/
+
+if [ -f docker.named.service ]; then
+    install -m 0644 docker.named.service /etc/systemd/system/docker.named.service
+fi
+
+systemctl try-restart docker.named.service
+
+exit 0