first commit

README.md

@@ -0,0 +1,184 @@
+# Kasm Session IP Fix
+
+`kasm-fix-session-ips.sh` repairs stale Kasm Workspaces session IP mappings after a host or Docker restart.
+
+The problem occurs when Docker assigns an existing Kasm workspace container a new address, while Kasm still keeps the old address in:
+
+- PostgreSQL: `public.kasms.container_ip`
+- nginx session config: `conf/nginx/containers.d/<container_id>.conf`
+
+This can cause Kasm sessions to stay in `running` state while opening the desktop returns `502 Bad Gateway`.
+
+The script:
+
+1. Finds all running Kasm container sessions in `public.kasms`.
+2. Reads each container's actual IP from Docker.
+3. Updates `public.kasms.container_ip` when it is stale.
+4. Updates the matching nginx `containers.d/<container_id>.conf`.
+5. Creates a backup of every nginx config it changes.
+6. Runs `nginx -t`.
+7. Reloads `kasm_proxy` only if the nginx configuration is valid and was changed.
+
+## Requirements
+
+- Kasm Workspaces single-server/containerised installation
+- Docker
+- PostgreSQL container named `kasm_db`
+- Kasm proxy container named `kasm_proxy`
+- Docker network named `kasm_default_network`
+- `flock`
+- root privileges
+
+Defaults can be overridden with environment variables:
+
+```text
+KASM_DB_CONTAINER=kasm_db
+KASM_PROXY_CONTAINER=kasm_proxy
+KASM_NETWORK=kasm_default_network
+DB_USER=kasmapp
+DB_NAME=kasm
+WAIT_SECONDS=120
+```
+
+## Install
+
+Clone or copy the files to the Kasm host, then install the script:
+
+```bash
+sudo install -m 0755 kasm-fix-session-ips.sh /usr/local/sbin/kasm-fix-session-ips
+```
+
+Run it manually:
+
+```bash
+sudo /usr/local/sbin/kasm-fix-session-ips
+```
+
+A healthy system where no changes are required should produce output similar to:
+
+```text
+Using nginx config root: /opt/kasm/1.17.0/conf/nginx
+Session e1241aac-ab8c-4e82-b1ae-db5962076b75:
+  container=354e7ee145644b71b9c5899fd35c4f7779111d5064bb9db9f3d720009b3e579c
+  db_ip=172.18.0.7
+  docker_ip=172.18.0.7
+Done: processed=1 db_fixed=0 nginx_fixed=0
+```
+
+If Docker changed the workspace address, the output will look similar to:
+
+```text
+db_ip=172.18.0.7
+docker_ip=172.18.0.13
+DB fixed: 172.18.0.7 -> 172.18.0.13
+nginx fixed: proxy_pass targets -> 172.18.0.13
+Validating nginx configuration...
+Reloading nginx...
+Done: processed=1 db_fixed=1 nginx_fixed=1
+```
+
+## systemd service
+
+Create `/etc/systemd/system/kasm-fix-session-ips.service`:
+
+```ini
+[Unit]
+Description=Repair stale Kasm Workspace container IP mappings
+After=docker.service network-online.target
+Requires=docker.service
+
+[Service]
+Type=oneshot
+ExecStart=/usr/local/sbin/kasm-fix-session-ips
+TimeoutStartSec=180
+```
+
+## systemd timer
+
+Create `/etc/systemd/system/kasm-fix-session-ips.timer`:
+
+```ini
+[Unit]
+Description=Periodically repair stale Kasm Workspace container IP mappings
+
+[Timer]
+OnBootSec=30s
+OnUnitActiveSec=2min
+AccuracySec=10s
+Persistent=true
+Unit=kasm-fix-session-ips.service
+
+[Install]
+WantedBy=timers.target
+```
+
+Enable the timer:
+
+```bash
+sudo systemctl daemon-reload
+sudo systemctl enable --now kasm-fix-session-ips.timer
+```
+
+Check its status:
+
+```bash
+systemctl status kasm-fix-session-ips.timer --no-pager
+systemctl list-timers kasm-fix-session-ips.timer
+```
+
+Run the repair immediately through systemd:
+
+```bash
+sudo systemctl start kasm-fix-session-ips.service
+```
+
+View logs:
+
+```bash
+journalctl -u kasm-fix-session-ips.service -n 100 --no-pager
+```
+
+## What gets changed
+
+For each running local Kasm workspace, the script compares:
+
+```text
+public.kasms.container_ip
+```
+
+with the actual Docker address on:
+
+```text
+kasm_default_network
+```
+
+If they differ, the database record is updated.
+
+It then updates only these KasmVNC-related nginx upstream ports in the corresponding session config:
+
+```text
+6901
+4901
+4902
+4903
+```
+
+For example:
+
+```text
+proxy_pass https://172.18.0.11:6901/;
+```
+
+becomes:
+
+```text
+proxy_pass https://172.18.0.7:6901/;
+```
+
+Before changing a session nginx config, the script creates a timestamped backup:
+
+```text
+<container_id>.conf.bak.YYYYMMDD-HHMMSS
+```
+
+If the file does not actually change, the unnecessary backup is removed.

kasm-fix-session-ips.sh

@@ -0,0 +1,212 @@
+#!/usr/bin/env bash
+set -Eeuo pipefail
+
+KASM_DB_CONTAINER="${KASM_DB_CONTAINER:-kasm_db}"
+KASM_PROXY_CONTAINER="${KASM_PROXY_CONTAINER:-kasm_proxy}"
+KASM_NETWORK="${KASM_NETWORK:-kasm_default_network}"
+
+DB_USER="${DB_USER:-kasmapp}"
+DB_NAME="${DB_NAME:-kasm}"
+
+WAIT_SECONDS="${WAIT_SECONDS:-120}"
+LOCK_FILE="${LOCK_FILE:-/run/kasm-fix-session-ips.lock}"
+
+log() {
+    printf '%s %s\n' "$(date '+%F %T')" "$*"
+}
+
+if [[ $EUID -ne 0 ]]; then
+    echo "Run as root: sudo $0" >&2
+    exit 1
+fi
+
+# Не допускаем одновременного запуска двух экземпляров.
+exec 9>"$LOCK_FILE"
+if ! flock -n 9; then
+    log "Another instance is already running; exiting."
+    exit 0
+fi
+
+wait_for_container() {
+    local name="$1"
+    local end=$((SECONDS + WAIT_SECONDS))
+
+    while (( SECONDS < end )); do
+        if [[ "$(docker inspect -f '{{.State.Running}}' "$name" 2>/dev/null || true)" == "true" ]]; then
+            return 0
+        fi
+        sleep 2
+    done
+
+    log "ERROR: container $name did not become running within ${WAIT_SECONDS}s"
+    return 1
+}
+
+wait_for_container "$KASM_DB_CONTAINER"
+wait_for_container "$KASM_PROXY_CONTAINER"
+
+# Ждём PostgreSQL.
+end=$((SECONDS + WAIT_SECONDS))
+
+until docker exec "$KASM_DB_CONTAINER" \
+    pg_isready -U "$DB_USER" -d "$DB_NAME" >/dev/null 2>&1
+do
+    if (( SECONDS >= end )); then
+        log "ERROR: PostgreSQL did not become ready within ${WAIT_SECONDS}s"
+        exit 1
+    fi
+
+    sleep 2
+done
+
+# Определяем host-side путь к /etc/nginx/conf.d автоматически.
+CONF_ROOT="$(
+    docker inspect "$KASM_PROXY_CONTAINER" \
+        --format '{{range .Mounts}}{{if eq .Destination "/etc/nginx/conf.d"}}{{.Source}}{{end}}{{end}}'
+)"
+
+if [[ -z "$CONF_ROOT" || ! -d "$CONF_ROOT/containers.d" ]]; then
+    log "ERROR: could not determine nginx containers.d directory"
+    log "CONF_ROOT='$CONF_ROOT'"
+    exit 1
+fi
+
+log "Using nginx config root: $CONF_ROOT"
+
+# Берём все активные container-based Kasm sessions.
+mapfile -t sessions < <(
+    docker exec "$KASM_DB_CONTAINER" psql \
+        -U "$DB_USER" \
+        -d "$DB_NAME" \
+        -At \
+        -F $'\t' \
+        -c "
+            SELECT
+                kasm_id::text,
+                container_id,
+                COALESCE(container_ip, '')
+            FROM public.kasms
+            WHERE operational_status = 'running'
+              AND container_id IS NOT NULL
+              AND container_id <> ''
+            ORDER BY kasm_id;
+        "
+)
+
+if (( ${#sessions[@]} == 0 )); then
+    log "No running Kasm container sessions found."
+    exit 0
+fi
+
+changed_db=0
+changed_nginx=0
+processed=0
+
+for row in "${sessions[@]}"; do
+    IFS=$'\t' read -r kasm_id container_id db_ip <<<"$row"
+
+    [[ -n "$kasm_id" && -n "$container_id" ]] || continue
+
+    # Для distributed Kasm чужие контейнеры просто пропускаются.
+    if ! docker inspect "$container_id" >/dev/null 2>&1; then
+        log "SKIP: $kasm_id container $container_id does not exist on this Docker host"
+        continue
+    fi
+
+    actual_ip="$(
+        docker inspect "$container_id" \
+            --format "{{with index .NetworkSettings.Networks \"$KASM_NETWORK\"}}{{.IPAddress}}{{end}}" \
+            2>/dev/null || true
+    )"
+
+    if [[ -z "$actual_ip" ]]; then
+        log "SKIP: $kasm_id container $container_id has no IP on $KASM_NETWORK"
+        continue
+    fi
+
+    ((processed+=1))
+
+    log "Session $kasm_id:"
+    log "  container=$container_id"
+    log "  db_ip=${db_ip:-<empty>}"
+    log "  docker_ip=$actual_ip"
+
+    #
+    # 1. Исправляем public.kasms.container_ip
+    #
+    if [[ "$db_ip" != "$actual_ip" ]]; then
+        updated="$(
+            docker exec "$KASM_DB_CONTAINER" psql \
+                -U "$DB_USER" \
+                -d "$DB_NAME" \
+                -qAt \
+                -c "
+                    UPDATE public.kasms
+                    SET container_ip = '$actual_ip'
+                    WHERE kasm_id = '$kasm_id'::uuid
+                      AND container_id = '$container_id'
+                      AND container_ip IS DISTINCT FROM '$actual_ip'
+                    RETURNING container_ip;
+                "
+        )"
+
+        if [[ "$updated" == "$actual_ip" ]]; then
+            log "  DB fixed: ${db_ip:-<empty>} -> $actual_ip"
+            ((changed_db+=1))
+        else
+            log "  WARNING: database row was not updated as expected"
+        fi
+    fi
+
+    #
+    # 2. Исправляем nginx containers.d/<container_id>.conf
+    #
+    conf="$CONF_ROOT/containers.d/$container_id.conf"
+
+    if [[ ! -f "$conf" ]]; then
+        log "  WARNING: nginx session config not found: $conf"
+        continue
+    fi
+
+    backup="${conf}.bak.$(date '+%Y%m%d-%H%M%S')"
+
+    cp -a "$conf" "$backup"
+
+    # KasmVNC:
+    #   6901 - main VNC/websockify
+    #   4901 - audio
+    #   4902 - audio input
+    #   4903 - upload/etc
+    #
+    # Меняем только IP в proxy_pass для этих портов.
+    sed -E -i \
+        "s#(proxy_pass[[:space:]]+https://)[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+:(6901|4901|4902|4903)/;#\1${actual_ip}:\2/;#g" \
+        "$conf"
+
+    if cmp -s "$conf" "$backup"; then
+        rm -f "$backup"
+    else
+        log "  nginx fixed: proxy_pass targets -> $actual_ip"
+        log "  backup: $backup"
+        ((changed_nginx+=1))
+    fi
+done
+
+#
+# 3. Перечитываем nginx только при реальном изменении.
+#
+if (( changed_nginx > 0 )); then
+    log "Validating nginx configuration..."
+
+    if ! docker exec "$KASM_PROXY_CONTAINER" nginx -t; then
+        log "ERROR: nginx -t failed."
+        log "nginx was NOT reloaded."
+        log "Backups were kept in $CONF_ROOT/containers.d/"
+        exit 1
+    fi
+
+    log "Reloading nginx..."
+    docker exec "$KASM_PROXY_CONTAINER" nginx -s reload
+fi
+
+log "Done: processed=$processed db_fixed=$changed_db nginx_fixed=$changed_nginx"