|
root / docs / notes / non-pid1-clobbers-socket.md
non-pid1-clobbers-socket.md markdown 89 lines 3.4 KB

Operator gotcha: do not invoke /sbin/init (zyginit) from a shell

TL;DR

Never run /sbin/init (or anything that exec's the zyginit binary)
from a shell on a running zyginit system.
Running it even with a
no-op-looking flag like --version will silently break the admin
socket and lock you out of zygctl, halt, reboot, poweroff,
and init <runlevel>. Recovery is a hard power-cycle.

Disposition: documentation-only. The behaviour is a side-effect of
zyginit's stale-socket recovery (anti-pattern #5 in docs/SMF_MIGRATION.md),
which is correct for PID 1 but unsafe for non-PID-1 invocations. We
work around this by simply not running /sbin/init manually.

What happens

When zyginit is invoked as a non-PID-1 process, it correctly detects
"non-init mode" and exits without taking over service supervision —
but on the way through it runs the early socket-setup path, which
unlinks /var/run/zyginit.sock and binds a new socket of its own.
When the process exits the new socket is closed; the filesystem path
is now a dead-end socket file with no listener.

PID 1's listening socket fd is still healthy, but its admin path is
no longer reachable by name. From that moment until reboot:

  • zygctl <anything-going-through-the-socket> fails with
    cannot connect to zyginit at /var/run/zyginit.sock
  • /sbin/halt, /sbin/reboot, /sbin/poweroff, /sbin/init <runlevel>
    all fail too — sysv-wrapper exec's /sbin/zygctl <verb>, which
    hits the same broken socket
  • init q cannot signal PID 1 to re-bind, because the signalling
    path goes through the same socket

Service supervision (the actual PID 1 work) continues uninterrupted.
The kernel and the running daemons are unaffected. It is a soft
brick of the admin plane only.

Reproducer

ssh root@hh-target
zygctl status                # works
/sbin/init --version         # appears to print a version banner
zygctl status                # fails
ls -la /var/run/zyginit.sock # mtime is "now", but path is dead
pgrep -fl zyginit            # only PID 1 — no rogue running

Recovery

Hard power-cycle from the hypervisor or the physical box. On libvirt:

virsh destroy hh-target && virsh start hh-target

virsh reboot (ACPI shutdown) does not work — zyginit has no
ACPI button-press handler at present, so the kernel never sees the
shutdown request.

Why this is filed as a note rather than a bug

The unlink-then-bind behaviour is correct for PID 1 (per
anti-pattern #5 in docs/SMF_MIGRATION.md). The non-PID-1 path
wandering through the same setup code is a code-organisation issue
that could be fixed in zyginit, but the workaround (don't exec
/sbin/init from a shell) is trivial — administrators should never
need to invoke the init binary directly anyway. The zyginit team
can address this whenever the early-startup code gets reworked;
until then this note exists so that anyone who hits the footgun
has a path to recovery.

Cross-references

  • docs/SMF_MIGRATION.md anti-pattern #5 — explains the unlink-then-bind
    pattern and why it exists for PID 1.
  • tools/sysv-wrapper/wrapper.c — confirms halt, reboot, poweroff,
    telinit, init <runlevel> all funnel through /sbin/zygctl, so
    the socket clobber breaks the whole sysv-init compat layer
    simultaneously.

History

First hit on hh-prototest 2026-05-05 during the Hammerhead-team
integration walk. Recovered via virsh destroy && virsh start.
Filed by Chris Tusa (christusa@gmail.com).