Note (2026-05-21): The console-login.toml referenced below is
now in Hammerhead's overlay tree at
base/usr/src/zyginit/overrides/console-login/service.toml.

Bug 001: supervisor leaves /dev/console as inherited fd 0 in service children

Summary

src/supervisor.reef's child-setup path redirects fd 1 and fd 2 to the
per-service log file but leaves fd 0 inherited from PID 1. PID 1 (zyginit)
runs setup_pid1_console() which opens /dev/console and dup2's the fd
onto 0/1/2 (with O_NOCTTY, correctly), so any service child inherits
fd 0 = /dev/console. The child then calls setsid(), becoming a session
leader with a /dev/console fd open.

That state — session leader with a tty fd inherited from before the
session was created — appears to be enough on Hammerhead for the kernel
to associate /dev/console as the new session's controlling tty even
though no explicit open() (with or without O_NOCTTY) was made by
the child for that fd. Symptom: ps -o tty shows TT=console for the
service, and any later daemon trying to claim ctty on /dev/console
(typically ttymon started by the console-login service in a higher
tier) fails with "cannot allocate controlling tty on /dev/console"
and login(1) cannot proceed → console-login enters a restart loop and
the system is unreachable from the physical console.

This was hit on hh-alpha1 immediately after first boot. SSH worked,
console login did not.

Severity

Medium. Affects any deployment where the kernel auto-claims ctty
under the inherited-fd-0-then-setsid pattern. The triggering daemons
are kernel-version- and toolchain-build-dependent (we observed it with
a newly-built syslogd on hh-alpha1; an older syslogd on hh-prototest
does not trip the same kernel path). Recoverable via SSH but blocks
the physical console use case.

Affected Version

zyginit 0.1.1 (as shipped in zyginit-0.1.1-source.tar.xz).

The defect is in src/supervisor.reef, the post-fork child setup near
line 333 in the version we vendored. We did not check 0.1.0.

Reproducer

1. Boot Hammerhead with zyginit as PID 1.
2. Have setup_pid1_console() open /dev/console and dup it onto 0/1/2.
3. Start any daemon that calls setsid() and continues running with
   the inherited fd 0 in scope (e.g. /usr/sbin/syslogd).
4. Inspect: ps -ef -o pid,sid,tty,comm | grep <daemon>.
   The TT column shows console even though the daemon never explicitly
   opened /dev/console without O_NOCTTY.
5. Start console-login (or any service running ttymon -d /dev/console)
   in a higher tier. ttymon prints the login prompt but login(1) then
   fails to claim ctty: "cannot allocate controlling tty on /dev/console
   — There may be another session active on this port."
6. zyginit restarts console-login; same failure repeats; restart loop.

Root cause (from the supervisor side)

src/supervisor.reef post-fork child setup (snippet from the version
we vendored):

if pid == 0
    // CHILD PROCESS
    process.process_setsid()

    // Redirect stdout/stderr to per-service log file
    let log_path = ...
    let log_fd = fd.fd_open(log_path, log_flags, 420)
    if log_fd >= 0
        fd.fd_dup2(log_fd, 1)
        fd.fd_dup2(log_fd, 2)
        fd.fd_close(log_fd)
    end if
    ...

Only fd 1 and fd 2 get redirected. fd 0 keeps PID 1's /dev/console
across the setsid(). From the kernel's perspective the new session is
born with an open fd referencing a tty that has no controlling session,
which under at least some illumos variants is enough to associate it
as the new session's ctty.

Suggested fix

In the child-setup path, before setsid(), redirect fd 0 to /dev/null
(or to the per-service log file, or close it entirely). For example:

if pid == 0
    // CHILD PROCESS

    // Redirect fd 0 to /dev/null BEFORE setsid so the new session is
    // never born holding an inherited /dev/console fd. Without this,
    // illumos may auto-claim /dev/console as the new session's ctty,
    // which then blocks ttymon (in console-login) from claiming it.
    let null_fd = fd.fd_open("/dev/null", fd.O_RDWR(), 0)
    if null_fd >= 0
        fd.fd_dup2(null_fd, 0)
        fd.fd_close(null_fd)
    end if

    process.process_setsid()

    // existing fd 1/2 → log file redirection
    ...

Optionally, add an explicit defensive TIOCNOTTY after setsid() —
open /dev/tty (which refers to the current ctty if any) and ioctl
TIOCNOTTY to revoke the association. This is a no-op when there's
no ctty, and a fix when one was unexpectedly inherited.

The fd-0-to-/dev/null change alone is the cheaper of the two and
matches the contract most service authors expect: services that need
stdin go open it explicitly.

Workarounds while pending

1. Per-service start.sh wrapper: prepend exec </dev/null to each
   daemon's exec line. This redirects fd 0 to /dev/null from inside
   the service script. Tested on alpha1 — did not eliminate the
   ctty grab in our specific syslogd build, suggesting the kernel
   path is more subtle than just fd-0-inherited (or our particular
   syslogd has a separate code path). See companion Hammerhead-side
   fix (TIOCNOTTY in untty()) which addressed our concrete symptom.

2. Reorder so console-login starts before any potential ctty-grabber:
   change the dependency in services/hammerhead/console-login.toml so
   it runs in an earlier tier than syslogd. ttymon claims ctty first;
   later daemons can't steal it.

Cross-references

• src/supervisor.reef — post-fork child setup
• src/main.reef — setup_pid1_console() (correctly uses O_NOCTTY)
• Hammerhead bug filed in parallel: syslogd defensive TIOCNOTTY in
  untty() to handle the kernel-side symptom from the cmd-side. That
  fix masks the specific syslogd case but doesn't address the broader
  class of any-daemon-may-trip-this — hence this RFE.

Reporter

Hammerhead build team (Chris Tusa, christusa@gmail.com)
Hit on hh-alpha1 first boot, 2026-05-06.
Recovery: SSH still worked; console login was the only blocked path.