|

bugs: archive completed 001 (fd-0 ctty) and 002 (group-type RFE)

Author: Chris Tusa <chris.tusa@leafscale.com>
Date: Jul 20, 2026 21:07
Changeset: 71b8fd133c85748877eb86d795dec8b914f86b8c
Branch: default

Changed files:

D bugs/001-supervisor-leaves-console-as-fd0-in-children.md
D bugs/002-rfe-group-target-service-type.md

Diff

diff -r c0018f136328 -r 71b8fd133c85 bugs/001-supervisor-leaves-console-as-fd0-in-children.md
--- a/bugs/001-supervisor-leaves-console-as-fd0-in-children.md	Mon Jul 20 19:49:45 2026 -0500
+++ /dev/null	Thu Jan 01 00:00:00 1970 +0000
@@ -1,150 +0,0 @@
-> Note (2026-05-21): The `console-login.toml` referenced below is
-> now in Hammerhead's overlay tree at
-> `base/usr/src/zyginit/overrides/console-login/service.toml`.
-
-# Bug 001: supervisor leaves /dev/console as inherited fd 0 in service children
-
-## Summary
-
-`src/supervisor.reef`'s child-setup path redirects fd 1 and fd 2 to the
-per-service log file but leaves fd 0 inherited from PID 1. PID 1 (zyginit)
-runs `setup_pid1_console()` which opens `/dev/console` and dup2's the fd
-onto 0/1/2 (with `O_NOCTTY`, correctly), so any service child inherits
-fd 0 = `/dev/console`. The child then calls `setsid()`, becoming a session
-leader with a `/dev/console` fd open.
-
-That state — session leader with a tty fd inherited from before the
-session was created — appears to be enough on Hammerhead for the kernel
-to associate `/dev/console` as the new session's controlling tty even
-though no explicit `open()` (with or without `O_NOCTTY`) was made by
-the child for that fd. Symptom: `ps -o tty` shows `TT=console` for the
-service, and any later daemon trying to claim ctty on `/dev/console`
-(typically `ttymon` started by the `console-login` service in a higher
-tier) fails with **"cannot allocate controlling tty on /dev/console"**
-and login(1) cannot proceed → `console-login` enters a restart loop and
-the system is unreachable from the physical console.
-
-This was hit on hh-alpha1 immediately after first boot. SSH worked,
-console login did not.
-
-## Severity
-
-**Medium.** Affects any deployment where the kernel auto-claims ctty
-under the inherited-fd-0-then-setsid pattern. The triggering daemons
-are kernel-version- and toolchain-build-dependent (we observed it with
-a newly-built `syslogd` on hh-alpha1; an older syslogd on hh-prototest
-does not trip the same kernel path). Recoverable via SSH but blocks
-the physical console use case.
-
-## Affected Version
-
-zyginit 0.1.1 (as shipped in `zyginit-0.1.1-source.tar.xz`).
-
-The defect is in `src/supervisor.reef`, the post-fork child setup near
-line 333 in the version we vendored. We did not check 0.1.0.
-
-## Reproducer
-
-1. Boot Hammerhead with zyginit as PID 1.
-2. Have `setup_pid1_console()` open `/dev/console` and dup it onto 0/1/2.
-3. Start any daemon that calls `setsid()` and continues running with
-   the inherited fd 0 in scope (e.g. `/usr/sbin/syslogd`).
-4. Inspect: `ps -ef -o pid,sid,tty,comm | grep <daemon>`.
-   The TT column shows `console` even though the daemon never explicitly
-   opened `/dev/console` without `O_NOCTTY`.
-5. Start `console-login` (or any service running `ttymon -d /dev/console`)
-   in a higher tier. ttymon prints the login prompt but `login(1)` then
-   fails to claim ctty: "cannot allocate controlling tty on /dev/console
-   — There may be another session active on this port."
-6. zyginit restarts `console-login`; same failure repeats; restart loop.
-
-## Root cause (from the supervisor side)
-
-`src/supervisor.reef` post-fork child setup (snippet from the version
-we vendored):
-
-```reef
-if pid == 0
-    // CHILD PROCESS
-    process.process_setsid()
-
-    // Redirect stdout/stderr to per-service log file
-    let log_path = ...
-    let log_fd = fd.fd_open(log_path, log_flags, 420)
-    if log_fd >= 0
-        fd.fd_dup2(log_fd, 1)
-        fd.fd_dup2(log_fd, 2)
-        fd.fd_close(log_fd)
-    end if
-    ...
-```
-
-Only fd 1 and fd 2 get redirected. fd 0 keeps PID 1's `/dev/console`
-across the `setsid()`. From the kernel's perspective the new session is
-born with an open fd referencing a tty that has no controlling session,
-which under at least some illumos variants is enough to associate it
-as the new session's ctty.
-
-## Suggested fix
-
-In the child-setup path, before `setsid()`, redirect fd 0 to `/dev/null`
-(or to the per-service log file, or close it entirely). For example:
-
-```reef
-if pid == 0
-    // CHILD PROCESS
-
-    // Redirect fd 0 to /dev/null BEFORE setsid so the new session is
-    // never born holding an inherited /dev/console fd. Without this,
-    // illumos may auto-claim /dev/console as the new session's ctty,
-    // which then blocks ttymon (in console-login) from claiming it.
-    let null_fd = fd.fd_open("/dev/null", fd.O_RDWR(), 0)
-    if null_fd >= 0
-        fd.fd_dup2(null_fd, 0)
-        fd.fd_close(null_fd)
-    end if
-
-    process.process_setsid()
-
-    // existing fd 1/2 → log file redirection
-    ...
-```
-
-Optionally, add an explicit defensive `TIOCNOTTY` after `setsid()` —
-open `/dev/tty` (which refers to the current ctty if any) and ioctl
-`TIOCNOTTY` to revoke the association. This is a no-op when there's
-no ctty, and a fix when one was unexpectedly inherited.
-
-The fd-0-to-/dev/null change alone is the cheaper of the two and
-matches the contract most service authors expect: services that need
-stdin go open it explicitly.
-
-## Workarounds while pending
-
-1. **Per-service `start.sh` wrapper**: prepend `exec </dev/null` to each
-   daemon's exec line. This redirects fd 0 to `/dev/null` from inside
-   the service script. Tested on alpha1 — did *not* eliminate the
-   ctty grab in our specific syslogd build, suggesting the kernel
-   path is more subtle than just fd-0-inherited (or our particular
-   syslogd has a separate code path). See companion Hammerhead-side
-   fix (TIOCNOTTY in `untty()`) which addressed our concrete symptom.
-
-2. **Reorder so console-login starts before any potential ctty-grabber**:
-   change the dependency in `services/hammerhead/console-login.toml` so
-   it runs in an earlier tier than syslogd. ttymon claims ctty first;
-   later daemons can't steal it.
-
-## Cross-references
-
-- `src/supervisor.reef` — post-fork child setup
-- `src/main.reef` — `setup_pid1_console()` (correctly uses `O_NOCTTY`)
-- Hammerhead bug filed in parallel: syslogd defensive `TIOCNOTTY` in
-  `untty()` to handle the kernel-side symptom from the cmd-side. That
-  fix masks the specific syslogd case but doesn't address the broader
-  class of any-daemon-may-trip-this — hence this RFE.
-
-## Reporter
-
-Hammerhead build team (Chris Tusa, christusa@gmail.com)
-Hit on hh-alpha1 first boot, 2026-05-06.
-Recovery: SSH still worked; console login was the only blocked path.
diff -r c0018f136328 -r 71b8fd133c85 bugs/002-rfe-group-target-service-type.md
--- a/bugs/002-rfe-group-target-service-type.md	Mon Jul 20 19:49:45 2026 -0500
+++ /dev/null	Thu Jan 01 00:00:00 1970 +0000
@@ -1,79 +0,0 @@
-# RFE 002: `type = "group"` service for grouping/targeting
-
-## Summary
-
-Add a `type = "group"` (or `"target"`) service kind to zyginit. A
-group service has no `[exec]` block; it exists only as a named
-synchronization/enable point. Its `[dependencies]` list expands into
-the set of services it represents.
-
-This is the equivalent of systemd's `.target` units. It lets admins
-enable/disable a feature set with one command without consolidating
-unrelated daemons under a single supervisor.
-
-## Motivation
-
-Hammerhead is converting the NFS, IPsec, and zones SMF service sets
-to zyginit. NFS in particular spreads across multiple independent
-daemons (`nfsmapid`, `statd`, `lockd`, `nfs4cbd`, plus the `client`
-mount oneshot) — each with its own lifecycle, restart policy, and
-log file.
-
-The natural design is one zyginit service per daemon. But that loses
-the SMF-era ergonomics of "enable nfs-client and the dependents come
-along". Without a group abstraction we have two bad options:
-
-1. **Combine multiple daemons into one zyginit service.** Cluttered:
-   one contract becomes N contracts, restart semantics get fuzzy,
-   log routing needs splitting, and the supervisor's child-death
-   handling no longer maps 1:1 to a service.
-
-2. **Document the bundle and tell admins to enable each member.**
-   Error-prone and ugly:
-   ```
-   zygctl enable nfs-cbd nfs-mapid nfs-statd nfs-lockd nfs-client
-   ```
-
-A group service neatly avoids both:
-```toml
-[service]
-name = "nfs-client"
-description = "NFSv4 client services"
-type = "group"
-[dependencies]
-requires = ["nfs-cbd", "nfs-mapid", "nfs-statd", "nfs-lockd",
-            "nfs-client-mount"]
-```
-Then `zygctl enable nfs-client` is sufficient. `zygctl status nfs-client`
-reports composite up/down derived from the members.
-
-## Suggested semantics
-
-- `type = "group"` services have no `[exec]` block. The supervisor
-  never forks anything for them.
-- Status: `running` iff all `requires` members are `running`;
-  `stopped` iff none are; `partial` (or `degraded`) if some.
-- `zygctl enable <group>` symlinks the group's TOML in `enabled.d/`
-  *and* recursively enables each member that is not already enabled
-  (or, simpler: refuses unless all members are also enabled —
-  consistent with how SMF dependencies block enable today).
-- `zygctl disable <group>` is symmetric.
-- `zygctl start <group>` / `stop <group>` operate on the member set.
-- Cycle detection treats group-edges the same as service-edges.
-
-## Use cases
-
-| Group | Members |
-|---|---|
-| `nfs-client` | nfs-cbd, nfs-mapid, nfs-statd, nfs-lockd, nfs-client-mount |
-| `nfs-server` | nfsd, mountd, nfs-statd, nfs-lockd |
-| `zones` | zones, zonestat |
-| `audit` | (just `auditd` after the auditset fold-in — not very useful for audit, but harmless) |
-| `network-routing` | forwarding, routing/ndp, network-iptun |
-
-Hammerhead would land this in roughly 4–6 group definitions.
-
-## Reporter
-
-Hammerhead build team (Chris Tusa, christusa@gmail.com)
-Filed during SMF-to-zyginit migration policy work, 2026-05-08.
diff -r c0018f136328 -r 71b8fd133c85 bugs/archive/001-supervisor-leaves-console-as-fd0-in-children.md
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/bugs/archive/001-supervisor-leaves-console-as-fd0-in-children.md	Mon Jul 20 21:07:33 2026 -0500
@@ -0,0 +1,150 @@
+> Note (2026-05-21): The `console-login.toml` referenced below is
+> now in Hammerhead's overlay tree at
+> `base/usr/src/zyginit/overrides/console-login/service.toml`.
+
+# Bug 001: supervisor leaves /dev/console as inherited fd 0 in service children
+
+## Summary
+
+`src/supervisor.reef`'s child-setup path redirects fd 1 and fd 2 to the
+per-service log file but leaves fd 0 inherited from PID 1. PID 1 (zyginit)
+runs `setup_pid1_console()` which opens `/dev/console` and dup2's the fd
+onto 0/1/2 (with `O_NOCTTY`, correctly), so any service child inherits
+fd 0 = `/dev/console`. The child then calls `setsid()`, becoming a session
+leader with a `/dev/console` fd open.
+
+That state — session leader with a tty fd inherited from before the
+session was created — appears to be enough on Hammerhead for the kernel
+to associate `/dev/console` as the new session's controlling tty even
+though no explicit `open()` (with or without `O_NOCTTY`) was made by
+the child for that fd. Symptom: `ps -o tty` shows `TT=console` for the
+service, and any later daemon trying to claim ctty on `/dev/console`
+(typically `ttymon` started by the `console-login` service in a higher
+tier) fails with **"cannot allocate controlling tty on /dev/console"**
+and login(1) cannot proceed → `console-login` enters a restart loop and
+the system is unreachable from the physical console.
+
+This was hit on hh-alpha1 immediately after first boot. SSH worked,
+console login did not.
+
+## Severity
+
+**Medium.** Affects any deployment where the kernel auto-claims ctty
+under the inherited-fd-0-then-setsid pattern. The triggering daemons
+are kernel-version- and toolchain-build-dependent (we observed it with
+a newly-built `syslogd` on hh-alpha1; an older syslogd on hh-prototest
+does not trip the same kernel path). Recoverable via SSH but blocks
+the physical console use case.
+
+## Affected Version
+
+zyginit 0.1.1 (as shipped in `zyginit-0.1.1-source.tar.xz`).
+
+The defect is in `src/supervisor.reef`, the post-fork child setup near
+line 333 in the version we vendored. We did not check 0.1.0.
+
+## Reproducer
+
+1. Boot Hammerhead with zyginit as PID 1.
+2. Have `setup_pid1_console()` open `/dev/console` and dup it onto 0/1/2.
+3. Start any daemon that calls `setsid()` and continues running with
+   the inherited fd 0 in scope (e.g. `/usr/sbin/syslogd`).
+4. Inspect: `ps -ef -o pid,sid,tty,comm | grep <daemon>`.
+   The TT column shows `console` even though the daemon never explicitly
+   opened `/dev/console` without `O_NOCTTY`.
+5. Start `console-login` (or any service running `ttymon -d /dev/console`)
+   in a higher tier. ttymon prints the login prompt but `login(1)` then
+   fails to claim ctty: "cannot allocate controlling tty on /dev/console
+   — There may be another session active on this port."
+6. zyginit restarts `console-login`; same failure repeats; restart loop.
+
+## Root cause (from the supervisor side)
+
+`src/supervisor.reef` post-fork child setup (snippet from the version
+we vendored):
+
+```reef
+if pid == 0
+    // CHILD PROCESS
+    process.process_setsid()
+
+    // Redirect stdout/stderr to per-service log file
+    let log_path = ...
+    let log_fd = fd.fd_open(log_path, log_flags, 420)
+    if log_fd >= 0
+        fd.fd_dup2(log_fd, 1)
+        fd.fd_dup2(log_fd, 2)
+        fd.fd_close(log_fd)
+    end if
+    ...
+```
+
+Only fd 1 and fd 2 get redirected. fd 0 keeps PID 1's `/dev/console`
+across the `setsid()`. From the kernel's perspective the new session is
+born with an open fd referencing a tty that has no controlling session,
+which under at least some illumos variants is enough to associate it
+as the new session's ctty.
+
+## Suggested fix
+
+In the child-setup path, before `setsid()`, redirect fd 0 to `/dev/null`
+(or to the per-service log file, or close it entirely). For example:
+
+```reef
+if pid == 0
+    // CHILD PROCESS
+
+    // Redirect fd 0 to /dev/null BEFORE setsid so the new session is
+    // never born holding an inherited /dev/console fd. Without this,
+    // illumos may auto-claim /dev/console as the new session's ctty,
+    // which then blocks ttymon (in console-login) from claiming it.
+    let null_fd = fd.fd_open("/dev/null", fd.O_RDWR(), 0)
+    if null_fd >= 0
+        fd.fd_dup2(null_fd, 0)
+        fd.fd_close(null_fd)
+    end if
+
+    process.process_setsid()
+
+    // existing fd 1/2 → log file redirection
+    ...
+```
+
+Optionally, add an explicit defensive `TIOCNOTTY` after `setsid()` —
+open `/dev/tty` (which refers to the current ctty if any) and ioctl
+`TIOCNOTTY` to revoke the association. This is a no-op when there's
+no ctty, and a fix when one was unexpectedly inherited.
+
+The fd-0-to-/dev/null change alone is the cheaper of the two and
+matches the contract most service authors expect: services that need
+stdin go open it explicitly.
+
+## Workarounds while pending
+
+1. **Per-service `start.sh` wrapper**: prepend `exec </dev/null` to each
+   daemon's exec line. This redirects fd 0 to `/dev/null` from inside
+   the service script. Tested on alpha1 — did *not* eliminate the
+   ctty grab in our specific syslogd build, suggesting the kernel
+   path is more subtle than just fd-0-inherited (or our particular
+   syslogd has a separate code path). See companion Hammerhead-side
+   fix (TIOCNOTTY in `untty()`) which addressed our concrete symptom.
+
+2. **Reorder so console-login starts before any potential ctty-grabber**:
+   change the dependency in `services/hammerhead/console-login.toml` so
+   it runs in an earlier tier than syslogd. ttymon claims ctty first;
+   later daemons can't steal it.
+
+## Cross-references
+
+- `src/supervisor.reef` — post-fork child setup
+- `src/main.reef` — `setup_pid1_console()` (correctly uses `O_NOCTTY`)
+- Hammerhead bug filed in parallel: syslogd defensive `TIOCNOTTY` in
+  `untty()` to handle the kernel-side symptom from the cmd-side. That
+  fix masks the specific syslogd case but doesn't address the broader
+  class of any-daemon-may-trip-this — hence this RFE.
+
+## Reporter
+
+Hammerhead build team (Chris Tusa, christusa@gmail.com)
+Hit on hh-alpha1 first boot, 2026-05-06.
+Recovery: SSH still worked; console login was the only blocked path.
diff -r c0018f136328 -r 71b8fd133c85 bugs/archive/002-rfe-group-target-service-type.md
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/bugs/archive/002-rfe-group-target-service-type.md	Mon Jul 20 21:07:33 2026 -0500
@@ -0,0 +1,79 @@
+# RFE 002: `type = "group"` service for grouping/targeting
+
+## Summary
+
+Add a `type = "group"` (or `"target"`) service kind to zyginit. A
+group service has no `[exec]` block; it exists only as a named
+synchronization/enable point. Its `[dependencies]` list expands into
+the set of services it represents.
+
+This is the equivalent of systemd's `.target` units. It lets admins
+enable/disable a feature set with one command without consolidating
+unrelated daemons under a single supervisor.
+
+## Motivation
+
+Hammerhead is converting the NFS, IPsec, and zones SMF service sets
+to zyginit. NFS in particular spreads across multiple independent
+daemons (`nfsmapid`, `statd`, `lockd`, `nfs4cbd`, plus the `client`
+mount oneshot) — each with its own lifecycle, restart policy, and
+log file.
+
+The natural design is one zyginit service per daemon. But that loses
+the SMF-era ergonomics of "enable nfs-client and the dependents come
+along". Without a group abstraction we have two bad options:
+
+1. **Combine multiple daemons into one zyginit service.** Cluttered:
+   one contract becomes N contracts, restart semantics get fuzzy,
+   log routing needs splitting, and the supervisor's child-death
+   handling no longer maps 1:1 to a service.
+
+2. **Document the bundle and tell admins to enable each member.**
+   Error-prone and ugly:
+   ```
+   zygctl enable nfs-cbd nfs-mapid nfs-statd nfs-lockd nfs-client
+   ```
+
+A group service neatly avoids both:
+```toml
+[service]
+name = "nfs-client"
+description = "NFSv4 client services"
+type = "group"
+[dependencies]
+requires = ["nfs-cbd", "nfs-mapid", "nfs-statd", "nfs-lockd",
+            "nfs-client-mount"]
+```
+Then `zygctl enable nfs-client` is sufficient. `zygctl status nfs-client`
+reports composite up/down derived from the members.
+
+## Suggested semantics
+
+- `type = "group"` services have no `[exec]` block. The supervisor
+  never forks anything for them.
+- Status: `running` iff all `requires` members are `running`;
+  `stopped` iff none are; `partial` (or `degraded`) if some.
+- `zygctl enable <group>` symlinks the group's TOML in `enabled.d/`
+  *and* recursively enables each member that is not already enabled
+  (or, simpler: refuses unless all members are also enabled —
+  consistent with how SMF dependencies block enable today).
+- `zygctl disable <group>` is symmetric.
+- `zygctl start <group>` / `stop <group>` operate on the member set.
+- Cycle detection treats group-edges the same as service-edges.
+
+## Use cases
+
+| Group | Members |
+|---|---|
+| `nfs-client` | nfs-cbd, nfs-mapid, nfs-statd, nfs-lockd, nfs-client-mount |
+| `nfs-server` | nfsd, mountd, nfs-statd, nfs-lockd |
+| `zones` | zones, zonestat |
+| `audit` | (just `auditd` after the auditset fold-in — not very useful for audit, but harmless) |
+| `network-routing` | forwarding, routing/ndp, network-iptun |
+
+Hammerhead would land this in roughly 4–6 group definitions.
+
+## Reporter
+
+Hammerhead build team (Chris Tusa, christusa@gmail.com)
+Filed during SMF-to-zyginit migration policy work, 2026-05-08.