# PID-1 Boot Design

**Date:** 2026-04-24
**Revision:** 2026-04-25 — BSD-pivot (post iter 1-5 findings)
**Status:** Revised; iter 6 implementation pending against the new service set
**Scope:** First boot of zyginit as PID 1 on Hammerhead (hh-prototest VM), replacing the SMF-managed boot chain with a zyginit-managed BSD-style boot.

## 1. Goal

Boot hh-prototest with `/sbin/init` pointing at zyginit, bringing the system to a state where the operator can SSH in and use the box. Deliver a small **BSD-style** service set (≈15 services) that uses direct command invocation (`ifconfig`, `dhcpagent`) instead of supervising SMF-coupled control-plane daemons. Tested rollback path via ZFS Boot Environments.

## 1.1 Design revision: BSD pivot (2026-04-25)

The original 21-service design (preserved in §4 history at the bottom of this doc) tried to supervise the existing illumos control-plane daemons — `dlmgmtd`, `ipmgmtd`, `inetd`, `rpcbind`, `fmd` — directly. Iter 1-5 (rev 24 through rev 50) surfaced a fundamental issue: these daemons are tightly coupled to **`svc.configd`** for their persistent property store via `libscf`. Setting `SMF_FMRI` in the environment bypasses their refusal-to-start banner, but they then silently degrade because `scf_handle_decode_fmri()` fails (no `/etc/svc/volatile/repository_door`), leaving them running but useless — `ipmgmtd` doesn't open its door socket; `ipadm` consequently fails with "Could not open handle to library"; `lo0` is never plumbed; `rpcbind` cascades into "could not find loopback transports"; the whole network stack is dead.

We had three choices:

- **A: Run `configd` under zyginit** — keeps the SMF property-store daemon alive, drops only `svc.startd`. Gets the daemons working but admits SMF is too entangled to fully escape.
- **B: Use `/lib/svc/method/*` scripts as zyginit's exec.start** — same effective coupling to `configd` as A; just hides it behind shell wrappers.
- **C: Pivot to BSD-style direct invocation** — drop the configd-coupled daemons entirely. Use `/sbin/ifconfig` directly to plumb interfaces and configure addresses. Use `/sbin/dhcpagent` directly for DHCP. Skip `inetd`/`rpcbind`/`fmd` for the default service set.

We chose **C**. Hammerhead retains the BSD primitives that make this work:

- `/sbin/ifconfig` exists and successfully plumbs interfaces without needing `ipmgmtd`
- `/etc/hostname.<if>` files supported (BSD-style interface naming/config)
- `/sbin/dhcpagent` is callable directly
- `/etc/init.d/`, `/etc/rc[0-3].d/` directories present (we won't use them, but the convention is there)

Reference for the approach: IRIX 6.5's `/etc/init.d/network` script (`os_research/irix-6517-src/eoe/cmd/initpkg/init.d/network`) does exactly this — `IFCONFIG=/usr/etc/ifconfig`, `ROUTE=/usr/etc/route`, flat-file configuration in `/etc/config/`, no daemon-based registry. zyginit-on-Hammerhead becomes essentially a Reef rewrite of pre-SMF Solaris boot semantics, replacing what IRIX/Solaris-9 did with shell.

Lessons from iter 1-5 also drove these other concrete fixes that survive in v2:

- **fd-setup at PID 1**: kernel doesn't preopen 0/1/2; zyginit must `open(/dev/console)` and `dup2()` onto 0/1/2 before any I/O (rev 45)
- **argv[0] in supervisor**: must include the program name as argv[0] when execvp'ing (rev 44)
- **root-fs ZFS/UFS dispatch**: `mount -o remount,rw /` errors on ZFS root; use `zfs set readonly=off` (rev 43)
- **Tier ordering enforcement**: between tiers, wait for oneshots to STOPPED and daemons to RUNNING before starting next tier (rev 47)
- **Persistent log dir**: `/var/log/zyginit/` (not `/var/run`) so logs survive reboot for diagnosis (rev 48)
- **Hammerhead path corrections**: `/bin/hostname` (not `/usr/bin/`), `/sbin/swapadd` (no `-a`), `/sbin/dumpadm`, etc.
- **cron FIFO is at `/etc/cron.d/FIFO`**, not `/var/run/cron_fifo`

A useful diagnostic capability also fell out: with the VM shut down, the BE's logs are readable from the dev host via `qemu-nbd` + zfsonlinux read-only import, then mounting `hhtest/ROOT/zyginit` to read `/var/log/zyginit/*.log`. This bypasses the GRUB-cant-pick-BE limitation on this Hammerhead version.

## 2. Non-goals

- **SMF removal from the BE.** Phase C, separate milestone. First boot leaves SMF binaries installed-but-unused. We just don't *invoke* them from zyginit's service set.
- **Supervising configd-coupled daemons.** `dlmgmtd`, `ipmgmtd`, `inetd`, `rpcbind`, `fmd` are explicitly out of zyginit's default service set. They depend on a running `svc.configd` that we no longer launch. They remain available on the BE for an admin who chooses to set up SMF separately.
- **Modern illumos datalink/IP management semantics.** No `dladm show-link`/`ipadm show-addr` integration. Configuration is via flat files (`/etc/hostname.<if>`, `/etc/defaultrouter`, `/etc/resolv.conf`). For laptops with WiFi roaming, link aggregation, IPMP, or VNICs for zones, this design is insufficient — those workflows need the SMF stack.
- **On-demand network services (inetd-style).** No `inetd` in the default set. Services that historically ran under `inetd` (telnet, rsh, finger, etc.) are not started. SSH is direct.
- **NFS / RPC.** `rpcbind` not started; no NFS in scope.
- **Fault management.** `fmd` not started; FMA events are not collected.
- **Single-user mode / runlevels.** `zygctl single` and `zygctl multiuser` remain stubs.
- **Crash-recovery re-adoption.** `ct_ctl_adopt` requires descendant relationship; only PID-1 in-place re-exec is supported.
- **`/sbin/halt`, `/sbin/reboot`, `/sbin/poweroff`, `/usr/sbin/shutdown` integration.** Continue to call `uadmin` directly. Clean shutdown goes through `zygctl`.
- **Name services (NSS), NTP, performance targets.** Deferred.

## 3. Architecture

### 3.1 Components

| Component | Path | Type | Purpose |
|---|---|---|---|
| `zyginit` | `/sbin/zyginit` | ELF binary | Init daemon (PID 1) |
| `zygctl` | `/sbin/zygctl` | ELF binary | Admin CLI |
| Config tree | `/etc/zyginit/` | Directory | Service TOMLs, per-service script dirs, `enabled.d/` symlink farm |
| Runtime tree | `/var/run/zyginit/` | Directory | Per-service log files, socket, transient state |
| Control socket | `/var/run/zyginit.sock` | Unix socket | zygctl ↔ zyginit IPC |

### 3.2 Boot sequence at runtime

1. Kernel execs `/sbin/init` (symlink to `/sbin/zyginit`)
2. zyginit detects PID (`getpid() == 1`) and enters PID-1 mode
3. zyginit scans `/etc/zyginit/enabled.d/` for symlinks
4. zyginit parses each TOML referenced; builds in-memory `ServiceDef` set
5. zyginit builds dependency graph via topological sort into tier ordering
6. zyginit starts services tier by tier:
   - Within a tier: sequential start (parallel start is a future optimization)
   - Between tiers: wait for all members of tier N to reach terminal state (RUNNING for daemons, STOPPED+exit=0 for oneshots) before starting tier N+1
7. zyginit opens event loop on: contract bundle fd, signal self-pipe, zygctl socket
8. System is "up" when all enabled services have started successfully

### 3.3 PID-1 vs non-PID-1 behavior

| Condition | Trigger | Behavior |
|---|---|---|
| PID 1 | SIGTERM / SIGINT | Halt: stop services in reverse tier order, sync, `uadmin(A_SHUTDOWN, AD_HALT, 0)` |
| PID 1 | `zygctl halt` | As above |
| PID 1 | `zygctl reboot` | Stop services, sync, `uadmin(A_SHUTDOWN, AD_BOOT, 0)` |
| PID 1 | `zygctl poweroff` | Stop services, sync, `uadmin(A_SHUTDOWN, AD_POWEROFF, 0)` |
| not PID 1 | SIGTERM / SIGINT | Stop services, return from `main()` (normal process exit) |
| not PID 1 | `zygctl halt` | Stop services, exit |
| not PID 1 | `zygctl reboot` / `poweroff` | Refuse ("not PID 1") — safety rail for dev/test |

## 4. Service set (BSD-pivot, v2)

15 zyginit services in `enabled.d/` by default (counted across §4.1–§4.6). Replaces the 21-service v1 design (full v1 service table viewable in `hg log -p` for this file) by dropping the configd-coupled daemons in favor of direct command invocation.

### 4.1 Tier 0 — no dependencies

| Service | Type | Script | Stop |
|---|---|---|---|
| `root-fs` | oneshot | `/etc/zyginit/root-fs/start.sh` (ZFS or UFS root: `zfs set readonly=off` or `mount -F ufs -o remount,rw /`) | `/etc/zyginit/root-fs/stop.sh` (matching readonly=on / remount,ro) |

### 4.2 Tier 1 — requires root-fs

| Service | Type | Script | Stop |
|---|---|---|---|
| `devfs` | oneshot | inline: `/usr/sbin/devfsadm` | — |
| `swap` | oneshot | inline: `/sbin/swapadd` (no `-a`; reads `/etc/vfstab`) | `/etc/zyginit/swap/stop.sh` (`swap -d` each active entry) |
| `crypto` | oneshot | `/etc/zyginit/crypto/start.sh` (`cryptoadm refresh`) | — |

### 4.3 Tier 2 — requires tier 1

| Service | Type | Script | Stop |
|---|---|---|---|
| `filesystem` | oneshot | `/etc/zyginit/filesystem/start.sh` (`mountall -l` + `zpool import -a` + `zfs mount -a`) | `/etc/zyginit/filesystem/stop.sh` (`umountall -l` + `zpool sync`) |
| `identity` | oneshot | `/etc/zyginit/identity/start.sh` (`hostname` from `/etc/nodename`) | — |
| `sysconfig` | oneshot | `/etc/zyginit/sysconfig/start.sh` (coreadm, dumpadm, keymap, scheduler, rctl, rmtmpfiles) | — |

### 4.4 Tier 3 — requires filesystem + identity

| Service | Type | Script | Stop |
|---|---|---|---|
| `network` | oneshot | `/etc/zyginit/network/start.sh` — **direct `ifconfig` invocation, no `ipmgmtd`**: plumb lo0, walk `/etc/hostname.<if>` files for each physical interface, `ifconfig <if> dhcp` (which forks `dhcpagent` directly) or static address per file content, install default route from `/etc/defaultrouter` if present | `/etc/zyginit/network/stop.sh` (kill dhcpagent, ifconfig unplumb interfaces) |

**Critical change from v1:** no `dlmgmtd` or `ipmgmtd` services. `ifconfig` plumbs interfaces directly via DLPI. `dhcpagent` is invoked directly via `ifconfig <if> dhcp start`, not via `ipadm -T dhcp`. This matches IRIX 6.5's `/etc/init.d/network` pattern (`IFCONFIG=/usr/etc/ifconfig`, `ROUTE=/usr/etc/route`).

#### `/etc/hostname.<if>` file format (OpenBSD-inspired, illumos-translated)

Verified on hh-prototest: `/sbin/ifconfig` is fully operational without `ipmgmtd` (plumb, address assignment, link state — all work). The OpenBSD `hostname.if(5)` format is line-oriented with each line being ifconfig args. We adopt the same line-oriented convention but translate to illumos's ifconfig syntax, since OpenBSD-specific keywords like `inet alias` map to illumos's `addif`.

Line forms supported by zyginit's `network/start.sh`:

| Line | Action |
|---|---|
| (empty file or whitespace) | `ifconfig <if> plumb up` — interface up, no address |
| `# comment` or blank line | skipped |
| `dhcp` | `ifconfig <if> plumb`; `ifconfig <if> dhcp start` (best effort — see DHCP caveat) |
| `inet <addr>/<prefix>` | `ifconfig <if> inet <addr>/<prefix> up` |
| `inet <addr> netmask <mask>` | `ifconfig <if> inet <addr> netmask <mask> up` |
| `addif <addr>/<prefix>` | additional alias address (illumos `addif`); requires interface already plumbed by an earlier line |
| `inet6 <addr>/<prefix>` | `ifconfig <if> inet6 <addr>/<prefix> up` (deferred to post-first-boot) |
| `up` / `down` | set link state explicitly |

Examples:

```
# /etc/hostname.vioif0 — DHCP via dhcpagent (best effort without ipmgmtd)
dhcp
```

```
# /etc/hostname.vioif0 — static IPv4
inet 192.168.122.50/24
```

```
# /etc/hostname.vioif0 — static IPv4 + alias
inet 192.168.122.50/24
addif 192.168.122.51/24
```

#### Default route

`/etc/defaultrouter` (one IPv4 address per line). `network/start.sh` runs `route -n add default <addr>` for each. Optional — DHCP-acquired routes don't need this file.

#### DHCP caveat

`dhcpagent` is linked against `libipadm.so.1`, which talks to `ipmgmtd`'s door for persistent state. Without `ipmgmtd` running:
- `dhcpagent` itself runs (self-contained DHCP state machine)
- DHCP request/response on the wire works (uses raw sockets, not libipadm)
- **Applying the lease to the interface** uses libipadm → ipmgmtd, which may degrade

Empirical test required on hh-prototest in iter 6 boot. If DHCP-via-ifconfig fails without ipmgmtd, fallback options:
1. Manually invoke `dhcpagent -a -f` (adopt mode + foreground) and configure interface from received lease via direct DLPI ioctl
2. Static IP only for the test VM (configure libvirt to have a known static address, set `/etc/hostname.vioif0` to `inet 192.168.122.50/24`)
3. Patch dhcpagent / replace with dhclient if available

The implementation plan should boot-test with static IP first to derisk the rest of the boot, then attempt DHCP as a follow-up.

### 4.5 Tier 4 — requires network (configd-free system daemons)

All daemons. All use `stop.method = "contract"` (default). `restart.on = "failure"`, `delay = 5`, `max_retries = 3`.

| Service | exec.start | Note |
|---|---|---|
| `syseventd` | `/usr/lib/sysevent/syseventd` | Kernel event delivery; doesn't read SMF properties |
| `syslogd` | `/usr/sbin/syslogd` | Reads `/etc/syslog.conf` directly; SMF-property reads degrade to defaults |
| `utmpd` | `/usr/lib/utmpd` | Cleans stale utmpx entries; no configd dependency |
| `pfexecd` | `/usr/lib/pfexecd` | RBAC profile-exec helper; reads `/etc/security/`, no configd |

### 4.6 Tier 5 — user-facing services

| Service | exec.start | Note |
|---|---|---|
| `cron` | `/etc/zyginit/cron/start.sh` | wrapper: `rm -f /etc/cron.d/FIFO; exec /usr/sbin/cron` |
| `sshd` | `/usr/lib/ssh/sshd -D` | `-D` keeps it foreground; reads `/etc/ssh/sshd_config` |
| `console-login` | `/etc/zyginit/console-login/start.sh` | wrapper exec'ing ttymon with properly-quoted prompt |

### 4.7 Optional / installable — TOMLs ship but symlinks not in `enabled.d/` by default

These TOMLs exist in `services/hammerhead/` but are NOT enabled by default. An admin can enable them via `zygctl enable <svc>` on hardware/configurations where they make sense:

| Service | When to enable |
|---|---|
| `powerd` | Real hardware with `/dev/srn` (not VMs) |
| `fmd` | When fault management is wanted (requires configd workaround or future SMF-coexistence design) |
| `dlmgmtd` / `ipmgmtd` / `inetd` / `rpcbind` | Only after a separate "run minimal SMF infrastructure under zyginit" design is implemented |

### 4.8 Dropped from the v1 design (configd-coupled, requires separate design to revisit)

These daemons require `svc.configd` running for `libscf` calls to succeed. They are intentionally absent from the BSD-pivot v2 service set:

- `dlmgmtd` — datalink management. Replaced by direct `ifconfig` plumbing. Drawback: no `dladm show-link` queries; persistent datalink state across reboots is via `/etc/hostname.<if>` instead.
- `ipmgmtd` — IP interface management. Replaced by direct `ifconfig`/`route` invocation. Drawback: no `ipadm` queries.
- `inetd` — on-demand service launching. Modern illumos `inetd` reads from configd. No replacement for first boot; can be re-added when needed (e.g., via `xinetd` or by accepting configd dependency).
- `rpcbind` — only needed for NFS/RPC; deferred until NFS is in scope.
- `fmd` — fault management daemon; heavy SMF coupling. Deferred.

### 4.9 SMF infrastructure (subsumed; no replacement)

- `svc.startd`, `svc.configd` — replaced by zyginit (or absent in v2)
- `manifest-import`, `early-manifest-import` — no XML manifests
- `boot-archive`, `boot-archive-update` — SMF-specific
- `logadm-upgrade`, `update-man-index` — admin/cron concerns, not init's
- All 7 SMF milestones (virtual groupings) — zyginit's dependency graph replaces these

### 4.10 Currently-disabled SMF services — also not resurrected in zyginit

Mirrors v1: `name-service-cache`, `rbac`, `process-security`, `auditd`, NFS/DNS/LDAP stacks all stay disabled.

## 5. TOML schema usage

Uses the schema already documented in `docs/DESIGN.md` §"Service Definition Format". The following subset is in active use for the 21 services:

```toml
[service]
name = "..."
description = "..."
type = "daemon" | "oneshot"

[exec]
start = "command or /etc/zyginit/<svc>/start.sh"

[dependencies]
requires = ["..."]
after = ["..."]

# Daemons only:
[stop]
method = "contract"          # default; may be omitted
signal = "TERM"              # default; may be omitted
timeout = 60                 # default; may be omitted

[restart]
on = "failure"
delay = 5
max_retries = 3

[contract]
param = ["inherit", "noorphan"]
fatal = ["hwerr"]

# Oneshots with reversal:
[exec]
stop = "/etc/zyginit/<svc>/stop.sh"

[restart]
on = "never"
```

**Defaults and omissions:**
- Omitted `exec.user` / `exec.group` → inherits from zyginit → root when PID 1. Explicit and documented so there's no ambiguity.
- Omitted `[stop]` block → uses contract-signal defaults (TERM, 60s timeout).
- Omitted `[contract]` block → uses template defaults from `setup_template()` in `contract.reef`.

**Deferred schema fields** (supported by zyginit, unused in first boot):
- `exec.user`, `exec.group`, `exec.working_directory`, `exec.environment` — none of the 21 services need privilege drop or per-service env
- `security.*`, `limits.*` — not implemented / not in first boot

**`contract.param = ["inherit", "noorphan"]` note:**
The NOORPHAN flag means that if zyginit-as-PID-1 dies without cleanly abandoning contracts, children receive SIGHUP. This is correct for PID-1 re-exec (kernel sees the same process, no SIGHUP fires). It is **incompatible** with a crash-recovery path that promotes a fresh zyginit PID — but such a path is an explicit non-goal per §2.

## 6. Method script conventions

All scripts live under `/etc/zyginit/<svc>/start.sh` (and optional `stop.sh`) unless the service is trivial enough for an inline TOML command.

**Template:**
```zsh
#!/bin/zsh
# <service> — <one-line purpose>
# Invoked by zyginit via exec.start in /etc/zyginit/<service>.toml

set -e
set -u

# ... steps ...

exit 0
```

**Conventions:**
1. Shebang: `#!/bin/zsh` (consistent with Hammerhead's shell policy; zsh is the system default, ksh93 is being phased out)
2. Error handling: `set -e` (POSIX-style — preferred over `setopt ERR_EXIT` for cross-shell readability) + `set -u`
3. Non-interactive only: no `read`, no tty-dependent behavior
4. Self-contained: no `source`ing of SMF helper scripts (`/lib/svc/share/*.sh`)
5. No SMF API calls: no `svcprop`, `svcadm`, `svccfg`. Config input comes via `exec.environment` in the TOML.
6. Explicit `exit 0` on success (not relying on implicit last-command exit)
7. Use `|| true` deliberately per-command where a specific failure is acceptable (e.g., `dumpadm -d /path || true` if the dump device doesn't exist yet)
8. Idempotent when the action allows: prefer `mount -o remount,rw /` (idempotent) over raw `mount` commands. Use "check then do" patterns for inherently non-idempotent operations (`ipadm show-if lo0 2>/dev/null || ipadm create-if lo0`).

**Execution environment zyginit provides to scripts:**
- Fresh process (fork + exec)
- Working dir: `/` unless TOML sets `exec.working_directory`
- Environment: minimal (PATH, HOME, etc.) plus whatever `exec.environment` specifies
- stdout/stderr: redirected to `/var/run/zyginit/log/<svc>.log`, truncated per start
- User: inherited from zyginit (root at PID 1) unless `exec.user` set

## 7. Shutdown orchestration

Shutdown is a first-class concern, not deferred. Unclean shutdown can corrupt ZFS; we do it right from the first boot.

### 7.1 Shutdown triggers

| Source | Type |
|---|---|
| `zygctl halt` | halt |
| `zygctl reboot` | reboot |
| `zygctl poweroff` | poweroff |
| SIGTERM / SIGINT (PID 1) | halt (conservative default) |
| SIGTERM / SIGINT (not PID 1) | exit (integration test path) |

Shutdown type is communicated to zyginit via the socket protocol (for zygctl commands) or inferred from signal source.

### 7.2 Shutdown walk

1. Set `shutdown_requested = true`, `shutdown_type = <type>`
2. Break out of main event loop
3. Walk services **in reverse tier order** (not insertion order)
4. For each service in the tier:
   - **Daemons** in `STATE_RUNNING`: `stop_service()` → SIGTERM via contract, wait for contract EMPTY event, mark `STATE_STOPPED`
   - **Oneshots** in `STATE_STOPPED` with `exec.stop` defined (or `/etc/zyginit/<svc>/stop.sh` present): fork/exec the stop command; wait for exit
   - **Oneshots** without stop: skip (no reversal defined)
5. Wait for all services in tier N to reach `STATE_STOPPED` before moving to tier N-1
6. After tier 0 is drained: `sync()` (flush pending writes)
7. Call `uadmin(A_SHUTDOWN, <fcn>, 0)`:
   - `AD_HALT` (0) — halt, operator decides next step physically
   - `AD_BOOT` (1) — reboot immediately
   - `AD_POWEROFF` (6) — ACPI/BMC chassis off

### 7.3 Stop scripts concretely required

| Service | stop.sh action |
|---|---|
| `root-fs` | `mount -o remount,ro /` (called after filesystem/stop.sh has umounted everything else) |
| `filesystem` | `umountall -l` |
| `swap` | `swap -d` each entry |

Other services either use the contract-signal daemon default or have nothing to reverse.

### 7.4 Required code changes

**`supervisor.reef`:**
- Extend shutdown path to walk tiers in reverse
- For each tier: first send stops to all daemons, wait for them to exit, then run stop scripts for oneshots that have them
- Currently `shutdown_services` skips `STATE_STOPPED` services — this must change

**`main.reef`:**
- Track `shutdown_type` variable (halt / reboot / poweroff)
- After `shutdown_services()`: call `sync()`, then `zyginit_shutdown(<fcn>)`

**FFI: `helpers.c` and `contract.reef` (or new `shutdown.reef`)**:
- `int zyginit_shutdown(int fcn)` → wraps `uadmin(A_SHUTDOWN, fcn, 0)`
- `extern "C" fn zyginit_shutdown(fcn: int): int`

**Linux stub**: `contract_linux_stubs.c` gets a matching stub returning -1 (should never actually fire on Linux).

**`zygctl`** gets three new subcommands: `halt`, `reboot`, `poweroff`. Each sends a specific command over the socket; zyginit sets `shutdown_type` accordingly.

**`zygctl`** also gets two stub subcommands: `single`, `multiuser`. Both print "not yet implemented — reserved for future runlevel support" and exit 1. Documented in `zygctl --help` so the surface is stable.

## 8. Rollout phases

### 8.1 Phase A — non-PID-1 dry run

Run zyginit on the live hh-prototest system alongside SMF, using a scratch config directory.

```sh
ZYGINIT_CONFIG_DIR=/tmp/zyginit-dry \
  /path/to/zyginit
```

**Goals:**
- TOMLs parse cleanly
- Dependency graph builds without cycles; tier order matches the spec
- zyginit supervises one or two non-conflicting test daemons (e.g., a second sshd on port 2222)
- Clean shutdown via SIGTERM works without uadmin (non-PID-1 path)

**Out of scope for Phase A:**
- Destructive oneshots (filesystem, network, devfs) — they'd conflict with SMF-managed copies. Skip these in the dry run.

### 8.2 Phase B — PID-1 boot in a fresh Boot Environment

1. `beadm create zyginit`
2. `beadm mount zyginit /mnt`
3. Install into `/mnt`:
   - `/mnt/sbin/zyginit`, `/mnt/sbin/zygctl`
   - `/mnt/etc/zyginit/*.toml` (21 files)
   - `/mnt/etc/zyginit/<svc>/start.sh` + `stop.sh` as specified above
   - `/mnt/etc/zyginit/enabled.d/` symlinks for all 21
   - Preserve old init: `mv /mnt/sbin/init /mnt/sbin/init.smf`
   - New init: `ln -s /sbin/zyginit /mnt/sbin/init`
4. `beadm umount zyginit; beadm activate zyginit; init 6`

### 8.3 Phase C — SMF removal (deferred, separate milestone)

Optional follow-up once Phase B is stable:
- In the zyginit BE: remove `/lib/svc/bin/svc.startd`, `/lib/svc/bin/svc.configd`, `/lib/svc/method/*`, `/etc/svc/`
- Activate, reboot, prove the system still works without SMF
- This is when "zyginit replaces SMF" is literally true.

### 8.4 Failure recovery ladder

In order of cost:

1. `zygctl reboot` → pick `hammerhead` BE from GRUB menu (seconds)
2. If GRUB menu inaccessible: boot into rescue media / single-user from install media, mount original BE, activate it
3. If the pool is damaged: `sudo cp hh-prototest-backup-2026-04-24-pre-zyginit-pid1.qcow2 hh-prototest.qcow2` on the dev host (2-3 min with VM shut down)
4. Last resort: VM rebuild from scratch (~20 min)

Disk-level backup captured at rev 21. Recovery drill is part of §9.F.

## 9. Success criteria for Phase B

Literal checklist run after first boot into the zyginit BE.

### A. Boot completes
- [ ] No `kernel panic: init died`
- [ ] `zyginit: entering event loop` appears on console
- [ ] Every tier finishes within 60s total
- [ ] Console login prompt appears on `/dev/console`

### B. Expected service states
- [ ] All 21 services present in `zygctl list`
- [ ] Tier 0-3 oneshots: `STOPPED` with `exit=0`
- [ ] Tier 3-5 daemons: `RUNNING`
- [ ] No service in `MAINTENANCE`

### C. Services actually functional
- [ ] `ssh zygaena@192.168.122.197` from dev host
- [ ] `logger -p user.info test` → visible in `/var/adm/messages` (syslogd)
- [ ] `dladm show-link` lists physical interface `up`
- [ ] `ipadm show-addr` lists lo0 + physical interface with IPv4 addresses
- [ ] `ping 192.168.122.1` succeeds
- [ ] `crontab -l` runs for root
- [ ] `zfs list` works, `hhtest` pool mounted

### D. zygctl end-to-end
- [ ] `zygctl status cron` shows `RUNNING` with uptime > 0
- [ ] `zygctl stop cron` → `STOPPED`, then `zygctl start cron` → `RUNNING`
- [ ] `zygctl log sshd` returns without error
- [ ] `zygctl reload` succeeds (SIGHUP path)

### E. Clean shutdown
- [ ] `zygctl halt`: reverse-tier-order stop output in console; system halted cleanly
- [ ] `zygctl reboot`: cycles; comes back via zyginit again
- [ ] `zygctl poweroff`: VM transitions to `shut off` in libvirt
- [ ] After any zygctl shutdown: `zpool status` clean, no replay-required flags

### F. Rollback confirmed
- [ ] 3 clean reboot cycles into the zyginit BE
- [ ] Reboot into original `hammerhead` BE from GRUB → SMF comes up normally
- [ ] `beadm activate hammerhead` from zyginit BE → reboot → lands on `hammerhead` BE
- [ ] Disk-level restore drill executed at least once (time to recovery measured)

### G. Explicit non-requirements
- No boot-time performance gate (measured for baseline only)
- No stability soak
- No DNS / hostname resolution testing
- No NTP / time sync
- No name services (NSS beyond /etc/passwd, /etc/hosts, /etc/group)
- No `zygctl single` / `zygctl multiuser` functionality (stubs only)
- No SMF removal (Phase C)

## 10. Observability during test

- Console access via `virsh console hh-prototest` for live boot watching.
- `/var/log/zyginit/<svc>.log` per-service logs (rev 48 changed default from `/var/run/zyginit/log` to `/var/log/zyginit` so logs survive reboot).
- **Out-of-band log access:** with VM shut down, mount the BE's qcow2 from the dev host:
  ```
  virsh destroy hh-prototest
  sudo qemu-nbd -c /dev/nbd0 -r /var/lib/libvirt/images/hh-prototest.qcow2
  sudo zpool import -R /tmp/hh-mnt -o readonly=on -N -f -d /dev/nbd0p2 hhtest
  sudo zfs mount hhtest/ROOT/zyginit
  # logs at /tmp/hh-mnt/var/log/zyginit/
  ```
  Restore: `zfs umount`, `zpool export hhtest`, `qemu-nbd -d /dev/nbd0`. Proven across 3 iterations.
- Capture console screenshot at each iter for visual cross-check.

## 11. Explicit deferrals (captured for future work)

Each of the following is a known follow-up, documented here so future milestones inherit the boundary:

- **configd-coupled daemons re-add**: if `dlmgmtd`/`ipmgmtd`/`inetd`/`rpcbind`/`fmd` are needed, design either a "minimal SMF infrastructure under zyginit" (configd as a zyginit-supervised daemon) or a per-daemon configd-bypass shim.
- **Runlevel targets**: `zygctl single`, `zygctl multiuser` stubs shipped; implementation needs a target-set schema extension (e.g., `service.runlevels = ["single", "multiuser"]`).
- **Milestone-style virtual services**: systemd-target equivalent; probably needed to implement runlevels cleanly.
- **`/sbin/halt`, `/sbin/reboot`, `/sbin/poweroff`, `/usr/sbin/shutdown` wrappers**: replace with versions that route through `zygctl`.
- **`init <n>` runlevel compatibility**: unsupported; use zygctl.
- **Crash-recovery re-adoption**: architecturally not supported (illumos descendant requirement); documented.
- **NOORPHAN revisit**: current template is correct for PID-1 re-exec; revisit only if a non-exec crash-recovery goal is added.
- **GC tuning under sustained uptime**: track separately.
- **SMF removal from disk**: Phase C — once zyginit is stable, optionally remove `/lib/svc/`, `/etc/svc/` from the BE entirely.
- **Boot performance targets**: no gate.
- **Additional services** (resource limits, audit, DNS, NFS, zones): add as needed post-first-boot.
- **Hostid persistence**: first boot reads `/etc/hostid`; generation logic (for first-ever boot) not yet implemented.
- **Modern datalink/IP semantics**: BSD-pivot loses `dladm`/`ipadm` query interfaces. For zone hosts, laptops, multi-homed servers, this design is insufficient; revisit when those use cases are in scope.

## 12. Pre-requisites completed before this spec

- **Contract probe**: 16/16 FFI probes pass on hh-prototest (rev 19)
- **contract.reef bugs fixed**: P_CTID 14→13, ctl fd O_WRONLY (rev 18)
- **Path rename**: `/etc/zyginit.d/` → `/etc/zyginit/` (rev 20, 21)
- **Signal handling fix**: dead-code flag-clearing block removed (rev 15)
- **Build split**: `helpers.c` / `contract_linux_stubs.c` separated (rev 16)
- **Disk backup**: `hh-prototest-backup-2026-04-24-pre-zyginit-pid1.qcow2` captured on dev host

**Pre-requisites added during iter 1-5 of v1 (kept in v2):**
- **fd-setup at PID 1** (rev 45): `setup_pid1_console()` opens `/dev/console` and dup2's onto 0/1/2 before any I/O
- **argv[0] fix** (rev 44): supervisor.start_service includes program name as argv[0]
- **uadmin/sync wiring** (rev 24-31): full PID-1 shutdown path
- **Tier-ordering enforcement** (rev 47): supervisor waits for tier N to settle before starting N+1
- **Persistent log dir** (rev 48): `/var/log/zyginit/` default
- **root-fs ZFS/UFS dispatch** (rev 43): correct readonly toggle per fstype
- **zygctl shutdown commands** (rev 33-34): halt/reboot/poweroff + single/multiuser stubs
- **Path corrections in scripts** (rev 49-50): `/bin/hostname`, `/bin/rm`, `/sbin/swapadd` (no `-a`), `/sbin/dumpadm`, etc.

## 13. Open questions / defer to implementation

Most v1 open questions are now answered (uadmin constants verified; ipmgmtd path verified; ttymon argv conventions known). v2 open questions answered through 2026-04-25 hh-prototest probing:

- ✅ **`ifconfig` capability without ipmgmtd**: `ifconfig lo0:99 plumb`, `ifconfig lo0:99 inet 127.99.0.1 netmask 255.0.0.0 up`, and `ifconfig lo0:99 unplumb` all worked silently while ipmgmtd was running. `ifconfig` operates at the DLPI/kernel-ioctl layer for plumbing; ipmgmtd integration is for *persistent state*, not for runtime configuration.
- ✅ **`ifconfig <if> dhcp <subcommand>` syntax**: per ifconfig(8) man page on hh-prototest, supported subcommands are `start | stop | drop | extend | inform | ping | release | status`. `ifconfig vioif0 dhcp status` returned valid info while pinging the running dhcpagent on TCP 4999.
- ✅ **`/etc/hostname.<if>` parsing**: Hammerhead's `ifconfig` does NOT read this file directly. The file is consumed by SMF method scripts. zyginit's `network/start.sh` will parse the file itself (line-oriented format documented in §4.4).
- ✅ **`/etc/defaultrouter`**: empty/absent on hh-prototest (DHCP supplies the route). Format: one IPv4 address per line. `network/start.sh` reads when present and DHCP isn't in use.

Remaining for iter 6 to empirically verify:

- ⚠ **DHCP-via-dhcpagent without ipmgmtd running**: dhcpagent links libipadm which talks to ipmgmtd. Plan: try `ifconfig <if> dhcp start` after starting dhcpagent fresh under zyginit; if it works, ship as default; if it fails, ship static-IP-only and document. The test VM has a libvirt-supplied dnsmasq DHCP server on 192.168.122.0/24 — perfect environment to verify.
- ⚠ **First-boot fallback**: if DHCP fails, our test config will use a static IP (e.g., `inet 192.168.122.50/24`) chosen outside libvirt's dnsmasq pool to avoid conflicts.