# zyginit — Architecture and Design **Status:** Implementation complete — testing on Hammerhead pending **Date:** 2026-03-05 **RFE Filed:** `reef-lang/bugs/RFE-001-init-system-requirements.md` **Reef 0.4.0:** Released 2026-03-05 — all 8 RFE-001 items implemented ## Problem Statement SMF (Service Management Facility) is architecturally overcomplex for Hammerhead's needs: - **XML service definitions** — verbose, hard to read/write, depends on libxml2 - **libxml2 dependency** — maintenance status uncertain, large dependency for service config - **Architectural complexity** — 3 cooperating daemons (svc.startd, svc.configd, master restarter) plus SQLite repository, door-based IPC, and XML-based service manifests - **Difficult to modify** — tightly coupled components make changes risky ## Proposed Replacement: zyginit A single-process service manager and supervisor written in Reef, with: - **TOML-based service definitions** — clean, readable, no XML - **Symlink-based enable/disable** — no database, inspectable with `ls` - **Dependency graph resolution** with parallel startup - **Service supervision** with automatic restart - **Hammerhead process contract integration** — kernel-level tracking of all service descendants - **Administrative CLI** (`zygctl`) for service control via Unix domain socket ### Naming | Component | Name | Description | |-----------|------|-------------| | Daemon | `zyginit` | PID 1, the init process | | CLI | `zygctl` | Admin tool (start/stop/status/enable/disable) | | Config dir | `/etc/zyginit/` | Service TOML definitions | | Enable dir | `/etc/zyginit/enabled.d/` | Symlinks to enabled services | | Socket | `/var/run/zyginit.sock` | Unix domain socket for zygctl | | Diagnostics | `/var/run/zyginit/` | Optional runtime state (tmpfs) | ### Why Reef - Compiles to native code via C/GCC — suitable for PID 1 - FFI to C is clean — can call libcontract and all POSIX APIs directly - Result/Option error handling is correct for systems code - Writing the init in Reef differentiates Zygaena/Hammerhead - Reef already powers the Coral package manager — proven for systems programming ## Architecture ``` +-----------+ | zygctl | (Reef CLI tool) +-----+-----+ | Unix socket (/var/run/zyginit.sock) +-----v-----+ | zyginit | (PID 1, Reef) | | | +-------+ | | | event | | poll() on: contract bundle fd, | | loop | | unix socket fd, signal pipe fd | +---+---+ | | | | | +---v---+ | | | dep | | topological sort of service graph | | graph | | built from TOML at boot | +---+---+ | | | | | +---v--------+ | | | contract | | libcontract FFI: | | manager | | template → fork → adopt → events | +------------+ | +-------+-------+ | +-------+-------+-------+-------+ | | | | | sshd cron fmd network ... (contract) (contract) (contract) (contract) ``` ## Service Definition Format (TOML) Service definitions are TOML files in `/etc/zyginit/`. Most services are purely declarative. For complex startup logic (database initialization, network setup), the `exec.start` field can point to a shell script in the service's subdirectory. ### Simple service (declarative) ```toml # /etc/zyginit/sshd.toml [service] name = "sshd" description = "Secure Shell Daemon" type = "daemon" # daemon | oneshot | transient [exec] start = "/usr/lib/ssh/sshd -D" [stop] method = "contract" # contract (default) | exec signal = "TERM" timeout = 60 [dependencies] requires = ["network", "filesystem"] after = ["name-services"] conflicts = ["other-mta"] # mutually exclusive; never run together [contract] param = ["inherit", "noorphan"] fatal = ["empty", "hwerr"] [restart] on = "failure" # always | failure | never delay = 5 max_retries = 3 ``` ### Complex service (script escape hatch) For services requiring complex startup logic, the TOML definition points to scripts in a subdirectory matching the service name: ```toml # /etc/zyginit/postgresql.toml [service] name = "postgresql" description = "PostgreSQL Database Server" type = "daemon" [exec] start = "/etc/zyginit/postgresql/start.sh" stop = "/etc/zyginit/postgresql/stop.sh" [dependencies] requires = ["filesystem", "network"] ``` ``` /etc/zyginit/ ├── postgresql.toml └── postgresql/ ├── start.sh # handles initdb, pg_upgrade, etc. └── stop.sh # clean shutdown with pg_ctl ``` ### TOML schema fields (planned — needs further discussion) Additional fields under consideration: - `exec.user`, `exec.group` — privilege separation (run as non-root) - `exec.working_directory` — chdir before exec - `exec.environment` — env vars: `["KEY=value", ...]` - `security.zone`, `security.chroot` — containment - `limits.max_fds`, `limits.max_memory` — resource controls - `logging.stdout`, `logging.stderr` — output destination (file, syslog) - `service.milestone` — grouping: `single-user`, `multi-user`, `network` ## State Management ### Design Rationale Research of other init systems informed the design: | System | Enable/Disable | State DB | Boot Order | |--------|---------------|----------|------------| | **systemd** | Symlinks in `.wants/` dirs | None | In-memory from unit files | | **dinit** | Symlinks in `waits-for.d/` | None | In-memory from service files | | **OpenRC** | Symlinks in runlevel dirs | None | In-memory (cached to `/run`) | | **SMF** | SQLite property | SQLite | In-memory from repository | Every modern init except SMF uses symlinks for enable/disable state and builds the dependency graph in memory at boot. SMF's SQLite repository is the #1 source of boot failures ("repository corruption"). zyginit follows the simpler pattern. ### Enabled/Disabled State (symlinks) Enabled state is managed via symlinks in `/etc/zyginit/enabled.d/`: ``` /etc/zyginit/ ├── sshd.toml ├── cron.toml ├── network.toml ├── filesystem.toml ├── postgresql.toml ├── postgresql/ │ ├── start.sh │ └── stop.sh └── enabled.d/ # symlinks = what starts at boot ├── sshd → ../sshd.toml ├── cron → ../cron.toml ├── network → ../network.toml └── filesystem → ../filesystem.toml ``` | Operation | What happens | |-----------|-------------| | `zygctl enable sshd` | Creates symlink `enabled.d/sshd → ../sshd.toml` | | `zygctl disable sshd` | Removes symlink `enabled.d/sshd` | | `zygctl start sshd` | Starts service now (transient — does not persist across reboot) | | `zygctl stop sshd` | Stops service now | | `zygctl status` | Reads in-memory state from zyginit over Unix socket | Benefits: - Inspectable with `ls /etc/zyginit/enabled.d/` - Editable with `ln -s` and `rm` — no special tools required - Always on root filesystem — available at PID 1 startup - No corruption risk — no database to corrupt ### Boot Order (derived from TOML) No pre-computed boot order is stored. At boot, zyginit: 1. Scans `enabled.d/` symlinks → list of enabled services 2. Reads each service's TOML → extracts `[dependencies]` 3. Topological sort → parallel startup order 4. Starts services respecting dependency constraints For <100 services, TOML parsing and topo sort complete in milliseconds. ### Runtime State (in-memory + optional tmpfs) Service runtime state (PIDs, running/stopped/failed, restart counters, exit codes) lives **in memory** inside the zyginit process. Accessible via `zygctl status` over the Unix domain socket. Optionally, zyginit writes diagnostic state to `/var/run/zyginit/` (tmpfs) for crash recovery analysis. This is not required for operation — if zyginit restarts, it re-scans TOML definitions and re-adopts existing process contracts. ### Filesystem Summary | Path | Filesystem | Available | Purpose | |------|-----------|-----------|---------| | `/etc/zyginit/*.toml` | root | Always | Service definitions | | `/etc/zyginit/enabled.d/` | root | Always | Enabled symlinks | | `/var/run/zyginit.sock` | tmpfs | After tmpfs mount | CLI communication | | `/var/run/zyginit/` | tmpfs | After tmpfs mount | Diagnostic state (optional) | ## Hammerhead Process Contracts Process contracts are a Hammerhead kernel mechanism (not available on Linux) accessed via the contract filesystem (`/system/contract/`): 1. Before fork(), open `/system/contract/process/template` and configure 2. After fork(), all child processes (and their children) belong to a contract 3. Contract owner receives events: empty (all exited), core dump, hardware error, fatal signal 4. Can signal ALL processes in a contract atomically — no orphaned grandchildren 5. On init restart, existing contracts are re-adopted — services keep running The C API (libcontract) is fd-based: open/ioctl/close on ctfs paths. No complex struct layouts — integer fds, integer flags, and string FMRIs. Clean for Reef FFI. ## Boot Sequence 1. Kernel execs `/sbin/init` → zyginit (PID 1) 2. Scan `/etc/zyginit/enabled.d/` → list of enabled services 3. Read each service's TOML → extract `[dependencies]` 4. Topological sort → boot order 5. Start services in parallel, respecting dependency order 6. Mount tmpfs, create `/var/run/zyginit.sock` 7. Enter poll(2) event loop: contract events, Unix socket, signal pipe ## Reef Language Requirements Filed as **RFE-001** with the Reef team (`~/repos/reef-lang/bugs/RFE-001-init-system-requirements.md`). **Response received** (`reef-lang/bugs/RFE-001-response.md`): All 8 items accepted for Reef 0.4.0. Estimated ~4 weeks. Key findings from Reef team: - **GC is not a blocker** — no auto-triggered STW pauses; signal change is 2 lines - **fork/exec already exist** in C runtime — just need stdlib wiring - **TOML parser already exists** — `encoding/toml.reef` (~560 lines) - **Cooperative yield points** already exist (`--quantum-checks` flag) ### Revised Priority (per Reef team analysis) | # | Enhancement | Effort | Status | |---|------------|--------|--------| | 1 | Process management (fork/exec/setsid) | Low | Reef 0.4.0 | | 2 | File descriptor operations | Low | Reef 0.4.0 | | 3 | String array marshaling | Low-Med | Reef 0.4.0 | | 4 | Signal enhancements | Low-Med | Reef 0.4.0 | | 5 | Unix domain sockets | Low | Reef 0.4.0 | | 6 | Event loop / poll | Low | Reef 0.4.0 | | 7 | TOML parser enhancements | Minimal | Reef 0.4.0 | | 8 | GC mode (configurable signals + --no-gc) | Medium | Reef 0.4.0 | ### Open Questions (from Reef team — resolved) 1. **poll(2) vs event ports**: poll(2) in stdlib, event ports via `extern "C"`. Poll is sufficient for 3-4 fds. 2. **fork() + Active Objects**: Single-threaded poll loop — no AO conflict. 3. **GC signal preference**: SIGRTMIN+0/SIGRTMIN+1 — no collision with standard signals. ### Reef Team Recommendations 1. Use `extern "C"` for Hammerhead-specific code (libcontract, ctfs) — keep in Hammerhead, not Reef stdlib 2. Start prototyping now with `process_spawn_shell()` + FFI declarations 3. Test with current GC — likely zero pauses for bounded working set 4. Use existing TOML parser (`import encoding.toml`) immediately ## Phased Approach ### Phase A: Interim (Now) - Keep SMF operational — it works, it's built, it boots - libxml2 now builds from source in hammerhead (commit 3b7533657) - Begin zyginit prototype using existing Reef 0.4.0 capabilities - Test TOML parser with service definition format ### Phase B: Prototype - Build zyginit prototype with basic service supervision - Test with a handful of services on standalone VM - Keep SMF as fallback ### Phase C: Migration - Convert all SMF service manifests to TOML definitions - Boot with zyginit as PID 1 - Validate with hh-audit ### Phase D: SMF Removal - Remove svc.startd, svc.configd, master restarter - Remove libscf, libxml2 dependency (if no other consumers) - Remove SMF manifest infrastructure from build ## FMA Integration FMA (Fault Management Architecture) is independent of SMF and will be retained: - Kernel drivers post ereport events via sysevent - fmd daemon runs diagnosis engines (CPU retire, disk fault, ZFS degraded) - fmd is a service managed by the init system — works with any init - No architectural coupling to SMF ## References - Hammerhead `libcontract(3LIB)`, `contract(5)`, `process(5)` - `cmd/svc/startd/contract.c`, `method.c`, `restarter.c` - dinit comparison: https://davmac.org/projects/dinit/ - Reef RFE: `~/repos/reef-lang/bugs/RFE-001-init-system-requirements.md` - Reef 0.4.0 changelog: `~/repos/reef-lang/docs/changelog-0.4.0.md` # zyginit Implementation Plan **Status:** Complete — All 7 phases implemented, plus post-phase enhancements **Date:** 2026-03-06 **Author:** Chris Tusa / Claude ## Project Layout Following Coral's conventions (`source_dirs = ["src"]`, `entry = "src/main.reef"`): ``` zyginit/ ├── reef.toml # Reef manifest ├── SRCHEADER.txt # Leafscale source header template ├── CLAUDE.md # Project context ├── .hgignore # Mercurial ignore ├── src/ │ ├── main.reef # Entry point — event loop (Phase 5) │ ├── config.reef # TOML parser + ServiceDef types (Phase 1) │ ├── depgraph.reef # Dependency graph + topo sort (Phase 2) │ ├── contract.reef # Hammerhead process contract FFI (Phase 3) │ ├── supervisor.reef # Service lifecycle management (Phase 4) │ └── socket.reef # Unix domain socket server (Phase 6) ├── docs/ │ ├── DESIGN.md # Architecture and design │ └── IMPLEMENTATION_PLAN.md # This file ├── services/ │ └── examples/ # Example TOML service definitions ├── tests/ │ └── (test TOML files, test programs) ├── stub/ # Original Rust-like stubs (design reference only) └── tools/ └── zygctl/ # Separate CLI binary ├── reef.toml └── src/ └── main.reef ``` ## Phase Dependency Graph ``` Phase 1: config.reef (TOML parsing, ServiceDef types) │ ▼ Phase 2: depgraph.reef (dependency graph, topo sort) │ ├──────────────────────┐ ▼ ▼ Phase 3: contract.reef Phase 6: socket.reef │ │ ▼ │ Phase 4: supervisor.reef ◄─┘ │ ▼ Phase 5: main.reef (event loop wiring) │ ▼ Phase 7: tools/zygctl (separate binary) ``` ## Phase 1: config.reef — TOML Config Parser **Purpose:** Parse `/etc/zyginit/*.toml` service definitions into ServiceDef structs. **Testable on Linux:** Yes — fully portable. ### Imports ```reef import encoding.toml as toml import io.file as file import core.str ``` ### Types ```reef type ServiceDef = struct name: string description: string service_type: int // 1=daemon, 2=oneshot, 3=transient start_cmd: string stop_cmd: string // empty if not specified stop_method: int // 1=contract, 2=exec stop_signal: string // "TERM", "HUP", etc. stop_timeout: int // seconds, default 60 requires: [string] // hard dependencies requires_count: int after: [string] // ordering dependencies after_count: int contract_param: [string] contract_param_count: int contract_fatal: [string] contract_fatal_count: int restart_policy: int // 1=always, 2=failure, 3=never restart_delay: int // seconds, default 5 max_retries: int // -1 = unlimited, default 3 end ServiceDef ``` ### Functions - `new_service_def(): ServiceDef` — create with defaults - `parse_toml_array(value, result, max): int` — parse `["a", "b"]` inline arrays - `parse_service_type(type_str): int` — "daemon"→1, "oneshot"→2, "transient"→3 - `parse_restart_policy(policy_str): int` — "always"→1, "failure"→2, "never"→3 - `parse_stop_method(method_str): int` — "contract"→1, "exec"→2 - `parse_service(filepath, svc): bool` — parse one TOML file into ServiceDef - `scan_enabled(config_dir, paths, max): int` — list enabled.d/ entries - `load_enabled_services(config_dir, services, max): int` — load all enabled services ### Key Implementation Notes - Reef's TOML parser flattens sections: `[service]` + `name = "sshd"` → key `"service.name"` - Inline arrays like `requires = ["a", "b"]` need custom parsing (Coral has proven pattern) - No readlink needed: symlink name in enabled.d maps directly to `.toml` ## Phase 2: depgraph.reef — Dependency Graph + Topological Sort **Purpose:** Build DAG from service dependencies, compute parallel boot tiers. **Testable on Linux:** Yes — fully portable. ### Imports ```reef import collections.hashmap as hashmap import core.str ``` ### Types ```reef type DepGraph = struct names: [string] count: int name_map: hashmap.HashMap[int] // name → index deps: [int] // flattened adjacency (MAX_SERVICES * MAX_DEPS) dep_count: [int] // deps per node in_degree: [int] // for topo sort end DepGraph ``` ### Functions - `new_depgraph(): DepGraph` - `add_service(graph, name): int` — returns assigned index - `add_dependency(graph, name, dep_name): bool` - `build_graph(graph, services, count): bool` — populate from ServiceDef array - `topo_sort(graph, tiers, tier_counts, max_tiers): int` — Kahn's algorithm - `has_cycle(graph): bool` ### Algorithm: Kahn's Topological Sort 1. Find all nodes with in_degree == 0 → tier 0 2. Remove those nodes, decrement in_degree of dependents 3. Nodes reaching in_degree 0 form next tier 4. Repeat until all placed 5. Remaining nodes with in_degree > 0 → cycle error Output: flat `tiers[string]` array + `tier_counts[int]` array. Services within a tier can start in parallel. ## Phase 3: contract.reef — Hammerhead Process Contract FFI **Purpose:** Reef wrappers around libcontract for kernel-level service process tracking. **Testable on Linux:** No — Hammerhead only. Compile-test only on Linux. ### FFI Declarations ```reef extern "C" fn ct_tmpl_activate(fd: int): int extern "C" fn ct_tmpl_clear(fd: int): int extern "C" fn ct_tmpl_set_informative(fd: int, events: int): int extern "C" fn ct_tmpl_set_critical(fd: int, events: int): int extern "C" fn ct_pr_tmpl_set_fatal(fd: int, events: int): int extern "C" fn ct_pr_tmpl_set_param(fd: int, params: int): int extern "C" fn ct_status_read(fd: int, detail: int, statp: pointer): int extern "C" fn ct_status_free(statp: pointer) extern "C" fn ct_status_get_id(statp: pointer): int extern "C" fn ct_ctl_abandon(fd: int): int extern "C" fn ct_ctl_adopt(fd: int): int extern "C" fn sigsend(idtype: int, id: int, sig: int): int ``` ### Types ```reef type Contract = struct id: int service_name: string bundle_fd: int end Contract ``` ### Functions - `setup_template(): int` — open/configure/activate contract template - `get_latest_contract(): int` — read contract ID after fork - `clear_template(tmpl_fd): int` - `open_bundle(): int` — open bundle fd for poll() - `kill_contract(contract_id, sig): int` — signal all processes in contract - `abandon_contract(contract_id): int` - `adopt_contract(contract_id): int` — re-adopt after zyginit restart ### Build Note Requires `-l contract` linker flag on Hammerhead. ### Risk: ct_status_read pointer semantics `ct_status_read` takes pointer-to-opaque-pointer. Needs `unsafe` block and careful FFI handling. Must prototype on Hammerhead VM. ## Phase 4: supervisor.reef — Service Lifecycle Management **Purpose:** Fork/exec services within contracts, manage state machine, restart policies. **Testable on Linux:** Partial — fork/exec works, contracts do not. ### Imports ```reef import config import contract import sys.process as process import sys.fd as fd import sys.signal as signal import collections.hashmap as hashmap import core.str ``` ### Types ```reef // State constants: DISABLED=0, WAITING=1, STARTING=2, RUNNING=3, // STOPPING=4, STOPPED=5, FAILED=6, MAINTENANCE=7 type ServiceRuntime = struct def: config.ServiceDef state: int contract_id: int // -1 if none pid: int // -1 if none restart_count: int last_exit_code: int // -1 if not exited last_start_time: int stop_requested_time: int end ServiceRuntime type ServiceTable = struct runtimes: [ServiceRuntime] count: int name_map: hashmap.HashMap[int] // name → index contract_map: hashmap.HashMap[int] // contract_id_str → index end ServiceTable ``` ### Functions - `new_service_table(max): ServiceTable` - `add_service(table, def): int` - `find_service(table, name): int` - `find_by_contract(table, contract_id): int` - `start_service(table, idx): bool` — fork, contract setup, exec - `stop_service(table, idx): bool` — signal or exec stop command - `handle_contract_event(table, contract_id, exit_code)` — apply restart policy - `check_stop_timeouts(table)` — escalate to SIGKILL if timeout exceeded - `check_restart_delays(table)` — restart services whose delay has elapsed - `state_name(state): string` — human-readable state - `get_status_line(table, idx): string` ### start_service() Flow 1. `contract.setup_template()` → template fd 2. `process.process_fork()` → pid 3. Child: `process.process_setsid()`, parse args, `process.process_exec()` 4. Parent: `contract.get_latest_contract()` → contract_id 5. `contract.clear_template(tmpl_fd)` 6. Update runtime: state=RUNNING, pid, contract_id, timestamp ### Restart Policy Logic (handle_contract_event) - `RESTART_ALWAYS` → schedule restart after delay - `RESTART_FAILURE` → restart only if exit_code != 0 - `RESTART_NEVER` → mark STOPPED - Check max_retries → MAINTENANCE if exceeded ## Phase 5: main.reef — Event Loop **Purpose:** Wire all modules together into the main poll(2) event loop. **Testable on Linux:** Partial — poll/signals/sockets yes, contracts no. ### Imports ```reef import config import depgraph import contract import supervisor import socket import sys.process as process import sys.signal as signal import sys.poll as poll import sys.fd as fd ``` ### Main Procedure Flow 1. Load config: `config.load_enabled_services()` 2. Build dependency graph: `depgraph.build_graph()` + `depgraph.topo_sort()` 3. Create service table: `supervisor.new_service_table()` 4. Start services tier by tier 5. Set up event sources: - Contract bundle fd: `contract.open_bundle()` - Signal self-pipe: `signal.selfpipe_create()` for SIGCHLD, SIGTERM, SIGHUP - Unix socket: `socket.create_socket()` 6. Main loop: `poll.poll_wait()` with 1s timeout - Contract events → `supervisor.handle_contract_event()` - Signals → reap children, shutdown, reload - Socket → `socket.handle_client()` - Periodic: check stop timeouts, restart delays ### Build Command ```bash reefc build --gc-signals rtmin # Linux dev reefc build --gc-signals rtmin -l contract # Hammerhead production ``` ## Phase 6: socket.reef — Unix Domain Socket Server **Purpose:** Accept zygctl connections, dispatch commands, send responses. **Testable on Linux:** Yes — fully portable. ### Imports ```reef import net.unix as unix import supervisor import config import core.str ``` ### Protocol - **Request:** `COMMAND [ARG]\n` (newline-terminated text) - **Response:** Plain text lines, connection closed after response ### Commands | Command | Description | |---------|-------------| | `status` | List all services with state | | `status ` | Status of one service | | `start ` | Start a service (transient) | | `stop ` | Stop a service | | `restart ` | Stop then start | | `enable ` | Create symlink in enabled.d | | `disable ` | Remove symlink from enabled.d | | `list` | List all known service definitions | ### Functions - `create_socket(): int` — bind and listen on `/var/run/zyginit.sock` - `handle_client(server_fd, table)` — accept, read, dispatch, respond, close - `parse_command(input, arg_out): string` - `cmd_status(table, arg): string` - `cmd_start(table, name): string` - `cmd_stop(table, name): string` - `cmd_restart(table, name): string` - `cmd_enable(name): string` - `cmd_disable(name): string` - `cmd_list(table): string` - `destroy_socket()` — cleanup ## Phase 7: tools/zygctl — CLI Tool **Purpose:** Admin CLI that connects to zyginit over Unix domain socket. **Testable on Linux:** Yes. ### Layout ``` tools/zygctl/ ├── reef.toml # entry = "src/main.reef", output = "zygctl" └── src/ └── main.reef ``` ### Imports ```reef import sys.args as args import net.unix as unix import core.str ``` ### Usage ``` zygctl status # list all services zygctl status # status of one service zygctl start # start a service (transient) zygctl stop # stop a service zygctl restart # restart a service zygctl enable # enable at boot (creates symlink) zygctl disable # disable at boot (removes symlink) zygctl list # list all service definitions ``` ### Implementation 1. Parse args 2. For enable/disable: operate locally on symlinks (no daemon needed) 3. All other commands: connect to `/var/run/zyginit.sock`, send command, print response ## Platform Testing Strategy | Phase | Linux | Hammerhead (standalone VM) | |-------|-------|------------------------| | 1 | Full test with example TOML | Same | | 2 | Full test with mock graphs | Same | | 3 | Compile only | Full test | | 4 | fork/exec without contracts | Full test | | 5 | Poll loop without contracts | Full test | | 6 | Full test | Same | | 7 | Full test | Same | ## Risk Items 1. **ct_status_read pointer semantics** — pointer-to-pointer FFI, prototype on Hammerhead 2. **TOML inline array parsing** — stdlib doesn't parse natively, use Coral's proven pattern 3. **Contract event binary format** — reading from bundle fd needs Hammerhead struct parsing 4. **String splitting** — verify `str.split()` availability in Reef 0.4.0 stdlib 5. **HashMap with struct values** — may need index-based indirection instead of `HashMap[ServiceDef]` # zyginit Release Notes ## 0.2.7 — 2026-07-21 **Bug-fix release.** Fixes bug 004 — running the `zyginit` binary as a non-PID-1 process (classically `zyginit -v` to check the version) fell through into supervisor mode and **clobbered the live PID-1 control socket**, silently breaking `zygctl` control of the running system until reboot. Three defenses: - **Version flags exit safely.** `-v`, `-V`, and `--version` print the version and exit — but only when not running as PID 1. The Hammerhead kernel passes `-v` (verbose) as a boot-arg to init, so that path must never short-circuit, or PID 1 would return from `main()` and be re-exec'd in a loop. - **The control socket is guarded.** `create_socket` now `connect()`-probes the path and refuses to unlink/rebind a socket a live server is already listening on; a non-PID-1 instance that cannot own the socket exits cleanly. PID 1 keeps running without a control socket rather than halting. - **Non-PID-1 supervisor mode is opt-in.** A non-PID-1 run now requires `--supervisor-test`; a bare `zyginit ...` on a host where zyginit already owns PID 1 refuses and exits before any setup. The integration harnesses and the CLAUDE.md dev-run invocation pass the flag. Builds clean and passes the full integration suite (104/104) on **Reef 0.7.7**; no source changes were required for the 0.7.7 toolchain (its newly-rejected program shapes are all Active-Object-related, which zyginit does not use). ## 0.2.6 — 2026-07-21 **Bug-fix release.** Fixes bug 003 — spurious `exit=-1` service failures in first-boot reports on Hammerhead. Root-cause analysis found `exit=-1` was never an exec-failure code (a real exec failure exits 127); it was the "could not resolve an exit code" sentinel, produced by two distinct defects: - **Exec robustness / no silent failure.** Under first-boot I/O contention (devfsadm + ZFS pool-finalize) the child's exec of a valid start command could transiently fail, and the child then exited silently with an empty per-service log. The child now retries `process_exec` (5×, 100 ms backoff) and, on final failure, records the reason to both the per-service `.log` (fd 2, survives a privilege drop) and a new central `/var/log/zyginit/zyginit.log`, then exits 127. - **Reap-race reconciliation** (the actual source of `-1`). The contract-empty event on the bundle fd can beat the child's exit-status posting; `handle_contract_event` used to commit the `-1` sentinel and tear down tracking, after which the catch-all reaper discarded the real code. It now defers when there is no hint, no resolvable code yet, and a child is still tracked (`rt.pid > 0`), so the unconditional `reap_children` re-dispatches with the true code within ~1 s. This fixes both the 127→-1 masking and the independent false-positive where an exit-0 oneshot that lost the race was reported FAILED. New central operational log: `/var/log/zyginit/zyginit.log` (distinct from the per-service logs). Adds integration Suite 9b (spawn-failure capture); 97/97 tests pass. The reap-race half rides the Hammerhead contract path (not exercised by the Linux stubs) and awaits an hh-alpha first boot to confirm in situ. ## 0.2.5 — 2026-07-17 **Reef 0.7.6 migration; no init-engine behavior change.** Migrated the suite to Reef 0.7.6: `Result`/`Option` idioms, TOML inline arrays, and the `socket` → `ctlsocket` module rename (avoids a clash with the stdlib `net.socket`, which `net.unix` now imports). Internal/tooling release — no source tarball was cut for 0.2.5. ## 0.2.4 — 2026-06-13 **Service mutual exclusion.** Adds the `[dependencies].conflicts` array: mutually-exclusive services both fail if both are enabled, and a start is refused if a conflicting peer is already running. Boot-time resolution fails both sides and surfaces the reason in `zygctl status`. A one-sided declaration is enforced symmetrically (both directions). ## 0.2.3 — 2026-06-02 **Boot/shutdown progress gauge relocated into the header divider.** The gauge previously lived on the volatile last screen row, where interleaved kernel/daemon writes would scroll it off. It now paints into the header's horizontal rule. `fmt_progress_bar` and `PROGRESS_WIDTH` were removed and `fmt_divider_gauge` added in `src/ui.reef`; two screen rows are reclaimed for the rolling tape. ## 0.2.2 — 2026-05-21 **Repo content reshape; no behavioral change.** zyginit is now strictly engine + examples + schema docs. The in-repo `services/hammerhead/` tree (22 production service TOMLs) has been removed — distros own all service-definition policy in their own overlay trees. The Hammerhead team has absorbed the prior set into `base/usr/src/zyginit/overrides/`. The `examples/` directory is slimmed to four heavily-commented canonical examples: - `daemon-with-restart/` — long-running daemon with `[restart]` policy - `oneshot-setup/` — type="oneshot" with `[dependencies].requires` - `task-with-condition/` — type="task" with `[condition].exists_file` - `with-stop-script/` — `[exec].stop` + companion `stop.sh` `scripts/install-to-be.sh` was deleted; its "deploy our services to a BE" purpose evaporates in an engine-only world. (Distros can ship their own equivalent.) `docs/SMF_MIGRATION.md` has been rewritten as a methodology guide (how to convert an SMF manifest to a zyginit TOML) rather than a catalog of our specific conversions. Tarball: ~22 KB smaller than 0.2.1. No `clang` / `reefc` / API changes. ## 0.2.1 — 2026-05-15 Patch release with two operational improvements: - `type = "transient"` in 0.1.x-style TOMLs no longer silently demotes to `daemon`. The parser now treats it as `task` (the renamed type) and emits a deprecation warning. This prevents a service that should run as a task from hanging in STATE_STARTING after an incomplete upgrade. - Missing `/etc/zyginit/services/` directory at boot now produces a distinct, actionable log message instead of silently entering the idle loop. The operator is told to run the migration tool from a recovery medium. ## 0.2.0 — 2026-05-15 **Breaking change.** The filesystem layout under `/etc/zyginit/` is restructured. Operators upgrading from 0.1.x MUST run the migration tool before installing 0.2.0 binaries: /path/to/scripts/migrate-layout.sh --dry-run # preview /path/to/scripts/migrate-layout.sh --apply # apply The tool is idempotent (safe to re-run) and refuses to proceed if both old and new layouts coexist for the same service. ### New: per-service directory layout Each service is self-contained: /etc/zyginit/services// ├── service.toml ├── start.sh (optional) └── stop.sh (optional) `enabled.d/` symlinks target `../services//` (the directory), not `../.toml` (the file). ### New: `type = "task"` service type For services that may run for a while but are expected to eventually exit cleanly — platform probes, one-time setup work, etc. Replaces the previously-unused `type = "transient"`. [service] name = "acpihpd" type = "task" Semantics: - Exit 0 → STATE_STOPPED (counts as online in the boot banner) - Exit ≠ 0 → STATE_FAILED (counts as failed) - Never restarts - Skips the daemonize handshake (tasks don't double-fork) ### New: `[condition]` block Declarative pre-flight gating. Services whose conditions fail at startup go into a new SKIPPED state — they don't fork, don't fail, don't loop. Supported keys: [condition] exists_file = "/dev/acpihp" exists_file_any = ["/dev/acpihp", "/dev/acpi"] command = "/usr/sbin/acpi_probe" # 5-second timeout All specified keys must pass (AND). Absent keys are no-ops. The command form is bounded by a 5-second timeout. ### New: SKIPPED state in `zygctl status` Skipped services render with the `·` (dim) glyph and the skip reason in the NOTES column. The boot banner and final card include a "skipped" counter alongside online/failed. ### Hammerhead vendoring The Hammerhead team runs the migration tool against their override tree in `base/usr/src/zyginit/overrides/` as part of consuming this source drop. Follow the same dry-run → apply pattern. ### Other changes - Renamed `SERVICE_TYPE_TRANSIENT` → `SERVICE_TYPE_TASK` in the API. - `type = "transient"` in TOML produces a deprecation warning and is treated as `type = "task"` (see 0.2.1). ## 0.1.8 — 2026-05-14 Late-failure race fix. A 1-second settle period after the last tier catches daemons that fork successfully but die immediately (e.g., acpihpd on Hammerhead without ACPI hardware). ## 0.1.7 — 2026-05-14 Boot screen no longer scrolls the `[Z]` banner off the top of the framebuffer. Drop the trailing newline that was advancing the cursor past the last row. ## 0.1.6 — 2026-05-14 illumos build fix: `` now explicitly included for `struct winsize` and `TIOCGWINSZ`. Linux's `` pulls those in transitively; illumos doesn't. ## 0.1.5 — 2026-05-14 (Broken on illumos: missing `` for TIOCGWINSZ — use 0.1.6+.) Spinner-in-tape redesign (option B): each tape row tracks its own `active` flag, lifecycle procs update rows in place, parallel-startup ready. TIOCGWINSZ for dynamic screen size. Oneshot single-count fix. Dynamic SERVICE column in `zygctl status`. ## 0.1.4 — 2026-05-14 Framebuffer-first mode detection (PID 1 + isatty + no TERM defaults to truecolor). Sets TERM=sun-color in zyginit env so children inherit it. Plain-mode emit no longer mislabels event types. ## 0.1.3 — 2026-05-12 Initial visual pass: `[Z]` sigil, palette, rolling tape, progress bar, spinner, final cards, mirrored shutdown UI, `zygctl status` banner+table. # SMF Migration — Handoff to Hammerhead This document hands off the remaining SMF-removal work from the zyginit project to the Hammerhead team. It covers source vendoring, build integration, the decision process for converting each SMF manifest to a zyginit TOML service definition, worked examples, anti-patterns drawn from the iter 1-20 development history, per-category guidance, and a coarse catalog of all 128 manifests found on `hh-prototest` as of 2026-05-02. **See also:** `docs/DESIGN.md` for the zyginit architecture reference; `docs/ROADMAP.md` for the forward-looking task checklist. --- ## 1. Handoff Brief *This section orients the Hammerhead team on what zyginit has done, what is left, and where to ask questions.* ### What zyginit has done (through rev 82) zyginit is a working replacement for `svc.startd`, `svc.configd`, and the SMF master restarter — a single Reef process that acts as PID 1, supervises services defined in TOML files, and exposes an administrative socket for `zygctl`. As of rev 82 (2026-05-02), zyginit boots Hammerhead to a fully-functional multi-user state on `hh-prototest` (192.168.122.50) without any SMF infrastructure running: - **Multi-user boot** — all tier-0 through tier-7 services start cleanly. - **Single-user mode** — boot arg `-s` or runtime `zygctl single` drops to `/sbin/sh` on `/dev/console`. - **Runtime runlevel transitions** — `zygctl single` / `zygctl multi` (and the `init s` / `init 3` sysv wrappers) work bidirectionally. - **Clean shutdown** — `zygctl halt` / `reboot` / `poweroff` perform a reverse-tier stop, ZFS sync, `umountall -l`, and `uadmin` from a forked non-PID-1 child. - **utmpx integration** — `who -b`, `who -r`, and `uptime` report correctly. - **~18 core services converted** — the methodology is established for the full Hammerhead service set. The production TOMLs now live in Hammerhead's overlay tree at `base/usr/src/zyginit/overrides/` (absorbed during Phase D). See the worked examples in Section 3 and the catalog in Section 7 for the conversion patterns. ### What the Hammerhead team must do 1. **Vendor zyginit source** into `hammerhead-userland` (see Section 5 for the file-by-file layout and build instructions). 2. **Wire the build** to produce `/sbin/init` (zyginit), `/sbin/zygctl`, and `/sbin/sysv-wrapper` with its symlinks. 3. **Source the production TOMLs** from Hammerhead's overlay tree at `base/usr/src/zyginit/overrides/` (already absorbed during Phase D); install them to `/etc/zyginit/` and symlink in `enabled.d/`. 4. **Validate** that the vendored source builds and boots identically to `hh-prototest` before starting new conversions. 5. **Convert the remaining ~110 SMF manifests** using the decision tree in Section 2, the worked examples in Section 3, the per-category guidance in Section 6, and the catalog in Section 7. 6. **Remove SMF infrastructure** — `svc.startd`, `svc.configd`, master restarter binaries, manifest infrastructure, and `libscf`-only consumers — from `hammerhead-userland` packaging. ### Done criteria for "SMF removal complete" - Hammerhead boots to multi-user without `svc.startd` or `svc.configd` ever being invoked (verify with `pgrep -fl svc.startd`; expect zero results). - `libscf` is not linked into any binary in the default boot path (acceptable to keep `libscf` installed for optional/zports consumers, but no required boot service calls into it). - All zyginit-managed services continue to work at least as well as the SMF-managed equivalents they replaced. - Manifest XML files and `svc:/` repository infrastructure are removed from default packaging (they may be retained in a `smf-compat` optional package for administrators who need to run SMF tooling temporarily). - `halt`, `reboot`, `poweroff`, and `init ` reach the hardware-level operation cleanly. ### Coordination - **Questions about existing zyginit behavior** — open an issue against the zyginit repo. Include the service name, the symptom, and the hh-prototest log output. - **TOML fields that don't fit a service's needs** — file an RFE against zyginit. Do not work around missing features in the service scripts; get the runtime feature first. - **Hammerhead team's scope** — modifying `hammerhead-userland` is the Hammerhead team's job. The zyginit CLAUDE.md constraint "DO NOT modify Hammerhead source code" applies to the zyginit-project Claude, not to the Hammerhead Claude. The Hammerhead Claude should freely modify its own tree. **See also:** Section 5 (Integration with Hammerhead), `docs/ROADMAP.md` "SMF Removal from Hammerhead." --- ## 2. Decision Tree *This section is the core reference for every manifest conversion. Walk each manifest through the three steps in order.* ### Step 1 — Should this service exist on Hammerhead at all? ``` Does Hammerhead ship this functionality? | +-- No: deprecated, SPARC-only, SMF infrastructure, or NIS/iSCSI/etc. | --> DROP. Remove from packaging; do not write a TOML. | The catalog (Section 7) lists the obvious drops. | +-- Possibly, but as an optional zports add-on (Kerberos KDC, iSNS, etc.) | --> DEFER. Out of scope for hammerhead-userland; leave for zports. | +-- Yes, ships in hammerhead-userland --> Continue to Step 2. ``` **Mandatory: do not skip Step 1.** Converting a service "because it exists" is the single biggest source of wasted effort. The 128 manifests include services Hammerhead does not and should never ship (SMF infrastructure, SPARC platform code, NIS, iSCSI target, Trusted Extensions label daemon, etc.). These are straightforward drops. ### Step 2 — Is the service coupled to SMF infrastructure at runtime? ``` Does the daemon call into svc.configd / libscf / the SMF repository door at runtime (not just at startup)? | +-- Yes, fundamentally requires libscf or the repository for ongoing | operation (e.g., svc.startd, svc.configd themselves) | --> DROP (if it IS SMF) or INVESTIGATE (if it merely uses libscf). | For investigate: look for a foreground/standalone mode flag. | Coordinate with the daemon's upstream if none exists. | +-- Yes at startup, but the daemon has a non-SMF mode flag | (-d, -f, -D, or similar) that skips the startup gate and the | daemonize double-fork | --> CONVERT using the foreground-mode pattern. See Example 1. | Known flags on Hammerhead daemons: | dlmgmtd -d (debug/foreground; bypasses SMF_FMRI gate) | ipmgmtd -f (identical pattern) | sshd -D (standard OpenSSH; no daemonize) | syslogd -d (foreground; standard illumos) | fmd -f (foreground; from fmd source) | Check each daemon's man page and source before assuming | SMF coupling is unavoidable. | +-- No SMF coupling, or coupling is purely cosmetic (e.g., SMF_FMRI env var used only for a cache-file name) --> Continue to Step 3. ``` **Getting Step 2 wrong was the entire iter 1-5 dead end.** When dlmgmtd was started without its foreground flag, it failed silently — no error on stderr (daemonize had already closed fd 2), and the network never came up. See `memory/pid1_boot_iter11_15.md` for the full diagnosis. ### Step 3 — What runs the service today? ``` What does the SMF exec_method actually invoke? | +-- Direct daemon binary (no method script) | --> CONVERT with a simple TOML. exec.start = the binary + args. | Most daemons with exec=':kill' for stop use the contract | signal by default — omit [stop] unless you need custom signal. | +-- /lib/svc/method/ shell script | | | +-- Script is trivial (rm a FIFO, mkdir, then exec the daemon) | | --> INLINE the setup into a wrapper start.sh. See Example 2. | | Services that take this path: cron (rm FIFO + exec), | | syslogd (touch /var/adm/messages + exec), console-login | | (write utmpx entry + exec ttymon). | | | +-- Script sources /lib/svc/share/smf_include.sh AND calls | | smf_present() or smf_clear_method_context() | | --> Check if smf_present() branch is the failure path only. | | If so, the script works fine without configd (smf_present | | returns false; the relevant branch is skipped). Test | | carefully. Many scripts are safe even with smf_include.sh. | | | +-- Script does substantial, maintained setup work | --> KEEP the method script as exec.start. Add SMF_FMRI to | [exec] environment if the script requires it for | non-configd purposes (e.g., FIFO name, log prefix). | See Example 2 alternate form. | +-- Oneshot configuration command (mountall, swapadd, dumpadm, coreadm) | --> CONVERT as type = "oneshot". See the worked examples in | Section 3 and the Hammerhead overrides for root-fs, swap, | filesystem patterns. | Onshots use restart.on = "never". | +-- inetd-managed service (not a standalone daemon) --> DROP unless there is a strong reason to keep inetd. Hammerhead does not run inetd. If the underlying functionality is needed (e.g., legacy RPC), the service needs its own TOML as a standalone daemon. ``` ### Field-by-field translation table | SMF manifest element | TOML field | Notes | |---|---|---| | `` | `[service] name` | Strip the category prefix (e.g., `network/ssh` → `"sshd"`) | | `