zygaena / zyginit
|
Files Changesets Issues Pull Requests Releases
root / docs / DESIGN.md
DESIGN.md markdown 340 lines 12.8 KB
Raw

zyginit — Architecture and Design

Status: Implementation complete — testing on Hammerhead pending
Date: 2026-03-05
RFE Filed: reef-lang/bugs/RFE-001-init-system-requirements.md
Reef 0.4.0: Released 2026-03-05 — all 8 RFE-001 items implemented

Problem Statement

SMF (Service Management Facility) is architecturally overcomplex for Hammerhead's needs:

  • XML service definitions — verbose, hard to read/write, depends on libxml2
  • libxml2 dependency — maintenance status uncertain, large dependency for service config
  • Architectural complexity — 3 cooperating daemons (svc.startd, svc.configd, master restarter)
    plus SQLite repository, door-based IPC, and XML-based service manifests
  • Difficult to modify — tightly coupled components make changes risky

Proposed Replacement: zyginit

A single-process service manager and supervisor written in Reef, with:

  • TOML-based service definitions — clean, readable, no XML
  • Symlink-based enable/disable — no database, inspectable with ls
  • Dependency graph resolution with parallel startup
  • Service supervision with automatic restart
  • Hammerhead process contract integration — kernel-level tracking of all service descendants
  • Administrative CLI (zygctl) for service control via Unix domain socket

Naming

Component Name Description
Daemon zyginit PID 1, the init process
CLI zygctl Admin tool (start/stop/status/enable/disable)
Config dir /etc/zyginit/ Service TOML definitions
Enable dir /etc/zyginit/enabled.d/ Symlinks to enabled services
Socket /var/run/zyginit.sock Unix domain socket for zygctl
Diagnostics /var/run/zyginit/ Optional runtime state (tmpfs)

Why Reef

  • Compiles to native code via C/GCC — suitable for PID 1
  • FFI to C is clean — can call libcontract and all POSIX APIs directly
  • Result/Option error handling is correct for systems code
  • Writing the init in Reef differentiates Zygaena/Hammerhead
  • Reef already powers the Coral package manager — proven for systems programming

Architecture

                    +-----------+
                    |  zygctl   |  (Reef CLI tool)
                    +-----+-----+
                          | Unix socket (/var/run/zyginit.sock)
                    +-----v-----+
                    |  zyginit  |  (PID 1, Reef)
                    |           |
                    | +-------+ |
                    | | event  | |  poll() on: contract bundle fd,
                    | | loop   | |  unix socket fd, signal pipe fd
                    | +---+---+ |
                    |     |     |
                    | +---v---+ |
                    | | dep   | |  topological sort of service graph
                    | | graph | |  built from TOML at boot
                    | +---+---+ |
                    |     |     |
                    | +---v--------+ |
                    | | contract   | |  libcontract FFI:
                    | | manager    | |  template → fork → adopt → events
                    | +------------+ |
                    +-------+-------+
                            |
            +-------+-------+-------+-------+
            |       |       |       |       |
         sshd    cron    fmd   network   ...
       (contract) (contract) (contract) (contract)

Service Definition Format (TOML)

Service definitions are TOML files in /etc/zyginit/. Most services are purely
declarative. For complex startup logic (database initialization, network setup),
the exec.start field can point to a shell script in the service's subdirectory.

Simple service (declarative)

# /etc/zyginit/sshd.toml
[service]
name = "sshd"
description = "Secure Shell Daemon"
type = "daemon"           # daemon | oneshot | transient

[exec]
start = "/usr/lib/ssh/sshd -D"

[stop]
method = "contract"       # contract (default) | exec
signal = "TERM"
timeout = 60

[dependencies]
requires = ["network", "filesystem"]
after = ["name-services"]

[contract]
param = ["inherit", "noorphan"]
fatal = ["empty", "hwerr"]

[restart]
on = "failure"            # always | failure | never
delay = 5
max_retries = 3

Complex service (script escape hatch)

For services requiring complex startup logic, the TOML definition points to
scripts in a subdirectory matching the service name:

# /etc/zyginit/postgresql.toml
[service]
name = "postgresql"
description = "PostgreSQL Database Server"
type = "daemon"

[exec]
start = "/etc/zyginit/postgresql/start.sh"
stop = "/etc/zyginit/postgresql/stop.sh"

[dependencies]
requires = ["filesystem", "network"]

/etc/zyginit/
├── postgresql.toml
└── postgresql/
    ├── start.sh              # handles initdb, pg_upgrade, etc.
    └── stop.sh               # clean shutdown with pg_ctl

TOML schema fields (planned — needs further discussion)

Additional fields under consideration:

  • exec.user, exec.group — privilege separation (run as non-root)
  • exec.working_directory — chdir before exec
  • exec.environment — env vars: ["KEY=value", ...]
  • security.zone, security.chroot — containment
  • limits.max_fds, limits.max_memory — resource controls
  • logging.stdout, logging.stderr — output destination (file, syslog)
  • service.milestone — grouping: single-user, multi-user, network

State Management

Design Rationale

Research of other init systems informed the design:

System Enable/Disable State DB Boot Order
systemd Symlinks in .wants/ dirs None In-memory from unit files
dinit Symlinks in waits-for.d/ None In-memory from service files
OpenRC Symlinks in runlevel dirs None In-memory (cached to /run)
SMF SQLite property SQLite In-memory from repository

Every modern init except SMF uses symlinks for enable/disable state and builds
the dependency graph in memory at boot. SMF's SQLite repository is the #1 source
of boot failures ("repository corruption"). zyginit follows the simpler pattern.

Enabled state is managed via symlinks in /etc/zyginit/enabled.d/:

/etc/zyginit/
├── sshd.toml
├── cron.toml
├── network.toml
├── filesystem.toml
├── postgresql.toml
├── postgresql/
│   ├── start.sh
│   └── stop.sh
└── enabled.d/                   # symlinks = what starts at boot
    ├── sshd → ../sshd.toml
    ├── cron → ../cron.toml
    ├── network → ../network.toml
    └── filesystem → ../filesystem.toml
Operation What happens
zygctl enable sshd Creates symlink enabled.d/sshd → ../sshd.toml
zygctl disable sshd Removes symlink enabled.d/sshd
zygctl start sshd Starts service now (transient — does not persist across reboot)
zygctl stop sshd Stops service now
zygctl status Reads in-memory state from zyginit over Unix socket

Benefits:

  • Inspectable with ls /etc/zyginit/enabled.d/
  • Editable with ln -s and rm — no special tools required
  • Always on root filesystem — available at PID 1 startup
  • No corruption risk — no database to corrupt

Boot Order (derived from TOML)

No pre-computed boot order is stored. At boot, zyginit:

  1. Scans enabled.d/ symlinks → list of enabled services
  2. Reads each service's TOML → extracts [dependencies]
  3. Topological sort → parallel startup order
  4. Starts services respecting dependency constraints

For <100 services, TOML parsing and topo sort complete in milliseconds.

Runtime State (in-memory + optional tmpfs)

Service runtime state (PIDs, running/stopped/failed, restart counters, exit codes)
lives in memory inside the zyginit process. Accessible via zygctl status
over the Unix domain socket.

Optionally, zyginit writes diagnostic state to /var/run/zyginit/ (tmpfs) for
crash recovery analysis. This is not required for operation — if zyginit restarts,
it re-scans TOML definitions and re-adopts existing process contracts.

Filesystem Summary

Path Filesystem Available Purpose
/etc/zyginit/*.toml root Always Service definitions
/etc/zyginit/enabled.d/ root Always Enabled symlinks
/var/run/zyginit.sock tmpfs After tmpfs mount CLI communication
/var/run/zyginit/ tmpfs After tmpfs mount Diagnostic state (optional)

Hammerhead Process Contracts

Process contracts are a Hammerhead kernel mechanism (not available on Linux) accessed
via the contract filesystem (/system/contract/):

  1. Before fork(), open /system/contract/process/template and configure
  2. After fork(), all child processes (and their children) belong to a contract
  3. Contract owner receives events: empty (all exited), core dump, hardware error, fatal signal
  4. Can signal ALL processes in a contract atomically — no orphaned grandchildren
  5. On init restart, existing contracts are re-adopted — services keep running

The C API (libcontract) is fd-based: open/ioctl/close on ctfs paths. No complex
struct layouts — integer fds, integer flags, and string FMRIs. Clean for Reef FFI.

Boot Sequence

  1. Kernel execs /sbin/init → zyginit (PID 1)
  2. Scan /etc/zyginit/enabled.d/ → list of enabled services
  3. Read each service's TOML → extract [dependencies]
  4. Topological sort → boot order
  5. Start services in parallel, respecting dependency order
  6. Mount tmpfs, create /var/run/zyginit.sock
  7. Enter poll(2) event loop: contract events, Unix socket, signal pipe

Reef Language Requirements

Filed as RFE-001 with the Reef team (~/repos/reef-lang/bugs/RFE-001-init-system-requirements.md).

Response received (reef-lang/bugs/RFE-001-response.md): All 8 items accepted
for Reef 0.4.0. Estimated ~4 weeks. Key findings from Reef team:

  • GC is not a blocker — no auto-triggered STW pauses; signal change is 2 lines
  • fork/exec already exist in C runtime — just need stdlib wiring
  • TOML parser already exists — encoding/toml.reef (~560 lines)
  • Cooperative yield points already exist (--quantum-checks flag)

Revised Priority (per Reef team analysis)

# Enhancement Effort Status
1 Process management (fork/exec/setsid) Low Reef 0.4.0
2 File descriptor operations Low Reef 0.4.0
3 String array marshaling Low-Med Reef 0.4.0
4 Signal enhancements Low-Med Reef 0.4.0
5 Unix domain sockets Low Reef 0.4.0
6 Event loop / poll Low Reef 0.4.0
7 TOML parser enhancements Minimal Reef 0.4.0
8 GC mode (configurable signals + --no-gc) Medium Reef 0.4.0

Open Questions (from Reef team — resolved)

  1. poll(2) vs event ports: poll(2) in stdlib, event ports via extern "C".
    Poll is sufficient for 3-4 fds.
  2. fork() + Active Objects: Single-threaded poll loop — no AO conflict.
  3. GC signal preference: SIGRTMIN+0/SIGRTMIN+1 — no collision with standard signals.

Reef Team Recommendations

  1. Use extern "C" for Hammerhead-specific code (libcontract, ctfs) — keep in Hammerhead, not Reef stdlib
  2. Start prototyping now with process_spawn_shell() + FFI declarations
  3. Test with current GC — likely zero pauses for bounded working set
  4. Use existing TOML parser (import encoding.toml) immediately

Phased Approach

Phase A: Interim (Now)

  • Keep SMF operational — it works, it's built, it boots
  • libxml2 now builds from source in hammerhead (commit 3b7533657)
  • Begin zyginit prototype using existing Reef 0.4.0 capabilities
  • Test TOML parser with service definition format

Phase B: Prototype

  • Build zyginit prototype with basic service supervision
  • Test with a handful of services on standalone VM
  • Keep SMF as fallback

Phase C: Migration

  • Convert all SMF service manifests to TOML definitions
  • Boot with zyginit as PID 1
  • Validate with hh-audit

Phase D: SMF Removal

  • Remove svc.startd, svc.configd, master restarter
  • Remove libscf, libxml2 dependency (if no other consumers)
  • Remove SMF manifest infrastructure from build

FMA Integration

FMA (Fault Management Architecture) is independent of SMF and will be retained:

  • Kernel drivers post ereport events via sysevent
  • fmd daemon runs diagnosis engines (CPU retire, disk fault, ZFS degraded)
  • fmd is a service managed by the init system — works with any init
  • No architectural coupling to SMF

References

  • Hammerhead libcontract(3LIB), contract(5), process(5)
  • cmd/svc/startd/contract.c, method.c, restarter.c
  • dinit comparison: https://davmac.org/projects/dinit/
  • Reef RFE: ~/repos/reef-lang/bugs/RFE-001-init-system-requirements.md
  • Reef 0.4.0 changelog: ~/repos/reef-lang/docs/changelog-0.4.0.md