# Coral Package Manager Guide Coral is the package manager for Zygaena, a ports-based operating system built on illumos. This guide covers all aspects of using Coral, from basic package management to creating repositories and signing packages. ## Table of Contents 1. [Installation and Setup](#installation-and-setup) 2. [Basic Usage](#basic-usage) 3. [Creating and Building Packages](#creating-and-building-packages) 4. [Package Groups](#package-groups) 5. [Alternative Dependencies](#alternative-dependencies) 6. [Creating Repositories](#creating-repositories) 7. [Package Signing](#package-signing) 8. [Database Management](#database-management) 9. [Port Development Tools](#port-development-tools) 10. [Configuration Reference](#configuration-reference) --- ## Installation and Setup ### Prerequisites Coral requires: - Zygaena or illumos-based system - Reef compiler 0.5.0 or later - Root privileges for package installation - GCC or compatible C compiler (for building from source) ### Directory Structure Coral uses the following directories: | Path | Purpose | |------|---------| | `/usr/zports/` | Ports tree (package build definitions) | | `/usr/zports/groups/` | Package group definitions | | `/etc/coral/coral.conf` | Main configuration file | | `/etc/coral/repos.d/` | Repository configurations | | `/etc/coral/keys/` | Signing keys | | `/var/lib/coral/installed/` | Installed package database (source of truth) | | `/var/lib/coral/db/` | Package database indexes (MsgPack) | | `/var/lib/coral/packages/` | Binary package cache | | `/var/lib/coral/sources/` | Source archive cache | | `/var/lib/coral/repos/` | Repository metadata cache | | `/var/lib/coral/trusted-keys/` | Trusted public keys | | `/var/tmp/coral/work/` | Build work directory | | `/var/tmp/coral/pkg/` | Package staging directory | ### Initial Setup 1. Create required directories: ```bash sudo mkdir -p /usr/zports/{core,base,devel,desktop,extra} sudo mkdir -p /usr/zports/groups sudo mkdir -p /etc/coral/repos.d sudo mkdir -p /etc/coral/keys sudo mkdir -p /var/lib/coral/{installed,packages,sources,repos,trusted-keys,db} sudo mkdir -p /var/tmp/coral/{work,pkg} ``` 2. Create initial configuration: ```bash sudo tee /etc/coral/coral.conf << 'EOF' [general] arch = "x86_64" jobs = 0 confirm = true color = true [build] logging = true quiet = false [security] require_signed_repos = false require_signed_packages = false EOF ``` 3. Initialize the package database: ```bash sudo coral db init ``` --- ## Basic Usage ### Installing Packages Install a single package: ```bash coral install vim ``` Install multiple packages: ```bash coral install vim nano htop ``` Install from a local package file: ```bash coral install /path/to/package-1.0.0-1.pkg.tar.xz ``` Install without confirmation: ```bash coral install -y vim ``` Force reinstall (also overrides conflict checks): ```bash coral install --force vim ``` Dry run (show what would be done without making changes): ```bash coral install -n vim ``` When a binary package is not found in any repository and `fallback_to_source = true` (default), Coral will attempt to build the package from the ports tree automatically. ### Removing Packages Remove a package: ```bash coral remove vim ``` Remove multiple packages: ```bash coral remove vim nano htop ``` ### Upgrading Packages Upgrade all packages: ```bash coral upgrade ``` Upgrade specific packages: ```bash coral upgrade vim nano ``` Check for available updates: ```bash coral outdated ``` ### Querying Packages List installed packages: ```bash coral list ``` Show package information: ```bash coral info vim ``` List files owned by a package: ```bash coral files vim ``` Find which package owns a file: ```bash coral owner /usr/bin/vim ``` Search for packages: ```bash coral search editor ``` Show dependency tree: ```bash coral depends vim ``` Verify package integrity: ```bash coral verify vim ``` ### Building Packages Build a package from source: ```bash coral build vim ``` The built package is stored in `/var/lib/coral/packages/`. ### Syncing Ports Update the ports tree: ```bash coral sync ``` --- ## Creating and Building Packages ### Port Structure A port consists of a directory under `/usr/zports///` containing: ``` /usr/zports/base/mypackage/ ├── package.toml # Package metadata ├── build.sh # Build script (zsh) ├── fix-build.patch # Optional patch files └── hammerhead-solaris-compat.patch # (referenced in package.toml [patches]) ``` ### package.toml The `package.toml` file defines package metadata: ```toml [package] name = "mypackage" version = "1.2.3" release = 1 description = "A useful package" url = "https://example.com/mypackage" license = "MIT" maintainer = "Your Name " arch = "x86_64" # Optional: alternatives this package provides alternatives = ["editor", "pager"] # Optional: packages this conflicts with (install aborts if any are installed) conflicts = ["otherpackage"] [dependencies] # Packages required at runtime runtime = ["libfoo", "libbar>=1.2"] # Packages required only for building build = ["gcc", "gmake"] ### Per-Package Path Overrides Packages can override the default installation prefix and sysconfdir. This is useful for packages that install to `/opt`, `/srv`, or other non-standard locations: ```toml [paths] prefix = "/opt/myapp" sysconfdir = "/opt/myapp/etc" ``` These override the system defaults from `coral.conf` and are exported as `$PREFIX` and `$SYSCONFDIR` to the build script. If omitted, the system-wide values are used. ### Source Files Source files can be specified using TOML table arrays (`[[source]]`), which supports mirror URLs for automatic failover: ```toml [[source]] file = "mypackage-1.2.3.tar.gz" urls = [ "https://example.com/mypackage-1.2.3.tar.gz", "https://mirror.example.org/mypackage-1.2.3.tar.gz" ] checksum = "abc123def456..." [[source]] file = "extra-data.tar.gz" urls = ["https://example.com/extra-data.tar.gz"] checksum = "789xyz..." extract = false # Don't extract this archive ``` Each `[[source]]` entry supports: | Field | Required | Description | |-------|----------|-------------| | `file` | Yes | Output filename to save as | | `urls` | Yes | Array of mirror URLs (tried in order until success) | | `checksum` | Yes | SHA256 checksum, or `"SKIP"` to skip verification | | `extract` | No | Whether to extract the archive (default: `true`) | **Mirror Failover**: When downloading, Coral tries each URL in the `urls` array in order. If a download fails or checksum verification fails, the next mirror is tried automatically. #### Local Source Files For packages where sources aren't available on public HTTP servers (private repos, bundled tarballs), you can place source files directly in the port directory and reference them by filename: ```toml [[source]] file = "reef-lang-0.1.10.tar.gz" urls = ["reef-lang-0.1.10.tar.gz"] # Local file in port directory checksum = "abc123..." ``` URLs that don't start with `http://` or `https://` are treated as local paths: - **Relative paths** (no leading `/`) are resolved relative to the port directory - **Absolute paths** are used as-is Local files are copied to the sources cache (`/var/lib/coral/sources/`) for consistency with downloaded files. You can also mix local and remote sources for fallback: ```toml [[source]] file = "mypackage-1.0.tar.gz" urls = [ "mypackage-1.0.tar.gz", # Try local first "https://example.com/mypackage-1.0.tar.gz" # Fall back to remote ] checksum = "abc123..." ``` #### Legacy Format (Deprecated) The older `[sources]` format is still supported for backward compatibility: ```toml [sources] urls = [ "https://example.com/mypackage-1.2.3.tar.gz" ] checksums = [ "abc123def456..." ] ``` **Note**: The legacy format does not support mirrors or the `extract` flag. ### Patches Coral can apply patches to source code before running `build.sh`. Patch files are placed in the port directory alongside `package.toml` and declared in a `[patches]` section: ```toml [patches] files = ["hammerhead-solaris-compat.patch", "fix-build.patch"] strip = 0 ``` | Field | Required | Default | Description | |-------|----------|---------|-------------| | `files` | Yes | — | Array of patch filenames, applied in order | | `strip` | No | `0` | Strip level passed to `patch -p` | Patches are applied from within the extracted source directory using `patch -p`. The `strip` value controls how many leading path components are removed from filenames in the diff header: - `strip = 0` — paths used as-is (e.g., diff against `configure` / `configure.orig`) - `strip = 1` — strips `a/` and `b/` prefixes (standard `git diff` output) **Example**: Adding Hammerhead OS recognition to ncurses' configure script: ```diff --- configure.orig +++ configure @@ -7066,7 +7066,7 @@ test "$cf_cv_shlib_version" = auto && cf_cv_shlib_version=rel ;; - (solaris2*) + (solaris2*|hammerhead*) # tested with SunOS 5.5.1 (solaris 2.5.1) and gcc 2.7.2 ``` [build] # Enable parallel builds parallel = true # Number of parallel jobs (0 = auto) jobs = 0 ``` ### Build Script (build.sh) The build script is executed in the source directory. Environment variables available: | Variable | Description | |----------|-------------| | `$DESTDIR` | Installation staging directory | | `$PREFIX` | Installation prefix (default: `/usr/local`, configurable in `coral.conf` or per-package) | | `$SYSCONFDIR` | System configuration directory (default: `/etc`, configurable in `coral.conf` or per-package) | | `$PKG_ROOT` | Alternate root directory (empty when installing to `/`) | | `$SRCDIR` | Extracted source directory | | `$PKGDIR` | Package staging directory (alias for `$DESTDIR`) | | `$CFLAGS` | C compiler flags | | `$CXXFLAGS` | C++ compiler flags | | `$LDFLAGS` | Linker flags (see [Linker Flags](#linker-flags-and-library-resolution) below) | | `$PKG_CONFIG_PATH` | pkg-config search path (set to `$PREFIX/lib/pkgconfig`) | | `$MAKEFLAGS` | Make flags (includes `-j` for parallel builds) | | `$JOBS` | Number of parallel jobs | | `$PKG_NAME` | Package name | | `$PKG_VERSION` | Package version | | `$BUILD_TRIPLE` | Canonical build triplet (e.g., `x86_64-zygaea-hammerhead`) | | `$MAKE` | Make command (`gmake`) | | `$TAR` | Tar command (`gtar`) | Example build.sh: ```bash #!/usr/bin/zsh set -e # Configure using $PREFIX and $SYSCONFDIR (don't hardcode paths) ./configure --prefix=$PREFIX --sysconfdir=$SYSCONFDIR # Build $MAKE ${MAKEFLAGS} # Install to staging directory $MAKE DESTDIR="${DESTDIR}" install ``` Build scripts are executed with `zsh` (the Zygaena standard shell). #### Linker Flags and Library Resolution Coral sets `LDFLAGS="-R$PREFIX/lib"` globally. This embeds an RPATH in every built binary so the runtime linker can find shared libraries installed under `$PREFIX` (e.g., `/usr/local/lib`) without requiring system-wide `crle` configuration. **Why only `-R` and not `-L`?** Coral intentionally does **not** set `-L$PREFIX/lib` globally. The `-L` flag changes the **build-time** linker search order, which could cause Coral-installed libraries to shadow base OS libraries in `/lib` and `/usr/lib`. For example, if both the base OS and a Coral port ship `libz.so`, a global `-L/usr/local/lib` would cause all ports to link against the Coral version instead of the base version, potentially introducing ABI mismatches or version conflicts. The `-R` flag only affects **runtime** resolution and does not change build-time search order, making it safe to set globally. **When a build script needs `-L`:** If a port's `configure` script cannot find a dependency's library on its own, the build script should add `-L` explicitly: ```bash export LDFLAGS="-L$PREFIX/lib $LDFLAGS" ``` This appends to Coral's RPATH setting rather than replacing it. Only add `-L` when `configure` fails to locate the library — most autotools-based projects find libraries via `pkg-config` (which Coral also configures via `$PKG_CONFIG_PATH`) or their own `--with-*` flags. #### pkg-config Coral sets `PKG_CONFIG_PATH` to `$PREFIX/lib/pkgconfig` so that `configure` scripts using `pkg-config` can find `.pc` files from Coral-installed packages. This is the preferred mechanism for library discovery — most modern autotools and meson projects use `pkg-config` and will automatically pick up the correct `-I`, `-L`, and `-l` flags from `.pc` files. ### Install Scripts Packages can include scripts that run during installation, removal, and upgrade. Create a `.SCRIPTS/` directory in the package: ``` .SCRIPTS/ ├── pre-install.sh # Runs before files are installed ($VERSION) ├── post-install.sh # Runs after files are installed ($VERSION) ├── pre-remove.sh # Runs before files are removed ($VERSION) ├── post-remove.sh # Runs after files are removed ($VERSION) ├── pre-upgrade.sh # Runs before upgrade ($OLD_VERSION $NEW_VERSION) └── post-upgrade.sh # Runs after upgrade ($OLD_VERSION $NEW_VERSION) ``` Scripts are executed with `zsh`. Version arguments are passed as positional parameters (`$1`, `$2`). **Hook behavior:** - Pre-hooks (`pre-install`, `pre-remove`, `pre-upgrade`) abort the operation if they exit non-zero - Post-hooks (`post-install`, `post-remove`, `post-upgrade`) log a warning but don't abort on failure **Upgrade flow:** When installing a newer version of an already-installed package, Coral runs: `pre-upgrade` → remove old files → install new files → `post-upgrade`. Example post-install.sh: ```bash #!/usr/bin/zsh # Update font cache after installing fonts fc-cache -f ``` Example pre-upgrade.sh: ```bash #!/usr/bin/zsh OLD_VERSION=$1 NEW_VERSION=$2 # Stop service before upgrade svcadm disable myservice ``` ### Config Files To mark files as configuration files (preserved during upgrades), create a `.CONFIG` file listing them: ``` /etc/mypackage/config.conf /etc/mypackage/settings.toml ``` Config files use checksum-based protection during upgrades: - **Fresh install**: config file is installed normally - **Upgrade, user has not modified config**: new version overwrites cleanly - **Upgrade, user has modified config**: new version is installed as `.pkg-new`, user's file is preserved On removal, user-modified config files are preserved (not deleted). ### Building the Package ```bash coral build mypackage ``` This will: 1. Resolve dependencies recursively (build uninstalled deps from source first) 2. Download sources (with automatic mirror failover) 3. Verify checksums 4. Extract sources 5. Apply patches 6. Run build script 7. Compress man pages (if `compress_man = true`) 8. Create package archive 9. Clean up build directories If a dependency is not installed, Coral builds it from source automatically using a topological sort of the full dependency tree. Circular dependencies are detected and reported. #### Build Command Options | Option | Description | |--------|-------------| | `--noclean` / `-n` | Skip cleanup of work and staging directories after build | | `--clean` / `-c` | Force cleanup even if config says to keep directories | | `--quiet` / `-q` | Hide subprocess output (still logged to file) | | `--verbose` / `-v` | Show all output (overrides quiet mode) | | `-y` | Skip confirmation prompts | | `--force` | Force rebuild even if package exists | Example: Keep build artifacts for debugging: ```bash coral build --noclean mypackage ``` #### Build Cleanup Behavior After a successful build, Coral cleans up: - **Work directory** (`/var/tmp/coral/work/`): Extracted sources and build files - **Staging directory** (`/var/tmp/coral/pkg/`): Package staging area Cleanup behavior can be controlled via: 1. Command-line flags (`--noclean`, `--clean`) 2. Configuration file settings (`clean_work`, `clean_pkg`) Downloaded source archives are preserved by default in `/var/lib/coral/sources/`. #### Build Logging By default, Coral logs all build output to a file in the work directory: ``` /var/tmp/coral/work//coral-build--.log ``` The log captures: - All Coral status messages (download, extract, patch, build, package) - Full subprocess output (build script, patch commands) - Commands executed Log files are cleaned up with the work directory unless `--noclean` is specified. Use `--quiet` to hide subprocess output from the terminal while still logging everything: ```bash coral build --quiet vim # Clean output, full log file coral build --verbose vim # Show all output (overrides quiet) ``` --- ## Package Groups Groups allow installing multiple related packages with a single command. ### Creating a Group Create a TOML file in `/usr/zports/groups/`: ```toml # /usr/zports/groups/base-devel.toml [group] name = "base-devel" description = "Base development tools" packages = [ "gcc", "gmake", "binutils", "autoconf", "automake", "libtool", "pkg-config" ] ``` ### Installing a Group Use the `@` prefix: ```bash coral install @base-devel ``` This expands to all packages in the group and installs them with dependency resolution. ### Listing Groups Groups are stored as `.toml` files in `/usr/zports/groups/`. List them with: ```bash ls /usr/zports/groups/ ``` --- ## Alternative Dependencies Alternatives allow a package to depend on one of several options. This is useful when multiple packages can satisfy a dependency. ### Version Constraints Dependencies can include version constraints using operators: ```toml [dependencies] runtime = [ "libfoo>=1.0", # Version 1.0 or newer "libbar<=2.0", # Version 2.0 or older "libqux=3.1.0", # Exactly version 3.1.0 "libbaz>0.9", # Strictly newer than 0.9 "libold<2.0", # Strictly older than 2.0 ] ``` If an installed package doesn't satisfy the version constraint, Coral includes it in the install order for upgrade. ### Specifying Alternatives In `package.toml`, use the colon syntax: ```toml [dependencies] runtime = [ "libfoo", "editor:vim,nano,emacs", # Any of these editors "database:postgresql,mysql" # Any of these databases ] ``` Format: `base_name:option1,option2,option3` ### How Alternatives Work 1. During dependency resolution, Coral checks if any alternative is installed 2. If one is installed, the dependency is satisfied 3. If none are installed, the first option is automatically selected 4. Users can override by pre-installing their preferred option ### Providing Alternatives A package can declare that it provides certain alternatives: ```toml [package] name = "vim" alternatives = ["editor", "vi"] ``` When another package depends on "editor", vim will satisfy that dependency. --- ## Creating Repositories Repositories distribute pre-built binary packages. ### Repository Structure ``` /path/to/repo/ ├── repo.toml # Repository manifest ├── repo.toml.sig # Manifest signature (optional) └── packages/ # Binary packages ├── vim-9.0-1.x86_64.pkg.tar.xz ├── vim-9.0-1.x86_64.pkg.tar.xz.sig └── ... ``` ### Initializing a Repository ```bash coral repo init /path/to/repo ``` This creates the directory structure and an empty manifest. ### Adding Packages Copy built packages to the `packages/` directory: ```bash cp /var/lib/coral/packages/*.pkg.tar.xz /path/to/repo/packages/ ``` ### Rebuilding the Manifest After adding packages, rebuild the manifest: ```bash coral repo rebuild /path/to/repo ``` This scans all packages and generates `repo.toml`: ```toml [repository] name = "myrepo" description = "My Package Repository" url = "https://repo.example.com" [packages.vim] version = "9.0" release = 1 description = "Vi IMproved" license = "Vim" category = "base" file = "vim-9.0-1.x86_64.pkg.tar.xz" sha256 = "abc123..." size = 12345678 compressed = 2345678 depends = ["ncurses", "libfoo"] [packages.nano] # ... more packages ``` ### Signing the Repository ```bash coral repo sign /path/to/repo ``` This signs `repo.toml` with your private key, creating `repo.toml.sig`. ### Configuring Clients Create a repository configuration file on client systems: ```toml # /etc/coral/repos.d/myrepo.repo [repository] name = "myrepo" url = "https://repo.example.com" priority = 100 enabled = true signed = true keyid = "myrepo-key" ``` ### Refreshing Repository Data ```bash coral repo refresh ``` ### Mirroring Export a repository for offline use: ```bash coral repo export /path/to/repo /path/to/output ``` --- ## Package Signing Coral uses Ed25519 signatures for package and repository verification. ### Key Management #### Initialize Keyring ```bash coral key init ``` Creates `/etc/coral/keys/` directory. #### Generate a Signing Key ```bash coral key generate mykey ``` Creates: - `/etc/coral/keys/mykey.key` (private key - keep secure!) - `/etc/coral/keys/mykey.pub` (public key - distribute this) #### List Keys ```bash coral key list ``` #### Export Public Key ```bash coral key export mykey > mykey.pub ``` #### Import a Public Key ```bash coral key import mykey.pub repokey ``` #### Trust a Key ```bash coral key trust repokey ``` Adds the key to `/var/lib/coral/trusted-keys/`. #### Revoke Trust ```bash coral key revoke repokey ``` ### Signing Packages When building a package, sign it: ```bash coral build mypackage coral key sign /var/lib/coral/packages/mypackage-1.0-1.pkg.tar.xz mykey ``` ### Signing Repositories ```bash coral repo sign /path/to/repo ``` Uses the first available private key, or specify one: ```bash coral repo sign /path/to/repo --key mykey ``` ### Verification Enable mandatory verification in `/etc/coral/coral.conf`: ```toml [security] require_signed_repos = true require_signed_packages = true ``` When enabled: - Repository manifests must have valid signatures - Packages must have valid signatures - Signatures must be from trusted keys ### Verification Process 1. Download package and signature 2. Find matching public key in trusted keys 3. Verify signature using Ed25519 4. Reject if verification fails (when required) --- ## Database Management Coral maintains MsgPack-based indexes for fast package and file lookups. The source of truth is always the installed package database (`/var/lib/coral/installed/`); indexes can be safely rebuilt at any time. ### Database Files | File | Purpose | |------|---------| | `/var/lib/coral/db/packages.db` | Indexed package metadata (name, version, description) | | `/var/lib/coral/db/files.db` | File-to-package mapping for O(log N) ownership lookups | | `/var/lib/coral/db/ports.db` | Ports index for fast search/list without parsing TOML from disk | ### Initializing the Database After initial system setup, initialize the database: ```bash coral db init ``` This creates the index files based on currently installed packages. ### Checking Database Status View the current database status: ```bash coral db status ``` Output shows: - Index location - Number of packages indexed - Number of files indexed - Number of ports indexed ### Rebuilding Indexes If indexes become corrupted or out of sync (e.g., after manual edits to `/var/lib/coral/installed/`), rebuild them: ```bash coral db rebuild ``` This scans all installed packages and regenerates both indexes from scratch. ### How Indexes Work **packages.db**: Stores package metadata in MsgPack format, encoded as Base64. Contains name, version, release, description, install date, and file count for each installed package. **files.db**: Maps file paths to owning packages. Files are sorted alphabetically, enabling O(log N) binary search lookups. This dramatically speeds up `coral owner` queries. ### Automatic Index Updates Indexes are automatically updated when: - A package is installed (`coral install`) - A package is removed (`coral remove`) - The ports tree is synced (`coral sync` — rebuilds ports.db) You do not need to manually rebuild indexes during normal operation. ### Performance | Operation | Without Index | With Index | |-----------|---------------|------------| | `coral list` (100 packages) | ~500ms | ~10ms | | `coral owner /usr/bin/vim` (50K files) | ~2000ms | <10ms | ### Using with Alternate Root All database commands support the `--root` flag for chroot environments: ```bash coral --root /mnt/system db init coral --root /mnt/system db status coral --root /mnt/system db rebuild ``` --- ## Port Development Tools The `coral ports` command provides tooling for port developers. ### Scaffolding a New Port Create a new port with boilerplate files: ```bash coral ports new hammerhead/my-port ``` This creates: - `{ports_dir}/hammerhead/my-port/package.toml` — package metadata template - `{ports_dir}/hammerhead/my-port/build.sh` — build script stub The generated `package.toml` includes all standard sections with placeholder values. Edit it to fill in your package's details, then test with: ```bash coral build hammerhead/my-port ``` ### Ports Index Cache The ports index (`ports.db`) caches metadata from all ports so that commands like `coral search`, `coral list --available`, and `coral depends --reverse` don't need to parse every `package.toml` from disk. Check index status: ```bash coral ports cache status ``` Manually rebuild the index: ```bash coral ports cache rebuild ``` The index is automatically rebuilt by `coral sync`, `coral db init`, and `coral db rebuild`. ### Dynamic Categories Categories are discovered dynamically by scanning the ports directory. Any subdirectory under the ports tree (excluding infrastructure directories like `pkgs/` and `groups/`) is treated as a category. Adding a new category directory is all that's needed — no code changes required. --- ## Configuration Reference ### /etc/coral/coral.conf ```toml [general] # Target architecture arch = "x86_64" # Parallel build jobs (0 = auto-detect CPU count) jobs = 0 # Ask for confirmation before operations confirm = true # Enable colored output color = true [paths] # Installation prefix (exported as $PREFIX to build scripts) prefix = "/usr/local" # System configuration directory (exported as $SYSCONFDIR to build scripts) sysconfdir = "/etc" # Ports tree location ports = "/usr/ports" # Built packages cache packages = "/var/lib/coral/packages" # Downloaded sources cache sources = "/var/lib/coral/sources" # Installed package database database = "/var/lib/coral/installed" # Build work directory work = "/var/tmp/coral/work" # Package staging directory pkg = "/var/tmp/coral/pkg" [build] # Default C compiler flags cflags = "-O2 -pipe" # Default C++ compiler flags cxxflags = "-O2 -pipe" # Default make flags makeflags = "" # Strip binaries strip = true # Compress man pages compress_man = true # Fall back to building from source if binary not available fallback_to_source = true # Log build output to file (in work directory) logging = true # Hide subprocess output from terminal (still logged) quiet = false [network] # Download timeout in seconds timeout = 30 # Number of retry attempts retries = 3 # Use proxy (reads http_proxy/https_proxy env vars) use_proxy = false [sync] # Auto-refresh repository metadata auto_refresh = false [cache] # Keep downloaded packages after install keep_packages = true # Keep downloaded sources after build keep_sources = true # Clean work directory after successful build (part of [build] section) clean_work = true # Clean package staging directory after successful build clean_pkg = true [security] # Require signed repository manifests require_signed_repos = false # Require signed packages require_signed_packages = false ``` ### Repository Configuration Repository files in `/etc/coral/repos.d/*.repo`: ```toml [repository] # Repository name (used in messages) name = "zygaena-main" # Base URL for the repository url = "https://repo.zygaena.org/main" # Priority (lower = higher priority) priority = 100 # Enable/disable this repository enabled = true # Repository is signed signed = true # Key ID for signature verification keyid = "zygaena-main" ``` --- ## Troubleshooting ### Common Issues **"Package not found"** - Check if the port exists: `ls /usr/zports/*/packagename` - Refresh repository metadata: `coral repo refresh` - Check repository configuration in `/etc/coral/repos.d/` **"Signature verification failed"** - Import the repository's public key: `coral key import` - Trust the key: `coral key trust keyname` - Check key hasn't been revoked **"Dependency resolution failed"** - Check for circular dependencies - Verify all dependencies are available - Try installing dependencies manually **"Build failed"** - Check build.sh for errors - Verify build dependencies are installed - Check logs in `/var/tmp/coral/work/` ### Debug Mode Enable verbose output: ```bash coral -v install vim ``` Dry run (show what would be done): ```bash coral -n install vim ``` --- ## Quick Reference | Command | Description | |---------|-------------| | `coral install pkg` | Install package | | `coral install @group` | Install package group | | `coral install -n pkg` | Dry run (show what would happen) | | `coral remove pkg` | Remove package | | `coral upgrade` | Upgrade all packages | | `coral list` | List installed packages | | `coral info pkg` | Show package info | | `coral files pkg` | List package files | | `coral owner /path` | Find file owner | | `coral search term` | Search packages | | `coral depends pkg` | Show dependencies | | `coral verify pkg` | Verify package integrity | | `coral build pkg` | Build from port (no install) | | `coral build --install pkg` | Build and install | | `coral build --noclean pkg` | Build, keep work directories | | `coral build --quiet pkg` | Build with quiet output | | `coral sync` | Update ports tree | | `coral outdated` | Show available updates | | `coral repo list` | List repositories | | `coral repo refresh` | Refresh repo metadata | | `coral key list` | List signing keys | | `coral key generate name` | Generate new key | | `coral clean status` | Show build artifact status | | `coral clean all` | Clean all build artifacts | | `coral clean --force all` | Clean all, override cache protection | | `coral db init` | Initialize database indexes | | `coral db status` | Show database status | | `coral db rebuild` | Rebuild indexes from installed packages | | `coral ports new cat/name` | Scaffold a new port | | `coral ports cache status` | Show ports index status | | `coral ports cache rebuild` | Rebuild ports index | --- ## License Coral is part of the Zygaena project. Copyright (C) 2025 Leafscale, LLC Licensed under BSD-2-Clause https://www.leafscale.com # Implementation Plan: MsgPack Package Database Indexes **Date:** 2026-01-29 **Status:** Planned **Target:** Coral v0.2 ## Overview Implement the MsgPack-based package database indexes that were planned as the key feature of Coral v0.2. Currently `src/core/pkgdb.reef` is entirely stubs that fall back to slow file scanning. This causes: - `coral list` - O(N) TOML file reads - `coral owner` - O(N*M) full scan of all packages' file lists (CRITICAL bottleneck) - `coral info` - O(1) but requires TOML parsing ## Target Performance | Command | Current | With MsgPack | |---------|---------|--------------| | `coral list` (100 pkgs) | ~500ms | ~10ms | | `coral owner /usr/bin/vim` (50K files) | ~2000ms | ~1ms | | `coral info vim` | ~50ms | ~5ms | --- ## File Format Specifications ### packages.db (MsgPack + Base64) ``` Structure: { "version": 1, "generated": "2026-01-29T...", "package_count": N, "packages": [ { "name": "vim", "version": "9.1", "release": 1, "description": "...", "installed_date": "...", "file_count": 423 }, ... ] } ``` ### files.db (MsgPack + Base64) ``` Structure: { "version": 1, "generated": "2026-01-29T...", "file_count": M, "files": { "usr/bin/vim": "vim", "usr/lib/libz.so": "zlib", ... } } ``` **Note:** Base64 encoding required because Reef's `file.readFile()`/`file.writeFile()` work with strings. --- ## Implementation Phases ### Phase 1: Foundation (in pkgdb.reef) Add new types: ```reef type PackagesIndex = struct version: int generated: string package_count: int entries: [PkgIndexEntry] loaded: bool end PackagesIndex type FilesIndex = struct version: int generated: string file_count: int file_paths: [string] // Sorted for binary search owners: [string] // Parallel array - owner at same index loaded: bool end FilesIndex ``` Add helper functions: - `base64_encode(buf: [int], length: int): string` - `base64_decode(encoded: string, buf: [int]): int` - `estimate_packages_buffer_size(pkg_count: int): int` - `estimate_files_buffer_size(file_count: int): int` ### Phase 2: Serialization Implement MsgPack serialization: ```reef fn serialize_packages_db(root: string): [int] // 1. Get list of installed packages from database module // 2. Allocate buffer based on count estimate // 3. Pack map header (4 keys: version, generated, package_count, packages) // 4. Pack array of package entries // Returns packed buffer end serialize_packages_db fn serialize_files_db(root: string): [int] // 1. Get all packages, collect all files with owners // 2. Sort file paths for binary search later // 3. Allocate buffer // 4. Pack map header + files map // Returns packed buffer end serialize_files_db ``` Implement `rebuild_indexes_rooted()`: ```reef fn rebuild_indexes_rooted(root: string): bool // 1. Serialize packages.db, base64 encode, write file // 2. Serialize files.db, base64 encode, write file // 3. Return success end rebuild_indexes_rooted ``` ### Phase 3: Deserialization Implement loading functions: ```reef fn load_packages_index(root: string): PackagesIndex // 1. Read base64 string from packages.db file // 2. Decode to binary buffer // 3. Unpack MsgPack structure // 4. Return populated PackagesIndex end load_packages_index fn load_files_index(root: string): FilesIndex // 1. Read base64 string from files.db file // 2. Decode to binary buffer // 3. Unpack MsgPack map into parallel arrays // 4. Return populated FilesIndex (already sorted from serialize) end load_files_index fn binary_search_file(index: FilesIndex, path: string): string // O(log N) binary search in sorted file_paths array // Return owner or empty string end binary_search_file ``` ### Phase 4: Update Public API Replace stub implementations: ```reef fn list_packages(names: [string], versions: [string], max_count: int): int let index = load_packages_index("") if not index.loaded return fallback_list_packages(names, versions, max_count) end if // Copy from index to arrays end list_packages fn find_file_owner(file_path: string): string let index = load_files_index("") if not index.loaded return database.find_owner(file_path) // Fallback end if return binary_search_file(index, normalize_path(file_path)) end find_file_owner fn get_indexed_package(name: string): PkgIndexEntry let index = load_packages_index("") if not index.loaded return fallback_get_indexed_package(name) end if // Search index for package by name end get_indexed_package ``` Add rooted variants: - `find_file_owner_rooted(file_path: string, root: string): string` ### Phase 5: Command Integration **install.reef** - After registering package: ```reef // Rebuild indexes after install pkgdb.rebuild_indexes_rooted(root_path) ``` **remove.reef** - After unregistering package: ```reef // Rebuild indexes after remove pkgdb.rebuild_indexes_rooted(root_path) ``` **owner.reef** - Use indexed lookup: ```reef // Change from database.find_owner() to pkgdb.find_file_owner() ``` --- ## Files to Modify | File | Changes | |------|---------| | `src/core/pkgdb.reef` | Main implementation - add types, serialization, deserialization, binary search | | `src/commands/install.reef` | Call `rebuild_indexes_rooted()` after registration | | `src/commands/remove.reef` | Call `rebuild_indexes_rooted()` after unregistration | | `src/commands/owner.reef` | Use `pkgdb.find_file_owner()` instead of `database.find_owner()` | --- ## Key Implementation Details ### Base64 Encoding (required for file I/O) Reef's `encoding.base64` module exists in reef-stdlib. Use: ```reef import encoding.base64 let encoded = base64.base64_encode(binary_string) let decoded = base64.base64_decode(encoded) ``` **Issue:** base64 module works with strings, but MsgPack produces `[int]` buffer. Need conversion helpers. ### MsgPack Usage (from reef-stdlib/encoding/msgpack.reef) ```reef import encoding.msgpack // Packing let buf = msgpack.msgpack_alloc_buffer(size) let pos = 0 pos = pos + msgpack.msgpack_pack_map_header(4, buf, pos) pos = pos + msgpack.msgpack_pack_string("version", buf, pos) pos = pos + msgpack.msgpack_pack_int(1, buf, pos) // ... etc // Unpacking let map_len = msgpack.msgpack_unpack_map_len(buf, offset) offset = offset + msgpack.msgpack_value_size(buf, offset) // ... iterate through map entries ``` ### Buffer to String Conversion Need helper to convert `[int]` buffer to string for base64: ```reef fn buffer_to_string(buf: [int], length: int): string mut result = "" mut i = 0 while i < length result = result + chr(buf[i]) i = i + 1 end while return result end buffer_to_string ``` --- ## Verification ### Test Commands ```bash # Initialize database (should create packages.db and files.db) sudo coral --root /tmp/coral-chroot db init ls -la /tmp/coral-chroot/var/lib/coral/db/ # Verify files exist file /tmp/coral-chroot/var/lib/coral/db/packages.db file /tmp/coral-chroot/var/lib/coral/db/files.db # Build and install a test package coral build core/zlib sudo coral --root /tmp/coral-chroot install zlib # Verify index updated sudo coral --root /tmp/coral-chroot db status # Test fast list time sudo coral --root /tmp/coral-chroot list # Test fast owner lookup time sudo coral --root /tmp/coral-chroot owner /usr/lib/libz.so # Test rebuild from scratch sudo rm /tmp/coral-chroot/var/lib/coral/db/*.db sudo coral --root /tmp/coral-chroot db rebuild ``` ### Performance Validation - `coral list` with 100 packages should complete in <50ms - `coral owner` should complete in <10ms regardless of file count - Index files should be created after install/remove --- ## Design Decisions 1. **Full Rebuild on Changes:** Each install/remove triggers a complete index rebuild. Simpler, always consistent, acceptable performance for typical package counts (<500 packages). 2. **Trust the Index:** Commands trust the index is correct. If indexes get out of sync (e.g., manual edits to installed/ directory), user runs `coral db rebuild` to fix. 3. **No Caching:** Reload index from disk on each command invocation. Keeps implementation simple and avoids stale memory issues. # Design: Package Format v2 **Date:** 2026-03-07 **Status:** Draft **Target:** Coral v0.3 **Authors:** Zygaena Team ## Overview Redesign of the Coral package internal format to eliminate the rsync dependency, add a rich file manifest for install/remove/verify operations, embed dependency metadata in packages, fix config file handling, and clean up the scripting model. These changes are possible now because Reef v0.5.0 provides native filesystem operations (`fs.stat`, `fs.perm`, `fs.link`, `fs.ops`) that replace the need for shell-outs to rsync, cp, chmod, chown, and ln. No package has shipped to production yet, so there is no backward-compatibility constraint. --- ## 1. Package Archive Structure The archive format remains `.pkg.tar.xz` (gtar + xz). The internal layout changes: ``` name#version-release.pkg.tar.xz .PKGINFO # TOML — package metadata + dependencies .MANIFEST # mtree-format file inventory (replaces .FOOTPRINT) .CONFIG # list of config-protected paths (unchanged) .SCRIPTS/ # optional hook scripts (zsh only) pre-install.sh post-install.sh pre-remove.sh post-remove.sh pre-upgrade.sh post-upgrade.sh usr/ etc/ ... # payload files at their install paths ``` ### Changes from v1 | Item | v1 | v2 | |------|----|----| | File manifest | `.FOOTPRINT` (flat path list) | `.MANIFEST` (mtree with metadata) | | Dependencies | not in package | `[dependencies]` in `.PKGINFO` | | Script hooks | 4 (install/remove) | 6 (+ upgrade pair) | | Script runners | `.reef`, `.sh`, bare | `.sh` (zsh) and bare only | | Install method | `rsync -a` | `fs.ops` driven by manifest | --- ## 2. .PKGINFO — Package Metadata TOML format. Adds a `[dependencies]` section with optional version constraints. ```toml [package] name = "openssl" version = "3.2.1" release = 1 description = "TLS/SSL and crypto library" url = "https://www.openssl.org" license = "Apache-2.0" maintainer = "admin@zygaena.org" arch = "x86_64" [dependencies] runtime = ["zlib", "libc"] build = ["perl", "makedepend"] ``` ### Version constraints Runtime and build dependencies support optional version operators: ```toml runtime = ["zlib>=1.3", "libc"] ``` Supported operators: `>=`, `<=`, `>`, `<`, `=`. No operator means "any version". Most dependencies will be unversioned. Version constraints are added only when a specific ABI or feature is required (e.g., a soname bump, a new API entry point). Exact version pinning (`=`) is discouraged — it creates rebuild cascades. The port's `package.toml` remains the authoritative source. The `[dependencies]` section in `.PKGINFO` is a copy embedded at build time so standalone packages are self-describing without the ports tree. ### Generation `generate_pkginfo()` in `build.reef` writes the `[dependencies]` section by reading `ctx.port.deps.runtime` and `ctx.port.deps.build`. --- ## 3. .MANIFEST — mtree File Inventory Replaces `.FOOTPRINT`. Uses mtree-style format: one line per filesystem entry, path first, then space-separated `key=value` attributes. ### Format ``` #mtree ./usr/bin type=dir mode=0755 uname=root gname=root ./usr/bin/passwd type=file mode=4755 uname=root gname=sys sha256=a1b2c3d4e5f6... ./usr/bin/zlib-flate type=file mode=0755 uname=root gname=root sha256=d4e5f6a7... ./usr/include type=dir mode=0755 uname=root gname=root ./usr/include/zconf.h type=file mode=0644 uname=root gname=root sha256=e5f6a7b8... ./usr/lib type=dir mode=0755 uname=root gname=root ./usr/lib/libz.a type=file mode=0644 uname=root gname=root sha256=9c0d1e2f... ./usr/lib/libz.so type=link uname=root gname=root link=libz.so.1 ./usr/lib/libz.so.1 type=link uname=root gname=root link=libz.so.1.3.1 ./usr/lib/libz.so.1.3.1 type=file mode=0755 uname=root gname=root sha256=f0a1b2... ``` ### Fields | Key | Required | Values | Notes | |-----|----------|--------|-------| | `type` | always | `file`, `dir`, `link` | First attribute after path | | `mode` | files, dirs | 4-digit octal | Includes setuid/setgid/sticky | | `uname` | always | user name | String, not numeric uid | | `gname` | always | group name | String, not numeric gid | | `sha256` | files only | 64-char hex | For install verification and `coral verify` | | `link` | symlinks only | target path | Relative or absolute link target | ### Sort order Entries are sorted lexicographically by path. This ensures: - Directories appear before their contents (install order: top-down) - Removal in reverse order processes files before their parent directories ### Extensibility Unknown `key=value` pairs are ignored by the parser. Future attributes (e.g., `xattr=...`, `acl=...`, `flags=...`) can be added without breaking existing Coral versions. However, ACLs and extended attributes are better handled in post-install scripts where policy decisions belong. ### Metadata exclusion `.PKGINFO`, `.MANIFEST`, `.CONFIG`, and `.SCRIPTS/` are not listed in the manifest. They are package metadata, not payload files. ### Generation `generate_manifest()` in `build.reef` replaces `generate_footprint()`. It walks the staging directory using `fs.stat` and `fs.perm` to read type, mode, owner, and group for each entry. File checksums are computed with `checksum.sha256_file()`. Symlink targets are read with `fs.link.readlink()`. ### Parser module New module: `util/mtree.reef` ``` export type ManifestEntry fn parse_manifest(content: string, entries: [ManifestEntry], max: int): int fn generate_manifest(pkg_dir: string): string fn entry_type(entry: ManifestEntry): string fn entry_path(entry: ManifestEntry): string fn entry_mode(entry: ManifestEntry): int fn entry_sha256(entry: ManifestEntry): string fn entry_link(entry: ManifestEntry): string fn entry_uname(entry: ManifestEntry): string fn entry_gname(entry: ManifestEntry): string end export ``` --- ## 4. Install Method — Replacing rsync ### Current flow (v1) ``` extract .pkg.tar.xz to temp dir → find symlinks, remove conflicts in root → rsync -a --exclude=metadata temp_dir/ root/ ``` ### New flow (v2) ``` extract .pkg.tar.xz to temp dir → parse .MANIFEST into entry array → for each entry (in manifest order): if type=dir: fs.ops.create_dir_all(root + path) -- create if missing fs.perm.chmod(root + path, mode) -- set permissions fs.perm.chown(root + path, uname, gname) -- set ownership if type=file: check if config-protected (see section 6) if config file with user modifications → install as .pkg else: remove any existing conflicting path (file, symlink, or dir) fs.ops.copy_file_preserve(src, root + path) fs.perm.chmod(root + path, mode) fs.perm.chown(root + path, uname, gname) verify sha256 matches manifest if type=link: remove any existing conflicting path fs.link.symlink(link_target, root + path) ``` ### Advantages over rsync - **Zero external dependencies** — no rsync required during bootstrap - **Manifest-driven** — every file placed is accounted for, no silent extras - **Per-file verification** — sha256 checked on install, not just blindly copied - **Config protection integrated** — checked inline during install, not as a separate pass - **Explicit permissions** — set from manifest, not inherited from build host filesystem quirks ### Error handling If any file operation fails, installation stops and returns an error. Partial installs are recorded so `coral remove` can clean up. This is safer than rsync's `exit code 23` (partial transfer accepted). --- ## 5. Scripting Model ### Hooks Six hooks, all executed with `zsh`: | Hook | Arguments | When | Abort on failure | |------|-----------|------|------------------| | `pre-install.sh` | `$VERSION` | Before files installed (fresh install) | Yes | | `post-install.sh` | `$VERSION` | After files installed (fresh install) | No* | | `pre-remove.sh` | `$VERSION` | Before files removed (full remove) | Yes | | `post-remove.sh` | `$VERSION` | After files removed (full remove) | No* | | `pre-upgrade.sh` | `$OLD_VERSION $NEW_VERSION` | Before files replaced (upgrade) | Yes | | `post-upgrade.sh` | `$OLD_VERSION $NEW_VERSION` | After new files installed (upgrade) | No* | *Post-scripts log failures as warnings but do not roll back, since the file operation has already completed. ### Execution All scripts are invoked as: ``` zsh "/path/to/.SCRIPTS/hook-name.sh" [args...] ``` The `.reef` script type is removed. Requiring the Reef compiler at install time creates a bootstrap dependency that cannot be satisfied when building the base OS. Bare executables (no extension) are still supported — Coral executes them directly, relying on a shebang line or ELF header. ### Script search order For each hook, Coral looks for (in order): 1. `hook-name.sh` — executed with `zsh` 2. `hook-name` — executed directly (must be executable with valid shebang or ELF) If neither exists, the hook is silently skipped (not an error). ### Upgrade flow ``` pre-upgrade.sh $OLD_VERSION $NEW_VERSION # abort if non-zero → remove old files (driven by installed manifest) → install new files (driven by new manifest) post-upgrade.sh $OLD_VERSION $NEW_VERSION # warn if non-zero ``` If no `pre-upgrade.sh` or `post-upgrade.sh` exists, Coral performs the file swap silently. It does **not** fall back to running pre-remove + post-install, as those hooks may have side effects inappropriate for an upgrade (e.g., post-install creating a fresh database vs. preserving an existing one). ### Stored scripts Remove scripts (`pre-remove.sh`, `post-remove.sh`) must be available when a package is removed, even if the original `.pkg.tar.xz` is gone. Coral stores a copy of `.SCRIPTS/` in the package database at install time (`/var/lib/coral/installed//scripts/`). Upgrade scripts are stored there as well, since they're needed when a future version replaces the current one. --- ## 6. Config File Protection — Checksum-Based ### Current behavior (v1, broken) Reads both files fully into memory and compares content with `str.equals()`. Both branches of the comparison (equal / not equal) execute the same code path — always installs as `.pkg`. The "files are identical" case should overwrite in place. ### New behavior (v2) Uses the sha256 checksum from the manifest to detect user modifications without reading the full installed file into memory twice. **On fresh install:** 1. Install config file normally to `dest_path` 2. Store the package-provided sha256 in the database for this file **On upgrade (file already exists):** 1. Compute sha256 of the currently installed file 2. Compare against the stored checksum from the *previously installed* version 3. Decision: | Installed matches old pkg checksum? | Action | |-------------------------------------|--------| | Yes (user has not modified) | Overwrite with new version | | No (user has modified) | Install new version as `dest_path.pkg-new`, keep user's file | 4. Update stored checksum to the new package's sha256 **On remove:** - Config files listed in `.CONFIG` are **not** removed if they differ from the package-provided checksum (user has modified them) - Config files that match the package checksum are removed normally This matches the proven dpkg conffile model. ### Implementation ``` fn install_config_file(src: string, dest: string, manifest_sha256: string, stored_sha256: string): bool if not fs.stat.exists(dest) // Fresh install — copy file fs.ops.copy_file_preserve(src, dest) return true end if // File exists — check if user modified it let installed_sha256 = checksum.sha256_file(dest) if installed_sha256 == stored_sha256 // User has not modified — safe to overwrite fs.ops.copy_file_preserve(src, dest) return true else // User has modified — preserve their version let new_path = dest + ".pkg-new" fs.ops.copy_file_preserve(src, new_path) // Log: "Config file preserved, new version at dest.pkg-new" return true end if end install_config_file ``` --- ## 7. Implementation Plan ### Phase 1: mtree module and manifest generation - New module `util/mtree.reef` — parser and generator - Update `build.reef`: replace `generate_footprint()` with `generate_manifest()` - Use `fs.stat`, `fs.perm`, `fs.link.readlink()`, `checksum.sha256_file()` ### Phase 2: fs.ops-based installer - Rewrite `install_files()` in `package.reef` to walk the manifest - Use `fs.ops.copy_file_preserve`, `fs.perm.chmod`, `fs.perm.chown`, `fs.link.symlink` - Remove rsync dependency from install path - Integrate config file checksum logic ### Phase 3: Enhanced .PKGINFO - Update `generate_pkginfo()` to emit `[dependencies]` section - Update `read_pkginfo()` to parse dependencies - Add `deps` field to `PackageInfo` type or extend existing `Dependencies` ### Phase 4: Script model cleanup - Add `pre-upgrade.sh` / `post-upgrade.sh` hook support - Remove `.reef` script type from `run_script()` and `run_stored_script()` - Change `.sh` invocation from `sh` to `zsh` - Pass version arguments to all hooks - Abort on non-zero exit from pre-* hooks - Update upgrade command to call upgrade hooks ### Phase 5: Removal and verify - Rewrite `remove_files()` to use manifest (directories tracked explicitly) - Use `fs.ops.remove_file`, `fs.ops.remove_tree`, `dir.remove_dir` - Implement `coral verify ` — walk stored manifest, check sha256 and permissions against live filesystem ### Dependencies - Reef >= 0.5.0 (for `fs.stat`, `fs.perm`, `fs.link`, `fs.ops`) - rsync is no longer required for package installation (remains used by `coral repo mirror` only) --- ## 8. Risks and Mitigations | Risk | Mitigation | |------|------------| | fs.ops.copy_file_preserve slower than rsync for large packages | Benchmark during Phase 2; rsync does buffered I/O and copy_file_preserve uses 8KB buffer — comparable. Manifest-driven install skips metadata files without needing exclude flags. | | mtree parser bugs cause install failures | Comprehensive test coverage in Phase 1; mtree is a simple line-oriented format with no nesting or escaping edge cases beyond spaces in paths. | | Packages with >8192 files overflow entry array | Use manifest file_count pre-scan or dynamic sizing. Current .FOOTPRINT has same limit. | | Upgrade hook ordering with dependency chains | Out of scope for this design. Coral currently upgrades packages individually. Dependency-ordered upgrades (topological sort) are a future enhancement. | # Package Format v2 — Task List **Design:** `docs/PACKAGE_FORMAT_V2_DESIGN.md` **Target:** Coral v0.3 **Date:** 2026-03-07 **Requires:** Reef >= 0.5.0 --- ## Phase 1: mtree module and manifest generation — COMPLETE - [x] Create `src/util/mtree.reef` module with `ManifestEntry` type - [x] Implement `parse_manifest()` — line-oriented parser, split on space, extract key=value pairs - [x] Implement `generate_manifest()` — walk directory tree using `fs.stat`, `fs.perm`, `fs.link.readlink()`, `checksum.sha256_file()` - [x] Implement accessor functions: `entry_type`, `entry_path`, `entry_mode`, `entry_sha256`, `entry_link`, `entry_uname`, `entry_gname` - [x] Handle sort order: lexicographic by path so dirs precede their contents - [x] Skip metadata files (`.PKGINFO`, `.MANIFEST`, `.CONFIG`, `.SCRIPTS/`) during generation - [x] Update `build.reef`: replace `generate_footprint()` with `generate_manifest()` using new mtree module - [x] Rename output file from `.FOOTPRINT` to `.MANIFEST` in build output - [ ] Test: build a package, verify `.MANIFEST` output contains correct types, modes, ownership, checksums - [ ] Test: parse a generated manifest back and verify round-trip correctness ## Phase 2: fs.ops-based installer — COMPLETE - [x] Rewrite `install_files()` in `package.reef` to parse `.MANIFEST` instead of `.FOOTPRINT` - [x] Implement dir install: `dir.create_dir_all()`, `fs.perm.chmod()`, `fs.perm.chown()` - [x] Implement file install: `fs.ops.copy_file_preserve()`, `fs.perm.chmod()`, `fs.perm.chown()` - [x] Implement symlink install: remove conflicting target, `fs.link.symlink()` - [x] Add post-copy sha256 verification against manifest checksum - [x] Integrate config file checksum logic (see Phase 2b below) - [x] Remove rsync call from `install_files()` - [x] Remove symlink conflict prep_cmd shell-out (handle inline during manifest walk) - [x] Error handling: stop on any failed file operation - [x] Update `read_footprint()` to read `.MANIFEST` first, fall back to `.FOOTPRINT` - [x] Update `register_from_extract_dir()` to store `manifest.mtree` in database - [x] Update `remove_files()` to use `fs.ops.remove_file` instead of shell-outs - [x] Add `remove_files_manifest()` for manifest-driven removal with explicit directory tracking - [x] Clean up stale rsync comments in `install.reef` - [ ] Test: install a package to a test root, verify all files/dirs/symlinks match manifest - [ ] Test: install package with setuid binary (mode=4755), verify mode preserved - [ ] Test: install package with symlinks, verify targets correct ### Phase 2b: Config file checksum logic — COMPLETE - [x] Update `install_config_file()` to accept manifest sha256 and stored sha256 parameters - [x] On fresh install: copy file normally - [x] On upgrade (file exists, matches old checksum): overwrite with new version - [x] On upgrade (file exists, differs from old checksum): install as `.pkg-new`, preserve user's file - [x] Remove old content-comparison logic (`str.equals(src_content, dest_content)`) - [x] On remove: skip config files where installed sha256 differs from package sha256 - [ ] Test: fresh install places config file and records checksum - [ ] Test: upgrade with unmodified config overwrites cleanly - [ ] Test: upgrade with user-modified config installs `.pkg-new` and preserves original ## Phase 3: Enhanced .PKGINFO — COMPLETE - [x] Add `runtime_deps`, `build_deps`, `runtime_deps_count`, `build_deps_count` fields to `PackageInfo` type in `types.reef` - [x] Update `new_package_info()` factory to initialize new fields - [x] Update `generate_pkginfo()` in `build.reef` to emit `[dependencies]` section - [x] Write `runtime` array from `ctx.port.deps.runtime` - [x] Write `build` array from `ctx.port.deps.build` - [x] Skip `[dependencies]` section if both arrays are empty - [x] Update `read_pkginfo()` in `package.reef` to parse `[dependencies]` section - [x] Handle legacy packages without `[dependencies]` gracefully - [x] Add `parse_toml_array()` helper to `package.reef` - [x] Handle version constraint syntax in dependency strings (`>=`, `<=`, `>`, `<`, `=`) - [x] Add version comparison utility for constraint checking - [ ] Test: build a package with dependencies, verify `.PKGINFO` contains correct `[dependencies]` - [ ] Test: parse `.PKGINFO` with dependencies and verify fields populated ## Phase 4: Script model cleanup — COMPLETE - [x] Remove `.reef` script type from `run_script()` in `package.reef` - [x] Remove `.reef` script type from `run_stored_script()` in `package.reef` - [x] Change `.sh` invocation from `sh` to `zsh` in `run_script()` - [x] Change `.sh` invocation from `sh` to `zsh` in `run_stored_script()` - [x] Update script search order: try `.sh` first, then bare executable - [x] Add `pre-upgrade.sh` / `post-upgrade.sh` hook support - [x] Pass `$VERSION` argument to pre-install, post-install, pre-remove, post-remove - [x] Pass `$OLD_VERSION $NEW_VERSION` arguments to pre-upgrade, post-upgrade - [x] Abort install/remove/upgrade if pre-* hook exits non-zero - [x] Log warning (don't abort) if post-* hook exits non-zero - [x] Store upgrade scripts in database alongside remove scripts at install time - [x] Implement upgrade flow: pre-upgrade → remove old files → install new files → post-upgrade - [ ] Test: package with pre-install that exits 0 — install proceeds - [ ] Test: package with pre-install that exits 1 — install aborts - [ ] Test: package with post-install that exits 1 — install completes with warning - [ ] Test: upgrade with pre-upgrade/post-upgrade scripts, verify version args passed ## Phase 5: Removal and verify — COMPLETE - [x] Rewrite `remove_files()` to use `fs.ops` instead of shell-outs (done in Phase 2) - [x] Add `remove_files_manifest()` with explicit directory tracking (done in Phase 2) - [x] Respect config file protection on remove (skip user-modified configs) (done in Phase 2b) - [x] Implement `coral verify ` command - [x] Read stored manifest from database - [x] For each file entry: check exists, compare sha256, compare mode/ownership - [x] For each symlink entry: check exists, verify link target - [x] For each dir entry: check exists - [x] Report: missing files, checksum mismatches, permission changes, broken symlinks - [ ] Test: remove a package, verify all files deleted and empty dirs cleaned up - [ ] Test: remove a package with user-modified config, verify config preserved - [ ] Test: verify a clean install — all files pass - [ ] Test: verify after modifying a file — checksum mismatch reported --- ## Cleanup - [x] Remove `generate_footprint()` from `build.reef` (replaced by `generate_manifest()`) - [x] Update `read_footprint()` to read `.MANIFEST` via mtree parser (with `.FOOTPRINT` fallback) - [x] Update `verify_package()` to check for `.MANIFEST` or `.FOOTPRINT` - [x] Remove rsync from install path (only remains in `repo.reef` mirror command) - [x] Clean up rsync comments in `install.reef` - [x] Update `docs/GUIDE.md` — fix .reef references, sh→zsh, .pkg→.pkg-new, add verify command, add version constraints, add upgrade scripts # Refactor: Source Mirrors Support **Status:** Planning **Version:** 0.1.3 **Date:** January 2026 --- ## Overview Refactor the source download system to support multiple mirror URLs per source file, with automatic fallback when primary sources are unavailable. --- ## Current Design ```toml [sources] urls = ["https://example.com/file.tar.gz"] checksums = ["sha256:abc123..."] ``` **Problems:** - Multiple URLs treated as separate files, not mirrors - No fallback on download failure - Checksum tied to array index, fragile --- ## New Design ### Package.toml Format ```toml [[source]] file = "vim-9.1.0.tar.gz" urls = [ "https://github.com/vim/vim/archive/v9.1.0.tar.gz", "https://mirror.example.com/vim-9.1.0.tar.gz", "https://backup.example.org/vim-9.1.0.tar.gz" ] checksum = "sha256:abc123..." [[source]] file = "illumos-compat.patch" urls = ["https://patches.zygaena.org/vim/illumos.patch"] checksum = "sha256:def456..." extract = false ``` ### New Fields | Field | Required | Default | Description | |-------|----------|---------|-------------| | `file` | Yes | - | Output filename (what to save as) | | `urls` | Yes | - | Array of mirror URLs (tried in order) | | `checksum` | No | "SKIP" | SHA256 checksum for verification | | `extract` | No | true | Whether to extract archive | --- ## Implementation Plan ### Phase 1: Type Definitions (types.reef) **New struct:** ```reef type Source = struct file: string // Output filename urls: [string] // Mirror URLs (tried in order) url_count: int // Number of URLs checksum: string // SHA256 checksum extract: bool // Whether to extract (default: true) end Source ``` **Update SourceInfo:** ```reef type SourceInfo = struct sources: [Source] // Array of Source structs count: int // Number of sources end SourceInfo ``` **Add factory function:** ```reef fn new_source(): Source ``` ### Phase 2: TOML Parsing (port.reef) Add support for parsing `[[source]]` table arrays: 1. Detect `source.` prefixed keys 2. Group by index (source.0.file, source.0.urls, etc.) 3. Parse each source into Source struct 4. Handle both old format (backward compat) and new format **Backward Compatibility:** - If `sources.urls` exists (old format), convert to new format - Each URL becomes a separate Source with auto-detected filename - Checksums mapped by index ### Phase 3: Download Logic (build.reef) Update `download_sources()`: ``` for each source in sources: if file exists in cache and checksum matches: skip download for each url in source.urls: try download to source.file if success: verify checksum if checksum ok: break to next source else: delete file, try next mirror else: try next mirror if all mirrors failed: return error ``` ### Phase 4: Extract Logic (build.reef) Update `extract_sources()`: ``` for each source in sources: if source.extract: extract archive else: copy file to work_dir (for patches, etc.) ``` ### Phase 5: Update Test Ports Convert test ports to new format: - test/ports/base/vim/package.toml - test/ports/base/tree/package.toml - test/ports/core/coral/package.toml - etc. --- ## File Changes Summary | File | Changes | |------|---------| | `src/types.reef` | Add Source struct, update SourceInfo, add new_source() | | `src/core/port.reef` | Parse [[source]] tables, backward compat for old format | | `src/commands/build.reef` | Mirror fallback in download, respect extract flag | | `src/util/http.reef` | Add download_with_mirrors() helper (optional) | | `test/ports/*/package.toml` | Update to new format | --- ## Example Migration ### Before (old format) ```toml [sources] urls = ["https://github.com/vim/vim/archive/v9.1.0.tar.gz"] checksums = ["SKIP"] ``` ### After (new format) ```toml [[source]] file = "vim-9.1.0.tar.gz" urls = [ "https://github.com/vim/vim/archive/v9.1.0.tar.gz" ] checksum = "SKIP" ``` ### With mirrors ```toml [[source]] file = "vim-9.1.0.tar.gz" urls = [ "https://github.com/vim/vim/archive/v9.1.0.tar.gz", "https://ftp.nluug.nl/pub/vim/unix/vim-9.1.tar.bz2", "https://mirror.freedif.org/vim/unix/vim-9.1.tar.bz2" ] checksum = "sha256:1234567890abcdef..." ``` --- ## Testing Checklist - [ ] Single source, single URL (basic case) - [ ] Single source, multiple URLs (mirror fallback) - [ ] Multiple sources - [ ] Primary URL fails, mirror succeeds - [ ] All mirrors fail (error handling) - [ ] Checksum verification after download - [ ] Checksum mismatch triggers next mirror - [ ] `extract = false` for non-archive files - [ ] Backward compatibility with old format - [ ] Cached file with valid checksum skips download --- ## Future Enhancements - **Priority/weight** for mirrors (prefer faster/closer) - **Parallel downloads** from multiple mirrors - **Resume partial downloads** - **Mirror health tracking** (remember failed mirrors) # Coral Roadmap **Target:** Coral v0.4.x **Date:** 2026-03-08 **Tracks:** Unwired features, cleanup, and new functionality --- ## Phase 1: Cleanup - [x] **1. Remove redundant `opts.no_color` field** - Removed `GlobalOptions.no_color` from `types.reef` and `main.reef`; color module reads flag directly - [x] **2. Remove legacy `SourceInfo.urls` / `SourceInfo.checksums` fields** - Removed unused backward-compat fields from `types.reef` and dead writes from `port.reef` - [x] **3. Clean up resolver exports** - Removed `resolve_alternative()`, `is_alternative_dep()`, `parse_alternatives()` from export block --- ## Phase 2: High Priority — Features users expect to work - [x] **4. Implement `--dry-run` / `-n` flag** - Added dry-run support to install, remove, build, and upgrade commands - [x] **5. Wire up `fallback_to_source` config option** - Install falls back to `coral build -y` when no binary package found and `fallback_to_source = true` - [x] **6. Implement package conflict detection** - Install checks `conflicts` field against installed packages; `--force` overrides - [x] **7. Implement signature verification on install** - When `require_signed_packages = true`, install verifies `.sig` file before proceeding - [x] **8. Implement `compress_man` build step** - After build, gzips uncompressed man pages in staging dir when `compress_man = true` --- ## Phase 3: Medium Priority — Partially wired features - [x] **9. Wire up `keep_packages` / `keep_sources` config** - `clean` command respects `keep_packages`/`keep_sources`; `--force` overrides protection - [x] **10. Wire up `use_proxy` config** - HTTP module accepts proxy config; curl disables proxy by default, enables when `use_proxy = true` - [x] **11. Wire up `auto_refresh` config** - Install and upgrade rebuild port index before execution when `auto_refresh = true` - [x] **12. Fix `coral diff` documentation** - Updated GUIDE.md to reference `coral outdated` instead of `coral diff` - [x] **13. Use specific exit codes throughout codebase** - Replaced generic `EXIT_ERROR()` with specific codes across all commands --- ## Completed - [x] Package Format v2 (mtree manifest, fs.ops installer, config protection) — v0.3.0 - [x] Script model rewrite (zsh, upgrade hooks, version args) — v0.3.0 - [x] Verify command — v0.3.0 - [x] CI update to Reef 0.5.0 — v0.3.0 - [x] Auto-install build dependencies — v0.3.1 - [x] Export $PREFIX, $SYSCONFDIR, $PKG_ROOT to build scripts — v0.3.2 - [x] Recursive build-from-source dependency resolver — v0.3.3 - [x] All 13 unwired features (cleanup, high-priority, medium-priority) — v0.3.5 # Stage 1: Coral Package Manager Implementation Plan **Document Version:** 2.5 **Created:** January 2026 **Last Updated:** January 2026 **Status:** In Progress (Stub Implementation Complete) ## Overview Stage 1 builds **Coral**, the package manager for Zygaena, written entirely in Reef. Coral is a single unified command with subcommands (similar to pacman/cargo). **Prerequisites:** Stage 0 Complete (Reef 0.1.3 installed on OmniOS at 192.168.168.148) --- ## Design Decisions | Decision | Choice | Rationale | |----------|--------|-----------| | **Name** | `coral` (ocean/reef theme) | Consistent with Zygaena (shark) and Reef (language) naming | | **Architecture** | Single binary with subcommands | Simpler for users, like pacman/cargo/git | | **Source Location** | `/home/ctusa/repos/zygaena/coral/` | Part of main zygaena repo for coordinated development | | **Install Location** | `/usr/local/bin/coral` (OmniOS), `/usr/bin/coral` (Zygaena) | Standard Unix paths | | **Compression** | xz (`.pkg.tar.xz`) | Good compression, widely available on illumos | | **HTTP** | Native Reef `net.http` (report issues to Reef team) | Avoid shell exec where possible | | **Privileges** | Root required for all operations | Multi-user system security | | **Confirmation** | Ask by default, `-y` to skip | Safe default, scriptable override | | **Colors** | Yes, when terminal supports it | Better UX, respects NO_COLOR standard | | **Port Categories** | core, base, devel, desktop, extra | Clear separation of concerns | | **Distribution** | Binary + Source packages | Binary for speed, source as fallback | | **Repository Access** | Local filesystem + HTTP | Both equally supported (ISO/USB and mirrors) | | **Package Signing** | Ed25519/Signify | Simple, minimal dependencies, proven security | | **Config Format** | TOML | Human-readable, matches port definitions | | **Third-party Repos** | Supported from day one | Enables community packages | --- ## Command Structure ``` coral [options] [packages...] BUILD/CREATE: coral build Build package from port directory coral build . Build from current directory INSTALL/REMOVE: coral install Install packages (resolves dependencies) coral remove Remove packages coral upgrade [pkg...] Upgrade packages (all if none specified) QUERY: coral info Show package details coral list List installed packages coral list -a List all available packages (from repos) coral files List files owned by package coral owner Find which package owns file coral search Search available packages coral depends Show dependency tree PORTS: coral sync Update ports collection and repo metadata (git/hg) coral diff Show packages with available updates REPOSITORY MANAGEMENT: coral repo init Create new repository structure coral repo add Add packages to repository coral repo remove Remove package from repository coral repo list List packages in repository coral repo rebuild Regenerate repo.toml manifest coral repo sign Sign repository manifest coral repo verify Verify repository signatures coral repo sync Rsync repository to mirror coral repo export Export repository for ISO/USB KEY MANAGEMENT: coral key init Initialize keyring directory coral key generate Generate new Ed25519 signing keypair coral key list List available keys coral key import Import public key coral key export Export public key coral key trust Mark key as trusted coral key revoke Revoke trust for key CACHE MANAGEMENT: coral cache clean Remove cached packages coral cache clean --sources Remove cached source tarballs coral cache clean --all Remove all cached files coral cache list Show cache contents and sizes GLOBAL OPTIONS: -y, --yes Don't ask for confirmation -f, --force Force operation (overwrite conflicts) -v, --verbose Verbose output -n, --dry-run Show what would be done -q, --quiet Minimal output --no-color Disable colored output --root Use alternative root (for building ISOs) EXAMPLES: coral install vim # Install vim and dependencies coral install vim nano -y # Install multiple, no confirmation coral build /usr/ports/core/hello coral search editor coral upgrade # Upgrade all packages coral remove firefox -y coral repo init /srv/packages coral key generate official ``` --- ## Configuration ### Main Configuration File Location: `/etc/coral/coral.conf` ```toml # Coral Package Manager Configuration [general] # System architecture arch = "x86_64" # Number of parallel build jobs (0 = auto-detect CPU count) jobs = 0 # Ask for confirmation before operations (override with -y) confirm = true # Enable colored output (respects NO_COLOR env var) color = true [paths] # Local ports collection ports = "/usr/ports" # Built package cache packages = "/var/lib/coral/packages" # Downloaded source tarballs sources = "/var/lib/coral/sources" # Installed package database database = "/var/lib/coral/installed" # Repository metadata cache repos = "/var/lib/coral/repos" # Build working directory (cleaned after build) work = "/var/tmp/coral/work" # Package staging directory pkg = "/var/tmp/coral/pkg" # Signing keys directory keys = "/etc/coral/keys" [build] # Default CFLAGS for building packages cflags = "-O2 -pipe -march=x86-64" # Default CXXFLAGS (defaults to cflags if empty) cxxflags = "${cflags}" # Make flags makeflags = "-j${jobs}" # Strip binaries after build strip = true # Compress man pages with gzip compress_man = true # What to do when binary package not available: # "ask" - prompt user to build from source # "yes" - automatically build from source # "no" - fail if binary not available fallback_to_source = "ask" [network] # Connection timeout in seconds timeout = 30 # Number of retry attempts retries = 3 # Use system proxy settings use_proxy = true [sync] # Auto-refresh repository metadata (hours, 0 = manual only) auto_refresh = 24 [cache] # Keep downloaded packages after installation keep_packages = true # Keep downloaded source tarballs keep_sources = true [security] # Require repositories to have valid signatures require_signed_repos = true # Require packages to have valid signatures require_signed_packages = true ``` ### Repository Configuration Directory Location: `/etc/coral/repos.d/` Each repository is configured in a separate TOML file. #### Official Repository Example File: `/etc/coral/repos.d/official.toml` ```toml [repository] # Unique identifier name = "official" # Human-readable description description = "Official Zygaena packages" # Enable/disable this repository enabled = true # Priority (higher = preferred when same package in multiple repos) priority = 100 # Mirrors to try (in order) [[mirrors]] url = "https://pkg.zygaena.org/stable/${arch}/" [[mirrors]] url = "https://mirror.example.com/zygaena/${arch}/" # Local fallback (e.g., installation media) [[mirrors]] url = "file:///mnt/cdrom/packages/" [security] # Require signed packages from this repo require_signed = true # Public key file for signature verification keyfile = "/etc/coral/keys/official.pub" ``` #### Third-Party Repository Example File: `/etc/coral/repos.d/community.toml` ```toml [repository] name = "community" description = "Community-maintained packages" enabled = true priority = 50 [[mirrors]] url = "https://community.example.org/zygaena/${arch}/" [security] require_signed = true keyfile = "/etc/coral/keys/community.pub" ``` --- ## Repository Manifest Format Each repository publishes a `repo.toml` manifest that coral downloads to understand available packages. ### Manifest Structure File: `repo.toml` (at repository root) ```toml [repository] # Repository identifier (must match repos.d config) name = "official" # Human-readable description description = "Official Zygaena packages" # Maintainer contact maintainer = "[email protected]" # Target architecture arch = "x86_64" # When this manifest was generated (ISO 8601) generated = 2026-01-23T12:00:00Z # Total number of packages package_count = 150 # Base URL for package downloads (relative to manifest location) base_url = "packages/" # Signing key identifier signing_key = "zygaena-official-2026" # Package entries [[package]] name = "bash" version = "5.2.21" release = 1 description = "The GNU Bourne Again shell" license = "GPL-3.0" category = "core" depends = ["glibc", "readline", "ncurses"] optdepends = [] provides = ["sh"] conflicts = [] size = 7834521 # Installed size in bytes compressed = 1823456 # Package file size in bytes filename = "bash-5.2.21-1.pkg.tar.xz" sha256 = "a1b2c3d4e5f6789..." signature = "sig:ed25519:..." # Optional inline signature [[package]] name = "vim" version = "9.1.0" release = 1 description = "Vi IMproved - enhanced vi editor" license = "Vim" category = "base" depends = ["ncurses"] optdepends = ["python:Python scripting support", "lua:Lua scripting support"] provides = ["vi"] conflicts = [] size = 12456789 compressed = 3456789 filename = "vim-9.1.0-1.pkg.tar.xz" sha256 = "fedcba987654321..." signature = "sig:ed25519:..." # ... more packages ... ``` ### Manifest Signature The manifest itself is signed. Signature stored in: `repo.toml.sig` ``` -----BEGIN ED25519 SIGNATURE----- -----END ED25519 SIGNATURE----- ``` --- ## Package Signing ### Key Format Coral uses Ed25519 keys (same as OpenBSD signify). **Private key:** `.sec` ``` untrusted comment: coral secret key ``` **Public key:** `.pub` ``` untrusted comment: coral public key ``` ### Key Storage - System keys: `/etc/coral/keys/` - User keys: `~/.coral/keys/` - Trusted keys database: `/var/lib/coral/trusted-keys` ### Signing Workflow ```bash # Generate a keypair for repository signing coral key generate official # Sign the repository manifest coral repo sign /srv/packages # Verify a repository coral repo verify /srv/packages # Import a third-party public key coral key import community.pub # Mark the key as trusted coral key trust community ``` --- ## Source Code Structure ``` /home/ctusa/repos/zygaena/ ├── coral/ # Package manager source │ ├── reef.toml # Reef project config │ └── src/ │ ├── main.reef # Entry point, CLI dispatch │ ├── commands/ │ │ ├── build.reef # coral build │ │ ├── install.reef # coral install │ │ ├── remove.reef # coral remove │ │ ├── upgrade.reef # coral upgrade │ │ ├── info.reef # coral info │ │ ├── list.reef # coral list │ │ ├── files.reef # coral files │ │ ├── owner.reef # coral owner │ │ ├── search.reef # coral search │ │ ├── depends.reef # coral depends │ │ ├── sync.reef # coral sync │ │ ├── diff.reef # coral diff │ │ ├── repo.reef # coral repo (subcommands) │ │ ├── key.reef # coral key (subcommands) │ │ └── cache.reef # coral cache (subcommands) │ ├── core/ │ │ ├── port.reef # Port parsing (package.toml) │ │ ├── package.reef # Package archive handling │ │ ├── database.reef # Installed package database │ │ ├── resolver.reef # Dependency resolution │ │ ├── config.reef # Configuration loading │ │ ├── repository.reef # Repository manifest handling │ │ └── signing.reef # Ed25519 signing/verification │ ├── util/ │ │ ├── http.reef # HTTP downloads (wraps net.http) │ │ ├── archive.reef # Tar/xz operations │ │ ├── checksum.reef # SHA256 verification │ │ ├── color.reef # Terminal colors │ │ └── prompt.reef # User prompts │ └── types.reef # Shared type definitions ├── ports/ # Port collection (Stage 2) │ ├── core/ │ ├── base/ │ ├── devel/ │ ├── desktop/ │ └── extra/ ├── docs/ │ ├── STAGE0_BOOTSTRAP.md │ ├── STAGE1_CORAL_PLAN.md # This document │ └── ... └── bootstrap/ ``` --- ## Port Categories | Category | Description | Examples | |----------|-------------|----------| | **core** | Essential system packages | kernel, libc, coreutils, bash | | **base** | Standard Unix utilities | vim, git, curl, openssh | | **devel** | Development tools & libraries | gcc, cmake, python, rust, ncurses-dev | | **desktop** | X11, window managers, GUI apps | xlibre, windowmaker, firefox | | **extra** | Everything else | games, themes, misc utilities | --- ## Reef Language Notes ### Reserved Keywords **IMPORTANT:** `run` is a reserved keyword for Active Objects in Reef. Command handlers must use `execute()` instead of `run()`. ### Key Syntax Rules - Blocks end with `end` keyword: `end if`, `end for`, `end while`, `end FunctionName`, `end module` - `fn` returns a value, `proc` does not return a value - String interpolation: `"Hello, ${name}"` (simple variables only, not expressions) - Mutable variables: `mut count = 0` - Immutable variables: `let value = 42` - Logical NOT: `not` (not `!`) - Array allocation: `new [type](size)` - Module declaration: `module path.to.module` ... `end module` - Export block: `export` ... `end export` - Entry point (`main.reef`) does NOT use a module declaration ### Module Structure Pattern ```reef module module.name export // Public interface declarations fn public_function(): type proc public_procedure() end export // Implementation fn public_function(): type // ... end public_function // Private helpers (not in export block) fn private_helper(): type // ... end private_helper end module ``` ### TOML Parsing Pattern The Reef stdlib uses parallel arrays for TOML parsing: ```reef import encoding.toml let keys = toml.toml_alloc_keys() // Allocate keys array (max 256) let vals = toml.toml_alloc_values() // Allocate values array let count = toml.toml_parse(content, keys, vals) let name = toml.toml_get(keys, vals, count, "package.name") let release = toml.toml_get_int(keys, vals, count, "package.release") ``` --- ## Reef Standard Library Modules ```reef // Core imports for Coral import io.file // readFile, writeFile, fileExists, readBinaryFile import io.dir // list_dir, dir_exists, create_dir_all, current_dir import io.path // join_path, dirname, basename, extension import core.str // split, join, contains, starts_with, ends_with, trim_ws import sys.args // count, get, has_flag (simple CLI parsing) import sys.flag // FlagParser for rich CLI parsing with help generation import sys.process // process_spawn_shell, process_wait import sys.env // get_env, set_env, has_env import io.console // readLine, confirm, printErr import encoding.toml // toml_parse, toml_get, toml_get_int, toml_has_key import net.http // HTTP client (HTTPS support coming from Reef team) import time.clock // Timestamps import collections.list // Dynamic lists import collections.map // Hash maps import crypto.sha256 // SHA256 checksums (native, pure Reef implementation) ``` ### Native vs Shell-out | Functionality | Implementation | Notes | |---------------|----------------|-------| | **SHA256** | Native `crypto.sha256` | Pure Reef, supports string and binary data | | **TOML parsing** | Native `encoding.toml` | Uses parallel arrays pattern | | **HTTP** | Native `net.http` | HTTP now, HTTPS coming from Reef team | | **File I/O** | Native `io.file`, `io.dir` | Full support | | **Tar/xz** | Shell-out to `tar`, `xz` | System tools, acceptable | | **Patch** | Shell-out to `patch` | Apply source patches (-p) | | **Ed25519 signing** | Shell-out to `signify` | Until Reef adds crypto.ed25519 | | **Git** | Shell-out to `git` | For `coral sync` ports update | | **Mercurial** | Shell-out to `hg` | Alternative VCS for ports update | --- ## Core Data Types ```reef // types.reef - Shared type definitions // Package metadata from package.toml type PackageInfo = struct name: string version: string release: int description: string url: string license: string maintainer: string arch: string category: string alternatives: [string] // Alternative sets this package belongs to alternatives_count: int conflicts: [string] // Packages that conflict with this conflicts_count: int end PackageInfo type Dependencies = struct runtime: [string] build: [string] optional: [string] // Optional dependencies with descriptions runtime_count: int build_count: int optional_count: int end Dependencies type SourceInfo = struct urls: [string] checksums: [string] count: int end SourceInfo type PatchInfo = struct files: [string] // Patch filenames in order strip: int // Strip level for patch -p (default: 1) count: int // Number of patches end PatchInfo type ConfigInfo = struct files: [string] // Config files to protect (installed as .pkg) count: int end ConfigInfo type ScriptsInfo = struct pre_install: string // Script filename or empty post_install: string pre_remove: string post_remove: string end ScriptsInfo type BuildConfig = struct parallel: bool jobs: int end BuildConfig // Complete port definition type Port = struct info: PackageInfo deps: Dependencies sources: SourceInfo patches: PatchInfo config: ConfigInfo scripts: ScriptsInfo build_cfg: BuildConfig port_dir: string end Port // Installed package record type InstalledPackage = struct info: PackageInfo install_date: string install_reason: string // "explicit" or "dependency" files: [string] // All files owned by this package files_count: int config_files: [string] // Config files (for upgrade handling) config_count: int repository: string // Which repo it came from end InstalledPackage // Remote package from repository manifest type RemotePackage = struct info: PackageInfo depends: [string] optdepends: [string] provides: [string] conflicts: [string] size: int // Installed size compressed: int // Download size filename: string sha256: string signature: string depends_count: int end RemotePackage // Repository definition type Repository = struct name: string description: string enabled: bool priority: int mirrors: [string] mirror_count: int keyfile: string require_signed: bool end Repository // Build result type BuildResult = struct success: bool package_path: string error_message: string end BuildResult // Signing key type SigningKey = struct name: string public_key: string fingerprint: string trusted: bool created: string end SigningKey ``` --- ## Key Implementation Details ### 1. Main Entry Point (main.reef) ```reef // main.reef - Entry point (no module declaration) // Entry point files do NOT use module declarations in Reef import sys.args import commands.build import commands.install import commands.remove import commands.repo import commands.key import commands.cache // ... other imports proc main() // Check for root (except for query commands) if not is_query_command() and not is_root() print_error("coral: must be run as root") return end if if args.count() < 2 print_usage() return end if let cmd = args.get(1) // Parse global options let opts = parse_global_options() // Dispatch to subcommand if cmd == "build" build.execute(opts) elif cmd == "install" install.execute(opts) elif cmd == "remove" remove.execute(opts) elif cmd == "upgrade" upgrade.execute(opts) elif cmd == "info" info.execute(opts) elif cmd == "list" list.execute(opts) elif cmd == "files" files.execute(opts) elif cmd == "owner" owner.execute(opts) elif cmd == "search" search.execute(opts) elif cmd == "depends" depends.execute(opts) elif cmd == "sync" sync.execute(opts) elif cmd == "diff" diff.execute(opts) elif cmd == "repo" repo.execute(opts) elif cmd == "key" key.execute(opts) elif cmd == "cache" cache.execute(opts) elif cmd == "--help" or cmd == "-h" print_usage() elif cmd == "--version" println("coral 1.0.0") else print_error("coral: unknown command '${cmd}'") print_usage() end if end main fn is_query_command(): bool let cmd = args.get(1) return cmd == "info" or cmd == "list" or cmd == "files" or cmd == "owner" or cmd == "search" or cmd == "depends" or cmd == "diff" or cmd == "--help" or cmd == "-h" or cmd == "--version" or cmd == "cache" end is_query_command fn is_root(): bool // Check if running as root (uid 0) let uid_str = env.get_env("EUID") if str.length(uid_str) == 0 // Fallback: try id command // For now assume root if EUID not set return true end if return uid_str == "0" end is_root ``` ### 2. Repository Management (commands/repo.reef) ```reef module commands.repo import sys.args import io.file import io.dir import io.path import encoding.toml import core.signing import util.color proc execute(opts: GlobalOptions) if args.count() < 3 print_repo_usage() return end if let subcmd = args.get(2) if subcmd == "init" repo_init(opts) elif subcmd == "add" repo_add(opts) elif subcmd == "remove" repo_remove(opts) elif subcmd == "list" repo_list(opts) elif subcmd == "rebuild" repo_rebuild(opts) elif subcmd == "sign" repo_sign(opts) elif subcmd == "verify" repo_verify(opts) elif subcmd == "sync" repo_sync(opts) elif subcmd == "export" repo_export(opts) else print_error("Unknown repo subcommand: ${subcmd}") print_repo_usage() end if end run proc repo_init(opts: GlobalOptions) if args.count() < 4 print_error("Usage: coral repo init ") return end if let repo_path = args.get(3) // Create directory structure dir.create_dir_all(path.join_path(repo_path, "packages")) // Create initial repo.toml let manifest = """ [repository] name = "unnamed" description = "New repository" maintainer = "" arch = "x86_64" generated = "" package_count = 0 base_url = "packages/" signing_key = "" """ file.writeFile(path.join_path(repo_path, "repo.toml"), manifest) print_success("Repository initialized at ${repo_path}") end repo_init proc repo_rebuild(opts: GlobalOptions) if args.count() < 4 print_error("Usage: coral repo rebuild ") return end if let repo_path = args.get(3) let pkg_dir = path.join_path(repo_path, "packages") // Scan all .pkg.tar.xz files let packages = dir.list_dir(pkg_dir) // Build manifest from package metadata // ... extract .PKGINFO from each package // ... generate repo.toml print_success("Repository manifest rebuilt") end repo_rebuild proc repo_sign(opts: GlobalOptions) if args.count() < 4 print_error("Usage: coral repo sign [--key ]") return end if let repo_path = args.get(3) let manifest_path = path.join_path(repo_path, "repo.toml") let key_name = args.get_flag_value("key") if str.length(key_name) == 0 key_name = "default" end if // Sign the manifest let sig = signing.sign_file(manifest_path, key_name) file.writeFile(manifest_path + ".sig", sig) print_success("Repository signed with key '${key_name}'") end repo_sign ``` ### 3. Key Management (commands/key.reef) ```reef module commands.key import sys.args import io.file import io.dir import io.path import core.signing import util.color proc execute(opts: GlobalOptions) if args.count() < 3 print_key_usage() return end if let subcmd = args.get(2) if subcmd == "init" key_init(opts) elif subcmd == "generate" key_generate(opts) elif subcmd == "list" key_list(opts) elif subcmd == "import" key_import(opts) elif subcmd == "export" key_export(opts) elif subcmd == "trust" key_trust(opts) elif subcmd == "revoke" key_revoke(opts) else print_error("Unknown key subcommand: ${subcmd}") print_key_usage() end if end run proc key_generate(opts: GlobalOptions) if args.count() < 4 print_error("Usage: coral key generate ") return end if let name = args.get(3) let keys_dir = "/etc/coral/keys" // Generate Ed25519 keypair let keypair = signing.generate_keypair() // Save private key (restricted permissions) let sec_path = path.join_path(keys_dir, name + ".sec") file.writeFile(sec_path, keypair.private_key) // TODO: chmod 600 on sec_path // Save public key let pub_path = path.join_path(keys_dir, name + ".pub") file.writeFile(pub_path, keypair.public_key) print_success("Generated keypair '${name}'") println(" Private key: ${sec_path}") println(" Public key: ${pub_path}") end key_generate proc key_list(opts: GlobalOptions) let keys_dir = "/etc/coral/keys" let files = dir.list_dir(keys_dir) println("Available signing keys:") println("") for f in files if str.ends_with(f, ".pub") let name = str.replace(f, ".pub", "") let has_private = file.fileExists(path.join_path(keys_dir, name + ".sec")) let key_type = "public" if has_private key_type = "public + private" end if println(" ${name} (${key_type})") end if end for end key_list ``` ### 4. Terminal Colors (util/color.reef) ```reef module util.color import sys.env import sys.args // ANSI color codes fn red(s: string): string if no_color() return s end if return "\x1b[31m" + s + "\x1b[0m" end red fn green(s: string): string if no_color() return s end if return "\x1b[32m" + s + "\x1b[0m" end green fn yellow(s: string): string if no_color() return s end if return "\x1b[33m" + s + "\x1b[0m" end yellow fn blue(s: string): string if no_color() return s end if return "\x1b[34m" + s + "\x1b[0m" end blue fn bold(s: string): string if no_color() return s end if return "\x1b[1m" + s + "\x1b[0m" end bold fn no_color(): bool // Check --no-color flag or NO_COLOR env var return args.has_flag("no-color") or env.has_env("NO_COLOR") end no_color // Formatted output helpers proc print_success(msg: string) println(green("✓") + " " + msg) end print_success proc print_error(msg: string) println(red("✗") + " " + msg) end print_error proc print_warning(msg: string) println(yellow("!") + " " + msg) end print_warning proc print_info(msg: string) println(blue("→") + " " + msg) end print_info ``` ### 5. User Prompts (util/prompt.reef) ```reef module util.prompt import sys.args import io.console import core.str fn confirm(message: string): bool // Skip if -y flag if args.has_flag("y") or args.has_flag("yes") return true end if print(message + " [Y/n] ") let response = console.readLine() let r = str.trim_ws(response) // Empty or Y/y means yes return str.length(r) == 0 or r == "Y" or r == "y" or r == "yes" end confirm fn confirm_packages(action: string, packages: [string], count: int): bool if count == 0 return false end if println("") println("Packages to ${action}:") for i in 0 to count println(" " + packages[i]) end for println("") return confirm("Proceed?") end confirm_packages fn ask_build_from_source(pkg_name: string): bool println("") print_warning("Binary package not available for '${pkg_name}'") return confirm("Build from source?") end ask_build_from_source ``` ### 6. HTTP Downloads (util/http.reef) ```reef module util.http import net.http import io.file import util.color import core.config // Download a file from URL to destination // Returns true on success fn download(url: string, dest: string): bool print_info("Downloading: ${url}") let cfg = config.load() // Try native Reef HTTP first let result = http.get(url, { timeout: cfg.network.timeout, retries: cfg.network.retries }) if result.status_code == 200 file.writeFile(dest, result.body) return true elif result.status_code >= 300 and result.status_code < 400 // Handle redirect let redirect_url = result.headers["Location"] if str.length(redirect_url) > 0 return download(redirect_url, dest) end if end if print_error("HTTP error: ${result.status_code}") return false end download // Download with progress callback fn download_with_progress(url: string, dest: string, callback: fn(int, int)): bool // Similar to download but calls callback(bytes_received, total_bytes) // Implementation depends on net.http capabilities return download(url, dest) // Fallback for now end download_with_progress // Note: If net.http fails on illumos, we'll report to Reef team // and can add a fallback to curl if absolutely necessary ``` ### 7. Build Command (commands/build.reef) ```reef module commands.build import sys.args import io.file import io.dir import io.path import core.port import core.package import util.color import util.http import util.checksum proc execute(opts: GlobalOptions) if args.count() < 3 print_error("Usage: coral build ") return end if let port_path = args.get(2) // Handle "." for current directory let actual_path = port_path if port_path == "." actual_path = dir.current_dir() end if // Load port let port_result = port.load(actual_path) if not port_result.success print_error("Failed to load port: ${port_result.error}") return end if let p = port_result.port println(bold("Building ${p.info.name}-${p.info.version}")) println("") // Step 1: Download sources print_info("Downloading sources...") if not download_sources(p) print_error("Failed to download sources") return end if // Step 2: Verify checksums print_info("Verifying checksums...") if not verify_sources(p) print_error("Checksum verification failed") return end if // Step 3: Extract sources print_info("Extracting sources...") if not extract_sources(p) print_error("Failed to extract sources") return end if // Step 4: Apply patches (if any) if p.patches.count > 0 print_info("Applying patches...") if not apply_patches(p) print_error("Failed to apply patches") return end if end if // Step 5: Run build print_info("Building...") if not run_build(p, opts.verbose) print_error("Build failed") return end if // Step 6: Create package print_info("Creating package...") let pkg_path = create_package(p) if str.length(pkg_path) == 0 print_error("Failed to create package") return end if println("") print_success("Package created: ${pkg_path}") end run fn download_sources(p: Port): bool for i in 0 to p.sources.count let url = p.sources.urls[i] let filename = path.basename(url) let dest = path.join_path(SOURCES_DIR, filename) if file.fileExists(dest) println(" " + green("✓") + " ${filename} (cached)") continue end if if not http.download(url, dest) return false end if end for return true end download_sources fn apply_patches(p: Port): bool let work_dir = path.join_path(WORK_DIR, p.info.name) let patches_dir = path.join_path(p.port_dir, "patches") let strip = p.patches.strip for i in 0 to p.patches.count let patch_file = p.patches.files[i] let patch_path = path.join_path(patches_dir, patch_file) println(" Applying: ${patch_file}") // patch -p -d < let cmd = "patch -p" + to_string(strip) + " -d " + work_dir + " < " + patch_path let pid = process.process_spawn_shell(cmd) let exit_code = process.process_wait(pid) if exit_code != 0 print_error("Patch failed: ${patch_file}") return false end if end for return true end apply_patches fn create_package(p: Port): string let pkg_name = p.info.name + "-" + p.info.version + "-" + to_string(p.info.release) + ".pkg.tar.xz" let pkg_path = path.join_path(PACKAGES_DIR, pkg_name) let pkg_staging = path.join_path(PKG_DIR, p.info.name) // Create .PKGINFO package.create_pkginfo(p, pkg_staging) // Create .FOOTPRINT package.create_footprint(pkg_staging) // Create tarball with xz compression let cmd = "cd " + pkg_staging + " && tar -Jcf " + pkg_path + " ." let pid = process.process_spawn_shell(cmd) let exit_code = process.process_wait(pid) if exit_code != 0 return "" end if return pkg_path end create_package ``` ### 8. Install Command (commands/install.reef) ```reef module commands.install import sys.args import core.resolver import core.database import core.port import core.repository import core.config import util.color import util.prompt proc execute(opts: GlobalOptions) if args.count() < 3 print_error("Usage: coral install ") return end if // Collect package names from args let packages: [string] = new [string](50) mut pkg_count = 0 for i in 2 to args.count() let arg = args.get(i) if not str.starts_with(arg, "-") packages[pkg_count] = arg pkg_count = pkg_count + 1 end if end for // Resolve dependencies for all packages let to_install: [string] = new [string](100) mut install_count = 0 for i in 0 to pkg_count install_count = resolver.resolve(packages[i], to_install, install_count) end for // Filter out already installed let needed: [string] = new [string](100) mut needed_count = 0 for i in 0 to install_count if not database.is_installed(to_install[i]) needed[needed_count] = to_install[i] needed_count = needed_count + 1 end if end for if needed_count == 0 print_info("All packages are already installed") return end if // Confirm with user if not prompt.confirm_packages("install", needed, needed_count) println("Aborted.") return end if let cfg = config.load() // Install each package for i in 0 to needed_count let pkg_name = needed[i] println("") println(bold("==> Installing ${pkg_name}")) // Try to get binary package from repository first let remote_pkg = repository.find_package(pkg_name) if str.length(remote_pkg.filename) > 0 // Binary package available print_info("Downloading ${pkg_name}...") let pkg_path = repository.download_package(remote_pkg) if str.length(pkg_path) > 0 print_info("Installing ${pkg_name}...") if install_package(pkg_path, pkg_name, opts) print_success("Installed ${pkg_name}") continue end if end if end if // Binary not available or download failed - try source let build_from_source = false if cfg.build.fallback_to_source == "yes" build_from_source = true elif cfg.build.fallback_to_source == "ask" build_from_source = prompt.ask_build_from_source(pkg_name) end if if not build_from_source print_error("Package not available: ${pkg_name}") return end if // Find port and build let p = port.find(pkg_name) if str.length(p.info.name) == 0 print_error("Port not found: ${pkg_name}") return end if print_info("Building ${pkg_name}...") let build_result = build_package(p, opts) if not build_result.success print_error("Build failed: ${build_result.error_message}") return end if print_info("Installing ${pkg_name}...") if not install_package(build_result.package_path, pkg_name, opts) print_error("Installation failed") return end if print_success("Installed ${pkg_name}") end for println("") print_success("Installation complete!") end run ``` --- ## Directory Constants ```reef // Zygaena/Coral paths const PORTS_DIR: string = "/usr/ports" const SOURCES_DIR: string = "/var/lib/coral/sources" const PACKAGES_DIR: string = "/var/lib/coral/packages" const REPOS_DIR: string = "/var/lib/coral/repos" const WORK_DIR: string = "/var/tmp/coral/work" const PKG_DIR: string = "/var/tmp/coral/pkg" const DB_DIR: string = "/var/lib/coral/installed" const KEYS_DIR: string = "/etc/coral/keys" const CONFIG_FILE: string = "/etc/coral/coral.conf" const REPOS_CONFIG_DIR: string = "/etc/coral/repos.d" ``` --- ## Package Format ### Port Structure ``` /usr/ports/// ├── package.toml # Package metadata (TOML) ├── build.reef # Build commands (Reef) ├── files/ # Additional files (optional) │ └── config.example ├── patches/ # Source patches (optional) │ └── fix-illumos.patch └── scripts/ # Install scripts (optional) ├── pre-install.reef ├── post-install.reef ├── pre-remove.reef └── post-remove.reef ``` ### package.toml Example ```toml [package] name = "vim" version = "9.1.0" release = 1 description = "Vi IMproved - enhanced vi editor" url = "https://www.vim.org" license = "Vim" maintainer = "[email protected]" # Alternative sets this package belongs to alternatives = ["vi", "editor"] # Packages this conflicts with (cannot be installed together) conflicts = ["vim-minimal"] [dependencies] runtime = ["ncurses"] build = ["ncurses-dev"] optional = [ "python:Python scripting support", "lua:Lua scripting support" ] [sources] urls = [ "https://github.com/vim/vim/archive/v${version}.tar.gz" ] checksums = [ "abc123def456..." ] [patches] # Patches are applied in order listed, after source extraction # Files are relative to the patches/ directory in the port files = [ "fix-illumos-termios.patch", "fix-ncurses-detection.patch" ] # Strip level for patch command (default: 1) strip = 1 [config] # Config files to protect during upgrades # These are installed as .pkg, user's version preserved as files = [ "/etc/vim/vimrc" ] [scripts] # Install scripts (Reef files in scripts/ directory) pre_install = "pre-install.reef" post_install = "post-install.reef" pre_remove = "pre-remove.reef" post_remove = "post-remove.reef" [build] parallel = true jobs = 0 ``` ### build.reef Example ```reef // Build script for vim fn configure(): string return "./configure --prefix=/usr --with-features=huge" end configure fn build(): string return "gmake -j${JOBS}" end build fn install(): string return "gmake install DESTDIR=${PKG}" end install ``` ### Package Groups (Meta-packages) Package groups are collections of packages that can be installed together as a bundle. Useful for large products like desktop environments or curated package sets. Groups are defined in `/usr/ports/groups/` as simple TOML files: ```toml # /usr/ports/groups/kde6.toml [group] name = "kde6" description = "KDE Plasma 6 Desktop Environment" [packages] # All packages installed when user runs: coral install @kde6 members = [ "kde-plasma6", "kde-kwin", "kde-extras", "kde-apps-base", "sddm" ] ``` Usage: ``` $ coral install @kde6 Installing group 'kde6' (5 packages): kde-plasma6, kde-kwin, kde-extras, kde-apps-base, sddm Proceed? [Y/n] ``` The `@` prefix indicates a group name rather than a package name. ### Alternatives (Dependency Options) Alternatives allow a dependency to be satisfied by one of several packages. Packages declare which alternative sets they belong to, and dependencies can require any member of that set. ```toml # In postfix/package.toml [package] name = "postfix" alternatives = ["mta"] # Member of the "mta" alternative set # In opensmtpd/package.toml [package] name = "opensmtpd" alternatives = ["mta"] # Also a member of "mta" # In mailman/package.toml - needs ANY mta [dependencies] runtime = ["mta:postfix,opensmtpd,sendmail"] # Pick one from this list ``` When installing a package with alternatives: ``` $ coral install mailman Resolving dependencies... Package 'mailman' requires 'mta' (one of: postfix, opensmtpd, sendmail) No provider for 'mta' is installed. Available options: 1) postfix 2) opensmtpd 3) sendmail Select [1-3]: 2 Packages to install: opensmtpd mailman Proceed? [Y/n] ``` If any member of the alternative set is already installed, the dependency is satisfied automatically. ### Conflict Detection Coral prevents file clobbering through two mechanisms: 1. **Explicit conflicts** - Declared in package.toml: ```toml conflicts = ["vim-minimal", "vim-tiny"] ``` 2. **File ownership tracking** - The installed package database tracks which package owns each file. During install: - If a file exists and is owned by another package → **error** (unless `--force`) - If a file exists and is not owned by any package → **warning** (orphan file) ### Config File Protection Config files listed in `[config].files` are protected during install/upgrade: | Scenario | Behavior | |----------|----------| | **Fresh install** | Package file installed as `/etc/foo.conf.pkg`, user copies to `/etc/foo.conf` | | **Upgrade, user hasn't modified** | Replace both `.conf` and `.conf.pkg` | | **Upgrade, user has modified** | Update `.conf.pkg` only, preserve user's `.conf` | | **Remove** | Remove `.conf.pkg`, warn if `.conf` differs | This ensures the user's customizations are never clobbered. The `.pkg` file always contains the package maintainer's version for reference. ### Install Scripts Install scripts are Reef programs executed at specific points: | Script | When | Use Cases | |--------|------|-----------| | `pre-install.reef` | Before files extracted | Create users/groups, stop services | | `post-install.reef` | After files installed | Run ldconfig, enable service, print notes | | `pre-remove.reef` | Before files removed | Stop services, backup data | | `post-remove.reef` | After files removed | Remove users, cleanup state | Example `post-install.reef`: ```reef // Post-install script for nginx proc main() // Run ldconfig to register shared libraries let pid = process.process_spawn_shell("/usr/sbin/ldconfig") process.process_wait(pid) // Print post-install notes println("") println("nginx installed successfully!") println("Enable with: svcadm enable nginx") println("Config file: /etc/nginx/nginx.conf.pkg") println("Copy to /etc/nginx/nginx.conf and customize.") end main ``` ### Binary Package (.pkg.tar.xz) ``` vim-9.1.0-1.pkg.tar.xz ├── .PKGINFO # Package metadata (TOML) ├── .FOOTPRINT # File listing with permissions ├── .SCRIPTS/ # Install scripts (if any) │ ├── pre-install.reef │ └── post-install.reef ├── usr/ │ ├── bin/vim │ ├── share/vim/ │ └── share/man/ ``` --- ## Implementation Schedule ### Phase 1: Core Infrastructure (Week 1-2) - [ ] Create directory structure - [ ] Set up reef.toml - [ ] Implement types.reef - [ ] Implement util/color.reef - [ ] Implement util/prompt.reef - [ ] Implement core/config.reef (configuration loading) - [ ] Implement main.reef (CLI dispatch) - [ ] Test net.http on illumos, report issues ### Phase 2: Build System (Week 3-4) - [ ] Implement core/port.reef (TOML parsing) - [ ] Implement util/http.reef (downloads) - [ ] Implement util/checksum.reef (SHA256) - [ ] Implement util/archive.reef (tar/xz) - [ ] Implement commands/build.reef - [ ] Test with hello port ### Phase 3: Package Management (Week 5-6) - [ ] Implement core/database.reef - [ ] Implement core/package.reef - [ ] Implement commands/install.reef - [ ] Implement commands/remove.reef - [ ] Implement commands/upgrade.reef - [ ] Implement commands/cache.reef ### Phase 4: Query Commands (Week 7) - [ ] Implement commands/info.reef - [ ] Implement commands/list.reef - [ ] Implement commands/files.reef - [ ] Implement commands/owner.reef - [ ] Implement commands/search.reef ### Phase 5: Repository System (Week 8-9) - [ ] Implement core/repository.reef (manifest parsing) - [ ] Implement core/signing.reef (Ed25519) - [ ] Implement commands/sync.reef (auto-detect git vs hg) - [ ] Implement commands/repo.reef (all subcommands) - [ ] Implement commands/key.reef (all subcommands) ### Phase 6: Advanced Features (Week 10) - [ ] Implement core/resolver.reef (dependency resolution) - [ ] Implement commands/depends.reef - [ ] Implement commands/diff.reef - [ ] Error handling improvements - [ ] Testing with multiple packages ### Phase 7: Polish & Documentation (Week 11-12) - [ ] Comprehensive testing - [ ] Man page generation - [ ] User documentation - [ ] Repository setup guide - [ ] Third-party repo guide --- ## Test Packages | Package | Category | Dependencies | Notes | |---------|----------|--------------|-------| | hello | core | None | GNU Hello, autotools | | tree | base | None | Single C file | | ncurses | devel | None | Library, needed by vim | | vim | base | ncurses | Tests dependency resolution | | jq | base | oniguruma | JSON processor | --- ## Success Criteria ### MVP (Minimum Viable Product) - [ ] `coral build` creates a package from a port - [ ] `coral install` installs packages with dependencies - [ ] `coral remove` removes packages - [ ] `coral list` shows installed packages - [ ] `coral search` finds ports - [ ] Colored output works - [ ] Confirmation prompts work ### Full Release - [ ] All subcommands functional (build, install, remove, upgrade, info, list, files, owner, search, depends, sync, diff, repo, key, cache) - [ ] Repository manifest system working - [ ] Package signing and verification working - [ ] Binary package download from mirrors working - [ ] Source fallback working with user prompts - [ ] 10+ packages buildable - [ ] Complex dependency chains work - [ ] Native HTTP working (or issues reported) - [ ] Clean error messages - [ ] Ready for Stage 2 (Core Ports) --- ## Distribution Infrastructure ### Official Repository Structure ``` https://pkg.zygaena.org/ ├── stable/ │ └── x86_64/ │ ├── repo.toml # Repository manifest │ ├── repo.toml.sig # Manifest signature │ └── packages/ │ ├── bash-5.2.21-1.pkg.tar.xz │ ├── bash-5.2.21-1.pkg.tar.xz.sig │ └── ... ├── testing/ │ └── x86_64/ │ └── ... └── keys/ ├── official-2026.pub # Current signing key └── official-2025.pub # Previous key (for verification) ``` ### Mirror Sync Process 1. **Build Server** (CI/CD): - Builds packages from ports - Signs packages with official key - Generates repo.toml manifest - Signs manifest - Uploads to master server 2. **Master Server**: - Authoritative package repository - Accepts uploads from build server only - Serves rsync to mirrors 3. **Mirror Servers**: - Sync from master via rsync - Serve packages via HTTP/HTTPS - Listed in official mirror list ### ISO/USB Distribution For offline installation or bootstrapping: ```bash # Export repository for ISO image coral repo export /srv/packages /mnt/iso/packages # Creates: # /mnt/iso/packages/ # ├── repo.toml # ├── repo.toml.sig # └── packages/ # └── *.pkg.tar.xz ``` --- ## Notes for Reef Team ### net.http Testing on illumos Will test the following and report issues: 1. Basic HTTP GET request 2. HTTPS (TLS) support 3. Redirect handling (301, 302) 4. Large file downloads 5. Connection timeout handling 6. Progress callbacks (if supported) ### Potential illumos-specific issues - Socket syscall constants - DNS resolution paths - TLS library linking ### crypto.ed25519 Requirements If not available in Reef stdlib, we will use the `signify` command-line tool. This is acceptable as signify is a standard tool on BSD/illumos systems. ### Shell-out Acceptability The following shell-outs are acceptable and expected: - `tar` / `xz` - Archive operations (standard system tools) - `patch` - Apply source patches with configurable strip level - `signify` - Ed25519 signing (until Reef adds native support) - `git` / `hg` - Ports collection sync (auto-detect VCS type) --- ## Revision History | Version | Date | Changes | |---------|------|---------| | 1.0 | January 2026 | Initial plan created | | 2.0 | January 2026 | Added configuration, repository system, signing, key management, cache management | | 2.1 | January 2026 | Clarified native vs shell-out; confirmed SHA256 is native via crypto.sha256; added Mercurial support for ports sync | | 2.2 | January 2026 | Added [patches] section to package.toml with ordered patch files and strip level | | 2.3 | January 2026 | Added conflict detection, config file protection (.pkg), and install scripts | | 2.4 | January 2026 | Clarified package groups (meta-packages) vs alternatives (dependency options) | | 2.5 | January 2026 | Added Reef language notes; Fixed syntax (run -> execute is reserved keyword); Updated module paths; Verified against Reef 0.1.3 spec | # Stage 2: Core Ports Implementation Plan **Document Version:** 1.0 **Created:** January 2026 **Status:** Planning **Prerequisites:** Stage 1 Complete (Coral package manager validated) --- ## Overview Stage 2 creates the foundational ports collection for Zygaena. These are real, working packages that can be built and installed using Coral on illumos. **Goal:** Build a self-hosting development environment where Zygaena can compile itself. --- ## Port Categories | Category | Purpose | Examples | |----------|---------|----------| | **core** | Essential system packages | libc, zlib, ncurses, libressl | | **base** | Standard Unix utilities | bash, vim, curl, git, tree | | **devel** | Development tools & libraries | gmake, pkgconf, autoconf, reef | --- ## Implementation Phases ### Phase 1: Zero-Dependency Ports Start with packages that have no external dependencies and are easy to build on illumos. | Port | Version | Description | Notes | |------|---------|-------------|-------| | `core/zlib` | 1.3.1 | Compression library | Foundation for many packages | | `base/tree` | 2.1.1 | Directory listing | Simple C, good test | | `base/less` | 643 | Pager | Simple, useful immediately | | `base/which` | 2.21 | Command locator | Single file | ### Phase 2: Core Libraries Libraries needed by most other packages. | Port | Version | Dependencies | Notes | |------|---------|--------------|-------| | `core/ncurses` | 6.4 | none | Terminal handling | | `core/readline` | 8.2 | ncurses | Line editing for bash/shells | | `core/libressl` | 3.9.2 | none | TLS/SSL (preferred over openssl) | | `core/libffi` | 3.4.6 | none | Foreign function interface | | `core/libedit` | 3.1 | ncurses | BSD line editor alternative | ### Phase 3: Base Utilities Essential command-line tools. | Port | Version | Dependencies | Notes | |------|---------|--------------|-------| | `base/bash` | 5.2.21 | ncurses, readline | Primary shell | | `base/vim` | 9.1 | ncurses | Editor | | `base/curl` | 8.6.0 | libressl, zlib | HTTP client | | `base/git` | 2.43.0 | curl, zlib, libressl | Version control | | `base/tmux` | 3.4 | ncurses, libevent | Terminal multiplexer | | `base/htop` | 3.3.0 | ncurses | Process viewer | ### Phase 4: Development Tools Tools needed to build software. | Port | Version | Dependencies | Notes | |------|---------|--------------|-------| | `devel/gmake` | 4.4.1 | none | GNU Make | | `devel/pkgconf` | 2.1.0 | none | Package config tool | | `devel/autoconf` | 2.72 | none | Configure script generator | | `devel/automake` | 1.16.5 | autoconf | Makefile generator | | `devel/libtool` | 2.4.7 | none | Library build tool | | `devel/cmake` | 3.28.1 | curl, zlib | Build system | ### Phase 5: Reef Language Package Reef itself so Zygaena can build Coral. | Port | Version | Dependencies | Notes | |------|---------|--------------|-------| | `devel/ocaml` | 5.1.1 | none | OCaml compiler (for building Reef) | | `devel/opam` | 2.1.5 | ocaml | OCaml package manager | | `devel/reef` | 0.1.4 | ocaml, opam | Reef language compiler | | `core/coral` | 1.0.0 | reef | Package manager itself | --- ## Port Structure Template Each port follows this structure: ``` /usr/ports/// ├── package.toml # Package metadata ├── build.sh # Build script (shell for now, reef later) ├── patches/ # illumos compatibility patches (if needed) │ └── *.patch └── files/ # Additional files (configs, services) ``` ### Example: core/zlib/package.toml ```toml [package] name = "zlib" version = "1.3.1" release = 1 description = "General-purpose compression library" url = "https://zlib.net" license = "Zlib" maintainer = "Chris Tusa " arch = "x86_64" [dependencies] runtime = [] build = [] [sources] urls = ["https://zlib.net/zlib-1.3.1.tar.gz"] checksums = ["9a93b2b7dfdac77ceba5a558a580e74667dd6fede4585b91eefb60f03b72df23"] [build] parallel = true jobs = 0 ``` ### Example: core/zlib/build.sh ```bash #!/bin/bash # Build script for zlib set -e # Environment provided by coral: # $SRCDIR - extracted source directory # $PKGDIR - installation destination (DESTDIR) # $JOBS - parallel job count cd "$SRCDIR" ./configure --prefix=/usr gmake -j$JOBS gmake install DESTDIR="$PKGDIR" ``` --- ## illumos Compatibility Notes ### Common Issues & Fixes | Issue | Solution | |-------|----------| | `make` vs `gmake` | Always use `gmake`, set `MAKE=gmake` | | `tar` vs `gtar` | Always use `gtar`, set `TAR=gtar` | | Missing `d_type` in dirent | Apply compatibility patch | | Different socket API | May need `-lsocket -lnsl` | | No `/proc/self/exe` | Use `getexecname()` instead | | Terminal ioctl differences | Patch with illumos-specific code | ### Build Environment Variables ```bash export PATH="/usr/gnu/bin:$PATH" export CC=gcc export CXX=g++ export MAKE=gmake export TAR=gtar export CFLAGS="-O2 -pipe -march=x86-64" export LDFLAGS="" ``` --- ## Dependency Graph (Simplified) ``` zlib | +-------+--------+--------+ | | | curl git cmake | | +---+---+ | libressl ncurses | +---+---+---+ | | | | bash vim tmux htop | readline ``` --- ## Success Criteria ### Minimum Viable Ports (MVP) - [ ] 5 working ports that build and install - [ ] Dependency chain works (e.g., ncurses -> vim) - [ ] Ports work on OmniOS test machine ### Full Stage 2 - [ ] 20+ ports covering core, base, devel categories - [ ] Reef can be built as a port - [ ] Coral can be built as a port - [ ] Self-hosting: can rebuild entire ports tree from itself - [ ] Binary repository with pre-built packages --- ## Implementation Order 1. **Set up ports tree structure** on illumos 2. **Create zlib port** - simplest, no deps, widely needed 3. **Create tree port** - simple C program, validate workflow 4. **Create ncurses port** - core library 5. **Create vim port** - depends on ncurses, user-facing 6. **Create libressl port** - TLS for curl/git 7. **Create curl port** - depends on libressl 8. **Create git port** - depends on curl 9. **Package Reef and Coral** - self-hosting milestone --- ## Verification Plan After each port: 1. `coral build ` - Verify it builds 2. `coral install ` - Verify it installs 3. Test the installed program works 4. `coral remove ` - Verify clean removal --- ## Notes - Start with shell build scripts (build.sh) for simplicity - Convert to Reef build scripts (build.reef) later if desired - Focus on getting packages working, not perfection - Document any illumos-specific patches needed - Keep patch files in the port for reproducibility # Coral RFE: chroot maintainer scripts under `--root` **Status:** Draft — ready to file with the Coral team **Filed against:** Coral 0.4.2 (`reef.toml` version = "0.4.2") **Reporter:** Chris Tusa **Consumer:** zyginstall (the Hammerhead system installer) — installs the OS into a target root via `coral install --root /mnt ...` from an ISO live environment. ## Summary When `coral install --root ` (and the remove/upgrade paths) run a package's maintainer scripts, the scripts execute against the **live host filesystem** with the real `/` as root. Only the file *payload* is written under ``; the scriptlets are not confined to it. There is no flag, config key, or env var to change this. For an OS installer that stages a target root and then installs packages into it, a package's `post-install.sh` that assumes it runs inside the installed system will instead act on the host (the ISO live environment) — silently corrupting the host or no-op'ing incorrectly. This makes package maintainer scripts unusable for `--root` installs. ## Current behavior (evidence) - File placement correctly honors `--root`: `install_files` writes every file to `path.join_path(root, rel_path)` (`src/core/package.reef:307`). ✅ - Script execution does **not**: the single executor `run_script` (`src/core/package.reef:508-536`) builds `zsh "/.SCRIPTS/.sh" ` and hands it to `process_spawn_shell` — no `chroot`, no `cd`, and `` is never passed in. The hook wrappers `run_pre_install` / `run_post_install` (`package.reef:539-565`) take only `(pkg_dir, version)`; `run_stored_script` (`package.reef:624-651`) takes `(scripts_dir, script_name, script_args)` and the stored hook wrappers (`run_stored_pre_remove` etc., `package.reef:654+`) take `(scripts_dir, version)` — the target root is architecturally invisible to all of them. - Call sites confirm the split: scripts run with `extract_dir`, files install with `install_root` (`src/commands/install.reef:443` / `:452` / `:461`). - No `chroot` anywhere in `src/` (only the standalone `tools/mkchroot/` helper). - No `$ROOT`/`$PKGROOT` env var is exported to scripts, so a script cannot even manually honor the root today (only `$VERSION`/`$OLD_VERSION`/`$NEW_VERSION` are passed positionally). ## Requested change 1. **Chroot maintainer scripts when the install root is non-`/`.** Thread the install root into the hook functions (`run_pre_install`, `run_post_install`, `run_pre_remove`, `run_post_remove`, `run_pre_upgrade`, `run_post_upgrade`, `run_stored_*` — all currently `(pkg_dir, version)`) and have `run_script` / `run_stored_script` emit `chroot "" zsh /.SCRIPTS/ `, with the `.SCRIPTS` dir staged inside `` so it's reachable post-chroot. 2. **An explicit control** — e.g. a `--chroot-scripts` / `--no-chroot-scripts` flag or a `chroot_scripts` config key — since none exists today. Define the fallback when `chroot(2)` isn't permitted (non-root caller): fail loudly rather than silently running against the host. 3. **Until (1) ships, a safety note** in the docs that `coral install --root ` stages *files* under `` but runs maintainer scripts against the host, and, as a stopgap, **export a `$CORAL_ROOT` env var** to scripts so a maintainer can defensively prefix paths. ## Impact on zyginstall (context for prioritization) Not an immediate hard blocker for the first two packages: `hammerhead-kernel` and `hammerhead-userland` have only one post-install script (a `svccfg import` guarded on `svc.configd` running) which no-ops on a Hammerhead/zyginit system. But it is a correctness landmine for any future package with a real post-install action, and the installer would otherwise have to special-case / manually re-run scripts under `chroot /mnt` itself. A stopgap `$CORAL_ROOT` env var (item 3) would let the installer proceed safely before the full chroot work lands. .\" Man page for coral(1) .\" Copyright (C) 2025 Leafscale, LLC .TH CORAL 1 "January 2026" "Coral 0.2.2" "Zygaena Package Manager" .SH NAME coral \- package manager for Zygaena .SH SYNOPSIS .B coral .RI [ options ] .I command .RI [ arguments ] .SH DESCRIPTION .B coral is a ports-based package manager for Zygaena (illumos). It can build packages from source ports, install binary packages from repositories, manage dependencies, and handle package signing and verification. .SH COMMANDS .SS "Package Operations" .TP .BI "build " port Build a package from a port. The port name should match a directory under /usr/ports//. The resulting package is stored in /var/lib/coral/packages/. .TP .BI "install " "package ..." Install one or more packages. Packages can be specified as: .RS .IP \(bu 2 Package name (downloaded from repository or built locally) .IP \(bu 2 Path to a .pkg.tar.xz file .IP \(bu 2 Group name prefixed with @ (e.g., @base-devel) .RE .TP .BI "remove " "package ..." Remove one or more installed packages. Runs pre-remove and post-remove scripts if present. .TP .BI "upgrade " "[package ...]" Upgrade installed packages to newer versions. If no packages are specified, upgrades all packages with available updates. .SS "Query Operations" .TP .BI "info " package Display detailed information about a package, including version, description, dependencies, and installation status. .TP .B list List all installed packages with their versions. .TP .BI "files " package List all files owned by an installed package. .TP .BI "owner " file Find which package owns a given file path. .TP .BI "search " term Search for packages matching the given term in ports and repositories. .TP .BI "depends " package Display the dependency tree for a package. .TP .B diff Show packages that have newer versions available. .SS "Repository Operations" .TP .B sync Synchronize the local ports tree with the remote repository (git pull). .TP .BI "repo " subcommand Manage package repositories. Subcommands: .RS .TP .B list List configured repositories .TP .B refresh Refresh repository metadata .TP .BI "add " "name url" Add a new repository .TP .BI "remove " name Remove a repository .TP .BI "init " path Initialize a new repository structure .TP .BI "rebuild " path Rebuild repository manifest from packages .TP .BI "sign " path Sign repository manifest .TP .BI "verify " path Verify repository signature .RE .SS "Key Management" .TP .BI "key " subcommand Manage signing keys. Subcommands: .RS .TP .B list List keys in the keyring .TP .BI "generate " name Generate a new Ed25519 signing key .TP .BI "import " "file name" Import a public key .TP .BI "export " name Export a public key .TP .BI "trust " name Add key to trusted keys .TP .BI "revoke " name Remove key from trusted keys .TP .B init Initialize the keyring directory .RE .SS "Cache Management" .TP .BI "cache " subcommand Manage package caches. Subcommands: .RS .TP .B clean Remove cached packages .TP .B clean-sources Remove cached source archives .TP .B clean-all Remove all cached data .TP .B stats Show cache statistics .RE .SS "Database Management" .TP .BI "db " subcommand Manage the package database indexes. The database provides fast O(log N) lookups for file ownership and package information. Subcommands: .RS .TP .B init Initialize the database indexes. Creates packages.db and files.db in /var/lib/coral/db/. Run this once during initial system setup. .TP .B rebuild Rebuild indexes from the installed package database. The source of truth is always /var/lib/coral/installed/; indexes can be safely rebuilt at any time. Use this if indexes become corrupted or out of sync. .TP .B status Display database status including number of indexed packages and files. .TP .B dump Display the contents of the files index (for debugging). .RE .SH OPTIONS .TP .BR \-h ", " \-\-help Display help message and exit. .TP .B \-\-version Display version information and exit. .TP .BR \-y ", " \-\-yes Assume yes to all prompts. Do not ask for confirmation. .TP .BR \-f ", " \-\-force Force operation. For install, reinstalls even if already installed. For remove, removes even with dependents. .TP .BR \-v ", " \-\-verbose Enable verbose output. Show additional details during operations. .TP .BR \-n ", " \-\-dry\-run Show what would be done without making changes. .TP .BR \-q ", " \-\-quiet Minimal output. Only show errors. .TP .B \-\-no\-color Disable colored output. .TP .BI \-\-root " path" Use an alternative root directory for installation. .SH FILES .TP .I /etc/coral/coral.conf Main configuration file. .TP .I /etc/coral/repos.d/ Repository configuration files (*.repo). .TP .I /etc/coral/keys/ Signing keys (private and public). .TP .I /usr/ports/ Ports tree containing package build definitions. .TP .I /usr/ports/groups/ Package group definitions (*.toml). .TP .I /var/lib/coral/installed/ Database of installed packages (source of truth). .TP .I /var/lib/coral/db/ Package database indexes for fast lookups. .TP .I /var/lib/coral/db/packages.db MsgPack index of installed package metadata. .TP .I /var/lib/coral/db/files.db MsgPack index mapping files to their owning packages (enables O(log N) lookups). .TP .I /var/lib/coral/packages/ Cached binary packages. .TP .I /var/lib/coral/sources/ Cached source archives. .TP .I /var/lib/coral/repos/ Cached repository manifests. .TP .I /var/lib/coral/trusted-keys/ Trusted public keys for verification. .TP .I /var/tmp/coral/work/ Build work directory. .TP .I /var/tmp/coral/pkg/ Package staging directory. .SH CONFIGURATION The configuration file /etc/coral/coral.conf uses TOML format: .nf [general] arch = "x86_64" jobs = 0 # 0 = auto-detect confirm = true color = true [paths] ports = "/usr/ports" packages = "/var/lib/coral/packages" [build] cflags = "-O2 -pipe" strip = true [security] require_signed_repos = false require_signed_packages = false .fi .SH EXAMPLES Build and install a package: .nf coral build vim coral install vim .fi Install multiple packages: .nf coral install vim nano htop .fi Install a package group: .nf coral install @base-devel .fi Search for packages: .nf coral search editor .fi Show package dependencies: .nf coral depends vim .fi Find which package owns a file: .nf coral owner /usr/bin/vim .fi Upgrade all packages: .nf coral upgrade .fi Generate a signing key: .nf coral key generate mykey .fi Sign a repository: .nf coral repo sign /path/to/repo .fi Initialize the package database: .nf coral db init .fi Check database status: .nf coral db status .fi Rebuild database indexes (if corrupted): .nf coral db rebuild .fi .SH EXIT STATUS .TP .B 0 Success. .TP .B 1 General error. .TP .B 2 Command line usage error. .TP .B 3 Package not found. .TP .B 4 Dependency resolution failed. .TP .B 5 Build failed. .TP .B 6 Permission denied (not root). .SH SEE ALSO .BR coral.conf (5), .BR coral-build (7), .BR coral-repo (7) .SH BUGS Report bugs at https://github.com/leafscale/zygaena/issues .SH AUTHORS Chris Tusa .br Leafscale, LLC \- https://www.leafscale.com .SH COPYRIGHT Copyright (C) 2025 Leafscale, LLC. Licensed under BSD-2-Clause. commit,short,date,author,email,subject b03ff6660806c4a313e230d4d3a8cd0b3ead96ab,b03ff66,2026-01-25T18:21:56-06:00,Chris Tusa,chris.tusa@leafscale.com,Initial commit: Coral package manager for Zygaena b6fcf6c0913b31534a91ee1b8f2ca1f1ad8391e6,b6fcf6c,2026-01-25T18:26:46-06:00,Chris Tusa,chris.tusa@leafscale.com,Add release script a9e5b80d4419658fe77405ad983b23bbb6e38d2e,a9e5b80,2026-01-25T18:31:58-06:00,Chris Tusa,chris.tusa@leafscale.com,updated gitignore to exclude releases/ binary 4b81063d9eb0b7bdc637d1b4f9278ea9c6fa7c78,4b81063,2026-01-26T22:54:49-06:00,Chris Tusa,chris.tusa@leafscale.com,Update http.reef for Reef 0.1.8 native downloads cc6f827268cc6648983d5cd764c73d4caa162376,cc6f827,2026-01-26T17:36:52-06:00,Chris Tusa,chris.tusa@leafscale.com,Fix http.reef for Reef 0.1.7 compatibility ce380ed668d8c0576c8a616d1d177e3e28047ba9,ce380ed,2026-01-26T17:41:36-06:00,Chris Tusa,chris.tusa@leafscale.com,Add user confirmation prompts and improve command workflows 3acc88aa18929868459745b0415f49e44fe51827,3acc88a,2026-01-26T18:12:02-06:00,Chris Tusa,chris.tusa@leafscale.com,Add --root and --prefix support for alternate root installation 22468ffc544902769dc01e151c47c97b4013cf89,22468ff,2026-01-26T18:17:04-06:00,Chris Tusa,chris.tusa@leafscale.com,Add rooted resolver for alternate root installation 199584532c5901135ddfef58d022396b11696766,1995845,2026-01-26T18:29:13-06:00,Chris Tusa,chris.tusa@leafscale.com,Add curl fallback for HTTP downloads on illumos 88e57e2a80b58a2036449006e8b80bafac2c78db,88e57e2,2026-01-26T19:54:26-06:00,Chris Tusa,chris.tusa@leafscale.com,Add Woodpecker CI pipeline for build and release 293b7610911fdbceba68e9bc53aa345e360018b8,293b761,2026-01-26T19:55:29-06:00,Chris Tusa,chris.tusa@leafscale.com,Fix Woodpecker CI pipeline for v3.12.0 compatibility 1bdcbc314a21d54b29f2e064c2ba3daaa491ffdf,1bdcbc3,2026-01-26T19:56:57-06:00,Chris Tusa,chris.tusa@leafscale.com,Add authentication to Reef compiler download 0b632431dd01bd70dcf4e47e202ca9021df48bf0,0b63243,2026-01-26T20:19:57-06:00,Chris Tusa,chris.tusa@leafscale.com,Debug secrets and try alternative environment syntax b460a2d41f4dd71cdccb001b7e1581460f49393c,b460a2d,2026-01-26T20:22:02-06:00,Chris Tusa,chris.tusa@leafscale.com,Fix YAML syntax for environment secrets 4bda1d38b31dae23cbc9022c629aac5d2766328c,4bda1d3,2026-01-26T20:25:05-06:00,Chris Tusa,chris.tusa@leafscale.com,Try Woodpecker built-in CI_FORGE_TOKEN e06e651b2400207ba828cf7a13c5d170d98a3630,e06e651,2026-01-26T20:28:47-06:00,Chris Tusa,chris.tusa@leafscale.com,Add FORGEJO_TOKEN back to build step ffd38319b6f10cc9a4959b30080652728dcebf01,ffd3831,2026-01-26T20:32:59-06:00,Chris Tusa,chris.tusa@leafscale.com,Trigger CI to test trusted security setting 5c069996e2b7375610215727baf70e22840329c9,5c06999,2026-01-26T20:34:38-06:00,Chris Tusa,chris.tusa@leafscale.com,Try secrets array syntax with source/target 77b6a6d2a0bc21d508d81c2f0ab4b3faf7ac896a,77b6a6d,2026-01-26T20:35:41-06:00,Chris Tusa,chris.tusa@leafscale.com,Revert to environment/from_secret syntax 6678097900ac1cde1b4e7bf73833c84c697ae1fe,6678097,2026-01-26T20:38:34-06:00,Chris Tusa,chris.tusa@leafscale.com,Add API debug to test reef-lang access d049473f63e4736dbc4190118f1d39fcf9c2d016,d049473,2026-01-26T20:43:27-06:00,Chris Tusa,chris.tusa@leafscale.com,"Add deeper token debug - show first 4 chars, write to file" 279134f09b1a989ba75b2bd81a79d95a6b6071a8,279134f,2026-01-26T20:52:17-06:00,Chris Tusa,chris.tusa@leafscale.com,Update secret name to coral-releases 72f3fd68f31ebdf11b8349b08eefd7089d5be0e3,72f3fd6,2026-01-26T20:53:25-06:00,Chris Tusa,chris.tusa@leafscale.com,Add test_secret to debug Woodpecker secrets d971e66f91c40c7631df907b5910f1192bac8bcc,d971e66,2026-01-26T20:54:54-06:00,Chris Tusa,chris.tusa@leafscale.com,TEMPORARY: hardcode token to test pipeline 82460c8577f9b0092a0de5931306893e2f259ae5,82460c8,2026-01-26T20:58:40-06:00,Chris Tusa,chris.tusa@leafscale.com,Fresh Woodpecker pipeline from scratch 7471b00b4f192dc3adb258c43d0c895afd805297,7471b00,2026-01-26T21:02:49-06:00,Chris Tusa,chris.tusa@leafscale.com,Fix YAML quoting for curl commands 3a97f201ceca288d5044fa197883a017176cfdeb,3a97f20,2026-01-26T21:03:30-06:00,Chris Tusa,chris.tusa@leafscale.com,Use curly braces for variable expansion 15fe57988156ad967a2475c418661039e769b1ac,15fe579,2026-01-26T21:06:39-06:00,Chris Tusa,chris.tusa@leafscale.com,Try API_KEY as variable name c9cb48facf63d97ddf4529bd93caf1db79ce6d49,c9cb48f,2026-01-26T21:08:30-06:00,Chris Tusa,chris.tusa@leafscale.com,"Revert to reef_token secret, clean up pipeline" aaa7a4f57c971ef4103996e85669db8c8a354e8e,aaa7a4f,2026-01-26T21:09:23-06:00,Chris Tusa,chris.tusa@leafscale.com,Add test_secret to debug secret injection 3a1cbd831fdbf3b695d93d661b8c61a8a49c5e9b,3a1cbd8,2026-01-26T21:27:00-06:00,Chris Tusa,chris.tusa@leafscale.com,Fix variable escaping for Woodpecker preprocessing 6e928a411ca49868f940fd307f068b6d17d75bf5,6e928a4,2026-01-26T21:27:44-06:00,Chris Tusa,chris.tusa@leafscale.com,Quote curl command to fix YAML parsing 2486d042722a126716c75dff08eb6b79a957f0ad,2486d04,2026-01-26T21:28:44-06:00,Chris Tusa,chris.tusa@leafscale.com,Debug: show tarball structure 1c191e93d4984e8bfc99c0ba30d3a6a7f3e565d2,1c191e9,2026-01-26T21:30:02-06:00,Chris Tusa,chris.tusa@leafscale.com,Fix reefc path and clean up debug statements 6bf7480082b3938723293cdcc958bdef16769cfd,6bf7480,2026-01-26T21:31:13-06:00,Chris Tusa,chris.tusa@leafscale.com,Switch to Ubuntu 24.04 for GLIBC 2.38+ compatibility f2ef479898aab95e3a478f4eb17e6e2c3ab4d654,f2ef479,2026-01-26T21:32:20-06:00,Chris Tusa,chris.tusa@leafscale.com,Set REEF_STDLIB environment variable for reefc ffbb7793c9819eea7efd4fbd3aafb85a8abfb210,ffbb779,2026-01-26T21:33:39-06:00,Chris Tusa,chris.tusa@leafscale.com,Use REEF_HOME instead of REEF_STDLIB c646c2f45e235c2713daac859a509faa244d4fcc,c646c2f,2026-01-26T21:34:25-06:00,Chris Tusa,chris.tusa@leafscale.com,Set both REEF_HOME and REEF_STDLIB 618ab8c7ed35cd0fe7d6ebab76f5829712ab4a38,618ab8c,2026-01-26T21:35:36-06:00,Chris Tusa,chris.tusa@leafscale.com,"Debug: extract without strip, show structure" 3a465aa5f2690e5dfdc02ed55e19f199a52952aa,3a465aa,2026-01-26T21:37:48-06:00,Chris Tusa,chris.tusa@leafscale.com,Set both REEF_HOME and REEF_STDLIB for reefc 537795eff98516a07b184c9d3f0d21a08f3115e0,537795e,2026-01-26T21:39:03-06:00,Chris Tusa,chris.tusa@leafscale.com,Debug: check stdlib contents in binary distribution 0bf8a2060e3b1af4f59433623dd3291add69aaab,0bf8a20,2026-01-26T21:40:04-06:00,Chris Tusa,chris.tusa@leafscale.com,Workaround: symlink stdlib to reef-stdlib (Reef packaging bug) 79d691addc6e79d4364bd8d6b289593e516d8a93,79d691a,2026-01-26T21:41:02-06:00,Chris Tusa,chris.tusa@leafscale.com,"Set REEF_HOME in environment block, add reef-runtime symlink and gcc" 5b95ae40fa223e8b196aa20b589dc7deeee2d9a2,5b95ae4,2026-01-26T22:48:54-06:00,Chris Tusa,chris.tusa@leafscale.com,Simplify Reef setup - just add bin to PATH 52d81e297fdf5f522faaa533f56523f6fee51a18,52d81e2,2026-01-26T22:49:56-06:00,Chris Tusa,chris.tusa@leafscale.com,Install build-essential for C headers 22599cf95715bb9b59b44d158ed3535ca341fe74,22599cf,2026-01-26T22:54:13-06:00,Chris Tusa,chris.tusa@leafscale.com,Bump version to 0.1.2 017c4012259061c3d101faf25007ba3a7837cb19,017c401,2026-01-26T23:04:38-06:00,Chris Tusa,chris.tusa@leafscale.com,Add gnu and illumos categories to port search 0318423949888750d43d046f420c1b4a284ac13f,0318423,2026-01-27T10:51:34-06:00,Chris Tusa,chris.tusa@leafscale.com,Add build cleanup options and implementation f3b3e306109d1528bc52d1499abcc45f2248654c,f3b3e30,2026-01-27T11:03:36-06:00,Chris Tusa,chris.tusa@leafscale.com,Fix dependency bug 457b3cb798fbb197bae46b57be4608227c447d70,457b3cb,2026-01-27T11:30:35-06:00,Chris Tusa,chris.tusa@leafscale.com,Add source mirrors refactor plan (Option B) 1af9101522207a7d3e431bc128a2327e9bb75d9e,1af9101,2026-01-27T12:14:54-06:00,Chris Tusa,chris.tusa@leafscale.com,Implement source mirrors support (Option B) 2d49d527619b5aeccb01cec1fb59b23b9d14346d,2d49d52,2026-01-27T13:15:05-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.1.3: Documentation updates and source mirrors 295065f656a9d12f975cfd721bb1bbd76be37996,295065f,2026-01-27T15:45:15-06:00,Chris Tusa,chris.tusa@leafscale.com,Update Reef compiler to 0.1.10 ba09e0184c81971064c76ff9ae19069c2ea01f4f,ba09e01,2026-01-27T15:48:45-06:00,Chris Tusa,chris.tusa@leafscale.com,Handle existing releases in CI pipeline 2f44050a6abaa9493f2b862a9cb7f55b9c1b3b53,2f44050,2026-01-27T17:28:16-06:00,Chris Tusa,chris.tusa@leafscale.com,Add local source support and empty package validation e062adc983f4c2616dd30d29365b522ba55bf4e5,e062adc,2026-01-27T17:41:23-06:00,Chris Tusa,chris.tusa@leafscale.com,"Release v0.1.4: Local sources, empty package validation" de2a6793c6d2c1df8054db0237ce5b0853002e94,de2a679,2026-01-27T22:00:43-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.1.5: Build logging and output ordering fix 39a6f92bca85838fe24528be9e02d2137708e817,39a6f92,2026-01-28T00:27:36-06:00,Chris Tusa,chris.tusa@leafscale.com,Add bug report for HTTP redirect and version selection issues e08240846c675b3904d67211370f7a36aa5e34c9,e082408,2026-01-28T11:33:02-06:00,Chris Tusa,chris.tusa@leafscale.com,Add bug report for source extraction hang c2a68d0ca690312b7eb1b7681f6c641dd8c227af,c2a68d0,2026-01-28T14:05:47-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.1.7: Bug fixes and new features 998a2cc975a783420807c20bd0ef5f07e6cb0cdd,998a2cc,2026-01-28T15:38:39-06:00,Chris Tusa,chris.tusa@leafscale.com,Update extraction bug: XZ files still broken in 0.1.7 e4efc01cb74e3e519596d36bb4fb276359f41b9e,e4efc01,2026-01-28T16:28:26-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.1.9: Fix XZ extraction hang with Reef 0.1.14 d418f8e8da7482cecf355ac30dea445b9dc8f5ba,d418f8e,2026-01-28T17:41:59-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.1.10: illumos compatibility and file tracking fixes 11a24e2ba15cb76159278e4fd8a174e222f21550,11a24e2,2026-01-29T20:22:08-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.2.1: Add --root support for alternate filesystem operations 6456fce16d606342244f913a1551f148995eeec8,6456fce,2026-01-29T20:23:20-06:00,Chris Tusa,chris.tusa@leafscale.com,Remove resolved bug reports and deprecated cache command aac35d47153677ba767e1811fc765b9f2a584721,aac35d4,2026-01-29T20:34:49-06:00,Chris Tusa,chris.tusa@leafscale.com,Move chroot scripts to tools/ and add coral binary to gitignore 12dcf11386c26692093183acbcc4408750d60541,12dcf11,2026-01-29T20:38:25-06:00,Chris Tusa,chris.tusa@leafscale.com,mkchroot: Generate scripts alongside chroot directory 9478e6215aa6435f0209f6222dfd4bcc311aaebd,9478e62,2026-01-29T21:01:07-06:00,Chris Tusa,chris.tusa@leafscale.com,Fix --root/--prefix flag parsing to work in any position 2e8a15f2a8c502adf6f009ea5a334cb210498a35,2e8a15f,2026-01-29T21:48:55-06:00,Chris Tusa,chris.tusa@leafscale.com,Implement MsgPack-based package database indexes a668688fbf217971cd9499bd44f80615a655a552,a668688,2026-01-29T22:20:13-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.2.2: Fix MsgPack parsing and add database documentation 8d1f81dfd4649ae7cab7090d3e7671d07ce21594,8d1f81d,2026-01-30T09:19:28-06:00,Chris Tusa,chris.tusa@leafscale.com,updated woodpecker to use reef-lang-0.1.15 9f1c0251f55dbc9bb3b1638fe45c3152d7db6e4e,9f1c025,2026-01-30T10:21:47-06:00,Chris Tusa,chris.tusa@leafscale.com,updated gitignore 826d764a8328b043a12c4f20ddf03b3c0e98bc7e,826d764,2026-01-30T17:31:25-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.2.3: Add download progress bar and allow non-root builds 623ec00c0079403ac97a77e244c7f0bdcb8b6257,623ec00,2026-01-30T18:43:31-06:00,Chris Tusa,chris.tusa@leafscale.com,Update woodpecker CI to use reef-lang-0.1.16 606628b29f8081499a217f3cda105514ecf56261,606628b,2026-02-19T20:38:07-06:00,Chris Tusa,chris.tusa@leafscale.com,Add hammerhead category to port search list 2c66115aab82b148e7c3839b4c075f2bfc7c2521,2c66115,2026-02-19T20:40:02-06:00,Chris Tusa,chris.tusa@leafscale.com,Support category/name format in port find b416c5a9e7f826f2e17b93f773a5ea8ce6691ec3,b416c5a,2026-02-19T22:45:31-06:00,Chris Tusa,chris.tusa@leafscale.com,"Release v0.2.4: Fix resolver bug, dynamic categories, ports index, ports command" 75d402d5455b0427e99d5b55b5f3962fc7e9f2e6,75d402d,2026-02-19T22:50:33-06:00,Chris Tusa,chris.tusa@leafscale.com,Update documentation for v0.2.4 features 3ae110affb94f89ea99887141ecdf87e5a22441b,3ae110a,2026-02-19T23:04:47-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.2.5: Fix files.db sort hang on large packages fae09e1490c6bb57a5b36fbfb6107064790347f1,fae09e1,2026-02-19T23:17:55-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.2.6: Fix files.db rebuild hang from massive over-allocation 38023ac9014277fccab175462941dce1a688f0e0,38023ac,2026-02-19T23:33:19-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.2.7: Fix install hang on large packages (161K+ files) 6d0400d05640a3b2e597718e94d11ac0b435a954,6d0400d,2026-02-19T23:47:07-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.2.8: Fix register hang by replacing string building with shell pipeline 373f7f46b378f7cdb32e933a120233069e913ca0,373f7f4,2026-02-19T23:56:38-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.2.9: Replace shell workaround with str.join() for footprint processing 601da8e5ef047189eec94f5b620a0c9b6cb49020,601da8e,2026-03-02T23:16:32-06:00,Chris Tusa,chris.tusa@leafscale.com,Fix example config and default arch to match Zygaena defaults 8c6d1a8dafa3b346e92bb9efac7a669c285012fb,8c6d1a8,2026-03-03T11:55:01-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.2.10 c0d1b543bcbea7f0791cecd0c017a07023c919df,c0d1b54,2026-03-03T12:12:18-06:00,Chris Tusa,chris.tusa@leafscale.com,Update CI to use reef-lang v0.3.4 f220c00e69ffcba8b5d1f89865504d32dd6d52d0,f220c00,2026-03-03T19:45:55-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.2.11: Fix build script shell invocation and CPU detection 66624612dfb38f4b6f46644ad3486fafa363b247,6662461,2026-03-07T19:47:23-06:00,Chris Tusa,chris.tusa@leafscale.com,Add mtree manifest module and replace .FOOTPRINT with .MANIFEST in build 5289616d94fb3a9b2d72b984be505528b572b1ba,5289616,2026-03-07T19:58:29-06:00,Chris Tusa,chris.tusa@leafscale.com,Replace rsync with manifest-driven fs.ops installer 373d4996b402e89933b1520032a4e954b225033c,373d499,2026-03-07T20:02:14-06:00,Chris Tusa,chris.tusa@leafscale.com,Add [dependencies] to .PKGINFO and update TODO status 261fd8e618ec70bdd848d2ac69e70742d0969452,261fd8e,2026-03-07T20:06:41-06:00,Chris Tusa,chris.tusa@leafscale.com,Add config file protection on package removal d7c9548c735d16ec08632cb05b648d4d8fb54190,d7c9548,2026-03-07T20:40:41-06:00,Chris Tusa,chris.tusa@leafscale.com,"Rewrite script model: remove .reef type, use zsh, add upgrade flow" 78422089d8bf212b3dec6d5a63654d6153e0618d,7842208,2026-03-07T21:06:29-06:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.3.0: Package Format v2 complete 3f74bcde5e57e13a31ca28c94f064978483e6cb2,3f74bcd,2026-03-07T22:15:22-06:00,Chris Tusa,chris.tusa@leafscale.com,Auto-install missing build dependencies before compilation cc4f8e31c68efe5092bae574414ed63f19e84871,cc4f8e3,2026-03-07T23:10:59-06:00,Chris Tusa,chris.tusa@leafscale.com,"Export $PREFIX, $SYSCONFDIR, $PKG_ROOT to build scripts" 02417f9862b4236f7ac76b54181c195328d205c8,02417f9,2026-03-08T00:14:11-06:00,Chris Tusa,chris.tusa@leafscale.com,Recursive build-from-source dependency resolver fbe0637b2dc158e1afc77202d5bd24687a2152f3,fbe0637,2026-03-08T12:09:49-05:00,Chris Tusa,chris.tusa@leafscale.com,Release v0.3.5: Wire up all unwired features and cleanup b88a15a3a11cdc6401038fe3d663e2fd1c68cf5c,b88a15a,2026-03-08T15:37:57-05:00,Chris Tusa,chris.tusa@leafscale.com,Fix infinite CPU loop in packaging step 604db08ff4f7276ebf76b4095640f76a67c05ffa,604db08,2026-03-08T20:05:35-05:00,Chris Tusa,chris.tusa@leafscale.com,Fix O(n²) manifest generation using StringBuilder (BUG-027 workaround) 2f040691a486473343c7b2ef8a1eab5bf31032f6,2f04069,2026-03-08T23:38:23-05:00,Chris Tusa,chris.tusa@leafscale.com,Cache config paths to reduce TOML parsing allocation pressure c388de7ff8f4f92a33896b965fac57541cb593d7,c388de7,2026-03-08T23:40:56-05:00,Chris Tusa,chris.tusa@leafscale.com,Update CI to use Reef 0.5.2 242ff329a523c8547db03a486611a774c589c022,242ff32,2026-03-09T16:12:57-05:00,Chris Tusa,chris.tusa@leafscale.com,"Release v0.3.9: Fix GC-related crashes, use self for install subprocess" 2c7e72fcd76d8974b724825140a7fe85e769af10,2c7e72f,2026-03-09T16:44:19-05:00,Chris Tusa,chris.tusa@leafscale.com,"Release v0.3.10: Fix install from file path, fix conflicts array crash" 4723f14f9809bc746a0c110284a7e91ac2a885a4,4723f14,2026-03-09T20:33:14-05:00,Chris Tusa,chris.tusa@leafscale.com,"Release v0.3.11: Fix reinstall/force install, add version subcommand" 65244f3fa976aa9ee3614cf01e0a1a1ccc8b3668,65244f3,2026-03-10T16:34:48-05:00,Chris Tusa,chris.tusa@leafscale.com,"Release v0.3.12: Install package after build, update CI to Reef 0.5.5" 7d9ff3efc23b38f310cd53b24e5a217abcc70ed4,7d9ff3e,2026-03-10T22:24:23-05:00,Chris Tusa,chris.tusa@leafscale.com,"Release v0.3.13: Fix redirect crash, illumos build compat, zero-alloc uid/gid" 507ca32e163e73594cc22535c8b72dac6bdee890,507ca32,2026-03-11T14:49:42-05:00,Chris Tusa,chris.tusa@leafscale.com,"Release v0.4.0: Align build system with Hammerhead porting guide, default prefix /usr/local" c5956c1b204ba2a39370ec9431f208c760f7d602,c5956c1,2026-03-11T20:52:10-05:00,Chris Tusa,chris.tusa@leafscale.com,"docs: add patches section, BUILD_TRIPLE, update prefix to /usr/local" dcc8974f3fb937f7a5f1d1b0c9f05b44e4ddd652,dcc8974,2026-03-11T22:29:39-05:00,Chris Tusa,chris.tusa@leafscale.com,Fix package removal failing to unregister from database 4e82c0f0cccfa2cecfebf54071392f418542775c,4e82c0f,2026-03-12T15:34:07-05:00,Chris Tusa,chris.tusa@leafscale.com,Add LDFLAGS and PKG_CONFIG_PATH to build environment 4ff188f470ff970c73ba8bdc422425e2d72213ec,4ff188f,2026-03-12T15:35:54-05:00,Chris Tusa,chris.tusa@leafscale.com,Simplify PKG_CONFIG_PATH (avoid ${} in string literal - reefc bug) 9b1904eb8b0ecb1d5e0c10feda09d36f2db98038,9b1904e,2026-03-12T15:39:49-05:00,Chris Tusa,chris.tusa@leafscale.com,"LDFLAGS: only set -R (RPATH), drop -L to avoid shadowing base libs" 1f1b687238b9b16d7819d7916f295a79ba8c0377,1f1b687,2026-03-12T15:40:59-05:00,Chris Tusa,chris.tusa@leafscale.com,"docs: explain LDFLAGS, PKG_CONFIG_PATH, and library resolution design" ec67d5dfb3effec2ea7ff826fc8537a62314e5e1,ec67d5d,2026-04-08T18:56:43-05:00,Chris Tusa,chris.tusa@leafscale.com,fix(build): add --install/--no-install flag parsing (default: no-install) aefab7046c5022e9647aa94158c7940c9a6213cd,aefab70,2026-04-08T18:57:34-05:00,Chris Tusa,chris.tusa@leafscale.com,fix(build): gate target install behind --install flag (fixes bug 007) 4ace9941b38dff3ecc442b19717df5a54ba2d7ee,4ace994,2026-04-08T18:58:22-05:00,Chris Tusa,chris.tusa@leafscale.com,docs: update build command usage for --install/--no-install flags b85d2815eb145523dfcaf26b131f396d6b418d03,b85d281,2026-04-08T19:29:22-05:00,Chris Tusa,chris.tusa@leafscale.com,bump version to 0.4.1 9ffd39abc2c212b57454651a84172a9a42809c67,9ffd39a,2026-05-04T20:15:04-05:00,Chris Tusal,chris.tusa@leafscale.com,fix: compile against Reef 0.5.20 strict type checking (fixes bug 009) c528299ec33440d40f1ee76448cdd2c9271c8469,c528299,2026-05-04T20:15:07-05:00,Chris Tusal,chris.tusa@leafscale.com,bump version to 0.4.2 a578ffb3457529c059f204edd65cb7364a259548,a578ffb,2026-05-04T20:24:57-05:00,Chris Tusal,chris.tusa@leafscale.com,ci: pin Reef toolchain to 0.5.20 # Coral Git History Historical record of the Coral git repository prior to the Mercurial + Isurus migration. Captured at migration time. Total commits: **120**. | Date | Commit | Author | Subject | |------|--------|--------|---------| | 2026-05-04T20:24:57-05:00 | `a578ffb` | Chris Tusal | ci: pin Reef toolchain to 0.5.20 | | 2026-05-04T20:15:07-05:00 | `c528299` | Chris Tusal | bump version to 0.4.2 | | 2026-05-04T20:15:04-05:00 | `9ffd39a` | Chris Tusal | fix: compile against Reef 0.5.20 strict type checking (fixes bug 009) | | 2026-04-08T19:29:22-05:00 | `b85d281` | Chris Tusa | bump version to 0.4.1 | | 2026-04-08T18:58:22-05:00 | `4ace994` | Chris Tusa | docs: update build command usage for --install/--no-install flags | | 2026-04-08T18:57:34-05:00 | `aefab70` | Chris Tusa | fix(build): gate target install behind --install flag (fixes bug 007) | | 2026-04-08T18:56:43-05:00 | `ec67d5d` | Chris Tusa | fix(build): add --install/--no-install flag parsing (default: no-install) | | 2026-03-12T15:40:59-05:00 | `1f1b687` | Chris Tusa | docs: explain LDFLAGS, PKG_CONFIG_PATH, and library resolution design | | 2026-03-12T15:39:49-05:00 | `9b1904e` | Chris Tusa | LDFLAGS: only set -R (RPATH), drop -L to avoid shadowing base libs | | 2026-03-12T15:35:54-05:00 | `4ff188f` | Chris Tusa | Simplify PKG_CONFIG_PATH (avoid ${} in string literal - reefc bug) | | 2026-03-12T15:34:07-05:00 | `4e82c0f` | Chris Tusa | Add LDFLAGS and PKG_CONFIG_PATH to build environment | | 2026-03-11T22:29:39-05:00 | `dcc8974` | Chris Tusa | Fix package removal failing to unregister from database | | 2026-03-11T20:52:10-05:00 | `c5956c1` | Chris Tusa | docs: add patches section, BUILD_TRIPLE, update prefix to /usr/local | | 2026-03-11T14:49:42-05:00 | `507ca32` | Chris Tusa | Release v0.4.0: Align build system with Hammerhead porting guide, default prefix /usr/local | | 2026-03-10T22:24:23-05:00 | `7d9ff3e` | Chris Tusa | Release v0.3.13: Fix redirect crash, illumos build compat, zero-alloc uid/gid | | 2026-03-10T16:34:48-05:00 | `65244f3` | Chris Tusa | Release v0.3.12: Install package after build, update CI to Reef 0.5.5 | | 2026-03-09T20:33:14-05:00 | `4723f14` | Chris Tusa | Release v0.3.11: Fix reinstall/force install, add version subcommand | | 2026-03-09T16:44:19-05:00 | `2c7e72f` | Chris Tusa | Release v0.3.10: Fix install from file path, fix conflicts array crash | | 2026-03-09T16:12:57-05:00 | `242ff32` | Chris Tusa | Release v0.3.9: Fix GC-related crashes, use self for install subprocess | | 2026-03-08T23:40:56-05:00 | `c388de7` | Chris Tusa | Update CI to use Reef 0.5.2 | | 2026-03-08T23:38:23-05:00 | `2f04069` | Chris Tusa | Cache config paths to reduce TOML parsing allocation pressure | | 2026-03-08T20:05:35-05:00 | `604db08` | Chris Tusa | Fix O(n²) manifest generation using StringBuilder (BUG-027 workaround) | | 2026-03-08T15:37:57-05:00 | `b88a15a` | Chris Tusa | Fix infinite CPU loop in packaging step | | 2026-03-08T12:09:49-05:00 | `fbe0637` | Chris Tusa | Release v0.3.5: Wire up all unwired features and cleanup | | 2026-03-08T00:14:11-06:00 | `02417f9` | Chris Tusa | Recursive build-from-source dependency resolver | | 2026-03-07T23:10:59-06:00 | `cc4f8e3` | Chris Tusa | Export $PREFIX, $SYSCONFDIR, $PKG_ROOT to build scripts | | 2026-03-07T22:15:22-06:00 | `3f74bcd` | Chris Tusa | Auto-install missing build dependencies before compilation | | 2026-03-07T21:06:29-06:00 | `7842208` | Chris Tusa | Release v0.3.0: Package Format v2 complete | | 2026-03-07T20:40:41-06:00 | `d7c9548` | Chris Tusa | Rewrite script model: remove .reef type, use zsh, add upgrade flow | | 2026-03-07T20:06:41-06:00 | `261fd8e` | Chris Tusa | Add config file protection on package removal | | 2026-03-07T20:02:14-06:00 | `373d499` | Chris Tusa | Add [dependencies] to .PKGINFO and update TODO status | | 2026-03-07T19:58:29-06:00 | `5289616` | Chris Tusa | Replace rsync with manifest-driven fs.ops installer | | 2026-03-07T19:47:23-06:00 | `6662461` | Chris Tusa | Add mtree manifest module and replace .FOOTPRINT with .MANIFEST in build | | 2026-03-03T19:45:55-06:00 | `f220c00` | Chris Tusa | Release v0.2.11: Fix build script shell invocation and CPU detection | | 2026-03-03T12:12:18-06:00 | `c0d1b54` | Chris Tusa | Update CI to use reef-lang v0.3.4 | | 2026-03-03T11:55:01-06:00 | `8c6d1a8` | Chris Tusa | Release v0.2.10 | | 2026-03-02T23:16:32-06:00 | `601da8e` | Chris Tusa | Fix example config and default arch to match Zygaena defaults | | 2026-02-19T23:56:38-06:00 | `373f7f4` | Chris Tusa | Release v0.2.9: Replace shell workaround with str.join() for footprint processing | | 2026-02-19T23:47:07-06:00 | `6d0400d` | Chris Tusa | Release v0.2.8: Fix register hang by replacing string building with shell pipeline | | 2026-02-19T23:33:19-06:00 | `38023ac` | Chris Tusa | Release v0.2.7: Fix install hang on large packages (161K+ files) | | 2026-02-19T23:17:55-06:00 | `fae09e1` | Chris Tusa | Release v0.2.6: Fix files.db rebuild hang from massive over-allocation | | 2026-02-19T23:04:47-06:00 | `3ae110a` | Chris Tusa | Release v0.2.5: Fix files.db sort hang on large packages | | 2026-02-19T22:50:33-06:00 | `75d402d` | Chris Tusa | Update documentation for v0.2.4 features | | 2026-02-19T22:45:31-06:00 | `b416c5a` | Chris Tusa | Release v0.2.4: Fix resolver bug, dynamic categories, ports index, ports command | | 2026-02-19T20:40:02-06:00 | `2c66115` | Chris Tusa | Support category/name format in port find | | 2026-02-19T20:38:07-06:00 | `606628b` | Chris Tusa | Add hammerhead category to port search list | | 2026-01-30T18:43:31-06:00 | `623ec00` | Chris Tusa | Update woodpecker CI to use reef-lang-0.1.16 | | 2026-01-30T17:31:25-06:00 | `826d764` | Chris Tusa | Release v0.2.3: Add download progress bar and allow non-root builds | | 2026-01-30T10:21:47-06:00 | `9f1c025` | Chris Tusa | updated gitignore | | 2026-01-30T09:19:28-06:00 | `8d1f81d` | Chris Tusa | updated woodpecker to use reef-lang-0.1.15 | | 2026-01-29T22:20:13-06:00 | `a668688` | Chris Tusa | Release v0.2.2: Fix MsgPack parsing and add database documentation | | 2026-01-29T21:48:55-06:00 | `2e8a15f` | Chris Tusa | Implement MsgPack-based package database indexes | | 2026-01-29T21:01:07-06:00 | `9478e62` | Chris Tusa | Fix --root/--prefix flag parsing to work in any position | | 2026-01-29T20:38:25-06:00 | `12dcf11` | Chris Tusa | mkchroot: Generate scripts alongside chroot directory | | 2026-01-29T20:34:49-06:00 | `aac35d4` | Chris Tusa | Move chroot scripts to tools/ and add coral binary to gitignore | | 2026-01-29T20:23:20-06:00 | `6456fce` | Chris Tusa | Remove resolved bug reports and deprecated cache command | | 2026-01-29T20:22:08-06:00 | `11a24e2` | Chris Tusa | Release v0.2.1: Add --root support for alternate filesystem operations | | 2026-01-28T17:41:59-06:00 | `d418f8e` | Chris Tusa | Release v0.1.10: illumos compatibility and file tracking fixes | | 2026-01-28T16:28:26-06:00 | `e4efc01` | Chris Tusa | Release v0.1.9: Fix XZ extraction hang with Reef 0.1.14 | | 2026-01-28T15:38:39-06:00 | `998a2cc` | Chris Tusa | Update extraction bug: XZ files still broken in 0.1.7 | | 2026-01-28T14:05:47-06:00 | `c2a68d0` | Chris Tusa | Release v0.1.7: Bug fixes and new features | | 2026-01-28T11:33:02-06:00 | `e082408` | Chris Tusa | Add bug report for source extraction hang | | 2026-01-28T00:27:36-06:00 | `39a6f92` | Chris Tusa | Add bug report for HTTP redirect and version selection issues | | 2026-01-27T22:00:43-06:00 | `de2a679` | Chris Tusa | Release v0.1.5: Build logging and output ordering fix | | 2026-01-27T17:41:23-06:00 | `e062adc` | Chris Tusa | Release v0.1.4: Local sources, empty package validation | | 2026-01-27T17:28:16-06:00 | `2f44050` | Chris Tusa | Add local source support and empty package validation | | 2026-01-27T15:48:45-06:00 | `ba09e01` | Chris Tusa | Handle existing releases in CI pipeline | | 2026-01-27T15:45:15-06:00 | `295065f` | Chris Tusa | Update Reef compiler to 0.1.10 | | 2026-01-27T13:15:05-06:00 | `2d49d52` | Chris Tusa | Release v0.1.3: Documentation updates and source mirrors | | 2026-01-27T12:14:54-06:00 | `1af9101` | Chris Tusa | Implement source mirrors support (Option B) | | 2026-01-27T11:30:35-06:00 | `457b3cb` | Chris Tusa | Add source mirrors refactor plan (Option B) | | 2026-01-27T11:03:36-06:00 | `f3b3e30` | Chris Tusa | Fix dependency bug | | 2026-01-27T10:51:34-06:00 | `0318423` | Chris Tusa | Add build cleanup options and implementation | | 2026-01-26T23:04:38-06:00 | `017c401` | Chris Tusa | Add gnu and illumos categories to port search | | 2026-01-26T22:54:13-06:00 | `22599cf` | Chris Tusa | Bump version to 0.1.2 | | 2026-01-26T22:49:56-06:00 | `52d81e2` | Chris Tusa | Install build-essential for C headers | | 2026-01-26T22:48:54-06:00 | `5b95ae4` | Chris Tusa | Simplify Reef setup - just add bin to PATH | | 2026-01-26T21:41:02-06:00 | `79d691a` | Chris Tusa | Set REEF_HOME in environment block, add reef-runtime symlink and gcc | | 2026-01-26T21:40:04-06:00 | `0bf8a20` | Chris Tusa | Workaround: symlink stdlib to reef-stdlib (Reef packaging bug) | | 2026-01-26T21:39:03-06:00 | `537795e` | Chris Tusa | Debug: check stdlib contents in binary distribution | | 2026-01-26T21:37:48-06:00 | `3a465aa` | Chris Tusa | Set both REEF_HOME and REEF_STDLIB for reefc | | 2026-01-26T21:35:36-06:00 | `618ab8c` | Chris Tusa | Debug: extract without strip, show structure | | 2026-01-26T21:34:25-06:00 | `c646c2f` | Chris Tusa | Set both REEF_HOME and REEF_STDLIB | | 2026-01-26T21:33:39-06:00 | `ffbb779` | Chris Tusa | Use REEF_HOME instead of REEF_STDLIB | | 2026-01-26T21:32:20-06:00 | `f2ef479` | Chris Tusa | Set REEF_STDLIB environment variable for reefc | | 2026-01-26T21:31:13-06:00 | `6bf7480` | Chris Tusa | Switch to Ubuntu 24.04 for GLIBC 2.38+ compatibility | | 2026-01-26T21:30:02-06:00 | `1c191e9` | Chris Tusa | Fix reefc path and clean up debug statements | | 2026-01-26T21:28:44-06:00 | `2486d04` | Chris Tusa | Debug: show tarball structure | | 2026-01-26T21:27:44-06:00 | `6e928a4` | Chris Tusa | Quote curl command to fix YAML parsing | | 2026-01-26T21:27:00-06:00 | `3a1cbd8` | Chris Tusa | Fix variable escaping for Woodpecker preprocessing | | 2026-01-26T21:09:23-06:00 | `aaa7a4f` | Chris Tusa | Add test_secret to debug secret injection | | 2026-01-26T21:08:30-06:00 | `c9cb48f` | Chris Tusa | Revert to reef_token secret, clean up pipeline | | 2026-01-26T21:06:39-06:00 | `15fe579` | Chris Tusa | Try API_KEY as variable name | | 2026-01-26T21:03:30-06:00 | `3a97f20` | Chris Tusa | Use curly braces for variable expansion | | 2026-01-26T21:02:49-06:00 | `7471b00` | Chris Tusa | Fix YAML quoting for curl commands | | 2026-01-26T20:58:40-06:00 | `82460c8` | Chris Tusa | Fresh Woodpecker pipeline from scratch | | 2026-01-26T20:54:54-06:00 | `d971e66` | Chris Tusa | TEMPORARY: hardcode token to test pipeline | | 2026-01-26T20:53:25-06:00 | `72f3fd6` | Chris Tusa | Add test_secret to debug Woodpecker secrets | | 2026-01-26T20:52:17-06:00 | `279134f` | Chris Tusa | Update secret name to coral-releases | | 2026-01-26T20:43:27-06:00 | `d049473` | Chris Tusa | Add deeper token debug - show first 4 chars, write to file | | 2026-01-26T20:38:34-06:00 | `6678097` | Chris Tusa | Add API debug to test reef-lang access | | 2026-01-26T20:35:41-06:00 | `77b6a6d` | Chris Tusa | Revert to environment/from_secret syntax | | 2026-01-26T20:34:38-06:00 | `5c06999` | Chris Tusa | Try secrets array syntax with source/target | | 2026-01-26T20:32:59-06:00 | `ffd3831` | Chris Tusa | Trigger CI to test trusted security setting | | 2026-01-26T20:28:47-06:00 | `e06e651` | Chris Tusa | Add FORGEJO_TOKEN back to build step | | 2026-01-26T20:25:05-06:00 | `4bda1d3` | Chris Tusa | Try Woodpecker built-in CI_FORGE_TOKEN | | 2026-01-26T20:22:02-06:00 | `b460a2d` | Chris Tusa | Fix YAML syntax for environment secrets | | 2026-01-26T20:19:57-06:00 | `0b63243` | Chris Tusa | Debug secrets and try alternative environment syntax | | 2026-01-26T19:56:57-06:00 | `1bdcbc3` | Chris Tusa | Add authentication to Reef compiler download | | 2026-01-26T19:55:29-06:00 | `293b761` | Chris Tusa | Fix Woodpecker CI pipeline for v3.12.0 compatibility | | 2026-01-26T19:54:26-06:00 | `88e57e2` | Chris Tusa | Add Woodpecker CI pipeline for build and release | | 2026-01-26T18:29:13-06:00 | `1995845` | Chris Tusa | Add curl fallback for HTTP downloads on illumos | | 2026-01-26T18:17:04-06:00 | `22468ff` | Chris Tusa | Add rooted resolver for alternate root installation | | 2026-01-26T18:12:02-06:00 | `3acc88a` | Chris Tusa | Add --root and --prefix support for alternate root installation | | 2026-01-26T17:41:36-06:00 | `ce380ed` | Chris Tusa | Add user confirmation prompts and improve command workflows | | 2026-01-26T17:36:52-06:00 | `cc6f827` | Chris Tusa | Fix http.reef for Reef 0.1.7 compatibility | | 2026-01-26T22:54:49-06:00 | `4b81063` | Chris Tusa | Update http.reef for Reef 0.1.8 native downloads | | 2026-01-25T18:31:58-06:00 | `a9e5b80` | Chris Tusa | updated gitignore to exclude releases/ binary | | 2026-01-25T18:26:46-06:00 | `b6fcf6c` | Chris Tusa | Add release script | | 2026-01-25T18:21:56-06:00 | `b03ff66` | Chris Tusa | Initial commit: Coral package manager for Zygaena |