unix has no runtime CTF since rev 258: every D program fails to compile #8
unix gets no CTF at runtime, so every D program fails to compile. DTrace is unusable on current Hammerhead.
Symptom
Any D program — not just -lv — fails on a current build:
# dtrace -n 'BEGIN { exit(0); }'
dtrace: invalid probe specifier BEGIN { exit(0); }: "/usr/lib/dtrace/procfs.d",
line 315: no symbolic type information is available for unix`kas:
Module does not contain any CTF data
procfs.d:315 is (T->t_procp->p_as == &kas). It is a **library** file, loaded for every compilation, so the failure is not specific to the probe requested. Plain dtrace -l` still works, which is why a probe-count check does not catch this.
Measurement
Two hosts, elfdump -c over every /system/object/*/object (the runtime image,
not the on-disk file):
| alpha11, kernel of 2026-08-10 (pre-Limine) | alpha12, rev 326 |
|
|---|---|---|
| modules with runtime CTF | 210 / 210 | 210 / 211 |
| the one without | — | unix |
unix CTF on disk |
present | present |
| any D program compiles | yes | no |
The data is on disk and is not loaded. Nothing is being stripped.
Mechanism
uts/common/krtld/kobj.c:1332, added by rev 258 (4b1062bdb561, uts: Limine
unix boots to login, 2026-08-21):
if (mp->flags & KOBJ_EXEC) {
#if defined(_DBOOT)
/* Limine already loaded this unix. Re-reading symbols from the
* archive's kernel/unix overwrites st_value with a different
* binary's addresses (kobj_kdi_init then jumps into garbage). */
mp->flags |= KOBJ_RELOCATED;
#else
... kobj_load_elfhdr / read shdrs / get_syms() / get_ctf() ...
#endif
kobj.o is always compiled with -D_DBOOT (uts/intel/Makefile.rules:308,
original, not new), so the #else arm is dead. That arm is the only caller of
get_ctf() for unix, so mp->ctfdata stays NULL and objfs exports no CTF.
The guard's stated reason concerns symbols (get_syms()), but it also removes
get_ctf(), which was not its target. A second consequence of the get_syms()
half is visible in probe counts: fbt unix is 5850 on the pre-Limine host vs 3687
at rev 326 (−2163), which accounts for essentially the entire dtrace -l delta
between the two hosts (60854 → 58709 fbt).
What this is not
It is not attributable to the GNU ld -r kmod work (REMOVE_SUNLD step A′):
unixis sunld-linked on both hosts (DT_SUNW_STRPAD+DT_SUNW_LDMACH
present, noDT_SUNW_KMOD). Its link did not change.- All 210 GNU-linked kmods on alpha12 — e.g.
zfs, whoseelfdump -dshows
SUNW_KMODwithSTRPAD/LDMACHabsent — have working runtime CTF. - 196 of 201 shared fbt modules have identical probe counts across the two
hosts; the exceptions areunixand four console modules that went up. - The guard landed at rev
258; A′'s first commit is272and it became the
default at276.
Suggested approach
Keep skipping get_syms(), but still read the ELF header and section headers and
call get_ctf(). Hazard to design around first: CTF object/function sections
are positional against the module's symbol table. Under the guard the runtime
symbols are Limine's preloaded image while the CTF would come from
kobj_open_file("/kernel/unix") via the boot archive. If those two images can
differ — which is precisely what the guard's comment asserts — the indices will not
correspond and DTrace would report confidently wrong types, which is worse than
the current hard failure. Establish whether the archive copy and the loaded image
are guaranteed identical before wiring CTF back up.
Found while running the REMOVE_SUNLD step A′ validation gate, whose Task 7 Step 2
this was blocking. A′ is exonerated; see the plan docs.
Blast radius is wider than filed: mdb -k is down too, from the same guard
Confirmed at rev 340 on hh-alpha12. The guard skips three calls for unix,
not the two originally reported. kobj_load_elfhdr() is in the dead arm as well,
so unix's mp->hdr is never populated:
if (mp->flags & KOBJ_EXEC) {
#if defined(_DBOOT)
mp->flags |= KOBJ_RELOCATED; /* <- the whole #else is dead */
#else
file = kobj_open_file(mp->filename);
if (kobj_load_elfhdr(file, mp) < 0) /* (1) never runs */
...
if (get_syms(mp, file) < 0) /* (2) never runs */
if (get_ctf(mp, file) < 0) /* (3) never runs */
#endif
ksyms_snapshot.c:148 builds /dev/ksyms by copying that header wholesale and
overriding only offsets and counts -- e_ident, and therefore the ELF magic,
comes straight from unix:
hdr.elf_hdr = ((struct module *)modules.mod_mp)->hdr;
hdr.elf_hdr.e_phoff = offsetof(ksyms_header_t, text_phdr);
hdr.elf_hdr.e_shoff = offsetof(ksyms_header_t, shdr);
...
modules is a bcopy of the head of krtld's list (kobj.c:525), which is unix
(KOBJ_EXEC|KOBJ_PRIM, kobj.c:865). So the header is a zeroed struct.
Measured:
# dd if=/dev/ksyms bs=16 count=1 | od -c
0000000 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
# mdb -k -e '::status'
mdb: /dev/ksyms is not an ELF file
On hh-alpha11 (kernel of 2026-08-10, pre-guard) the same bytes are
177 E L F 002 001 001 006 and mdb -k reports "debugging live kernel (64-bit)
on hammerhead". Note this supersedes the standing belief that mdb is broken by
--export-dynamic -- mdb itself works; it is /dev/ksyms that is malformed.
Full symptom set, one root cause
| skipped call | symptom |
|---|---|
get_ctf() |
no runtime CTF for unix -> every D program fails to compile |
get_syms() |
fbt unix probes 5850 -> 3687 |
kobj_load_elfhdr() |
/dev/ksyms has no ELF header -> mdb -k fails |
Both kernel debuggers are therefore down on current Hammerhead.
Still not A′'s
Re-measured at rev 340 (full build of tip, GNU kmods default): 212 loaded
modules, exactly one without runtime CTF (unix); zfs shows DT_SUNW_KMOD
with STRPAD/LDMACH absent, i.e. GNU-linked, and has working CTF. ksyms takes
its header from unix alone, which is sunld-linked on both arms, so KMOD_LINKER
cannot reach any of this.
Raises the priority
Fixing the CTF half alone would leave mdb -k broken. Whoever takes this should
restore all three calls together, subject to the symbol-table correspondence
hazard already noted above -- which applies to get_syms() most of all, since
that is the one the guard was actually written to prevent.
Fixed in rev 342 (cda2fff331b8). Verified on a full hh-build of tip plus a marker-gated boot matrix.
Two changes were needed, not one
1. kobj.c — restore the three skipped calls, with a check.
Rather than reverting the guard outright, the block now verifies the on-disk image
is the one being run: DT_SYMTAB is read from the running .dynamic, so it must
equal the .dynsym sh_addr the file claims. On a mismatch it keeps the boot
dynsym, skips the file's symbols and CTF, and prints why — the kernel boots
degraded and loudly rather than jumping into garbage.
The guard's original premise was checked before removing it: deploy --target,
the path that refreshed the archive but never the ESP, is now hard-disabled, and
ESP == archive == BE was measured byte-identical on a deployed disk.
2. limine_xboot.c — rename a file-static that shadowed a kernel global.
Restoring the block initially broke the boot differently:
/kernel/fs/objfs: undefined symbol 'modules'
WARNING: mod_load: cannot load module 'objfs'
WARNING: Cannot mount /system/object
The adapter declared static struct boot_modules modules[HH_MAX_MODULES], which
collides with the kernel global struct modctl modules (conf/param.c). Statics
are absent from unix's .dynsym, so it was invisible while unix ran on the
dynsym; loading the full .symtab let the local shadow the global. Pre-Limine
unix has exactly one modules symbol, this build had two. Renamed to
hh_boot_modules.
The guard was concealing this second bug, which would surface for anything
reading unix's full symbol table.
Verification (rev 342, full build, hh-alpha12)
| check | before | after |
|---|---|---|
| runtime CTF census | 211 modules, 1 without (unix) |
212, 0 without |
dtrace -n 'BEGIN{exit(0)}' |
failed to compile | rc=0, probe fires |
dtrace -lvn 'fbt:genunix::entry' |
rc=1, 0 lines | rc=0, 110928 lines (pre-Limine: 110928) |
fbt unix probes |
3687 | 5861 (pre-Limine 5850) |
| total probes | 62346 | 64566 (pre-Limine 64497) |
/dev/ksyms |
16 zero bytes | valid ELF, .symtab 0x123480, correct e_entry |
modules in unix .symtab |
2 | 1 |
| console | undefined symbol 'modules' |
clean |
Build: build_ok=y, 0 errors after "Building OS-Net source".
Boot matrix, deploy-marker verified by sha256: 28/28 — 1 vCPU 4/4, 2 vCPU
12/12, 4 vCPU 12/12, every boot 17 daemons / 0 failed.
mdb -k is NOT fixed by this, and that is correct
The ksyms half of this issue is fixed — /dev/ksyms is now well-formed. mdb -k
still cannot attach, but for an unrelated defect this change merely stopped
masking: /dev/kmem returns EIO for every kernel-image address because those
physical pages are not in phys_install. Filed as
#9 with a reproducer
that does not involve mdb. Closing this one on the DTrace and ksyms criteria;
mdb -k tracks in #9.
Assignees
No assignees
Labels
No labels
Severity
High