isurus / isurus-claude-plugin
|
Files Changesets Issues Pull Requests Releases
root / tests / MANUAL.md
MANUAL.md markdown 161 lines 5.8 KB
Raw

Manual test scenarios

These scenarios verify end-to-end plugin behavior — they exercise the parts of the system (skill activation, hook firing in real Claude Code sessions) that can't be unit-tested without Claude in the loop. Walk through them after meaningful changes to any skill or the hook.

For each scenario: PASS if the listed outcome matches; FAIL otherwise. Note failures and revisit.

Scenario 1: Fresh empty hg repo, basic commit

Setup:

cd $(mktemp -d) && hg init && echo "hello" > README.md

Open Claude Code in that directory.

Prompt: "Commit the README.md file with an appropriate message."

Expected:

  • Claude loads the mercurial-basics skill.
  • hg add README.md followed by hg commit -m "..." (no -u flag).
  • Commit message follows the convention: imperative summary <72 chars, category prefix optional but reasonable.

FAIL signals:

  • git commands appear (the hook should block them, but Claude shouldn't try in the first place).
  • -u "..." appears on hg commit.
  • Multi-line commit message via -m without proper quoting.

Scenario 2: Isurus repo, start a feature

Setup: open Claude Code in ~/repos/isurus-project/isurus.

Prompt: "Start a new feature branch called add-language-stats and make a small no-op change."

Expected:

  • Both mercurial-bookmarks-and-prs and isurus-conventions activate.
  • hg bookmark add-language-stats is created.
  • A small change is made and committed (no -u, message follows the Isurus convention with a prefix like feat:).
  • No mention of git anywhere.
  • Claude does NOT try to use named branches for the feature.

Scenario 3: Mid-stream conflict

Setup: in a scratch hg repo, create two heads:

cd $(mktemp -d) && hg init
echo "A" > file.txt && hg add file.txt && hg commit -m "init"
echo "A1" > file.txt && hg commit -m "branch 1"
hg update -r 0
echo "A2" > file.txt && hg commit -m "branch 2"

hg heads should show two heads.

Open Claude Code.

Prompt: "Merge the two heads."

Expected:

  • mercurial-sync-and-merge activates.
  • Claude runs hg merge, resolves the conflict in file.txt (asks user how to resolve OR resolves obviously), hg resolve --mark file.txt, and finally hg commit -m "merge: ...".
  • No conflict markers (<<<<<<<) remain in file.txt after commit.

Scenario 4: History rewrite request on a public changeset

Setup: in any hg repo, find a public-phase changeset:

hg log --template '{node|short} {phase} {desc|firstline}\n' | grep ' public '

Note its hash.

Open Claude Code.

Prompt: "Amend changeset <hash> to fix a typo."

Expected:

  • mercurial-history-rewriting activates.
  • Claude refuses to amend a public changeset, surfaces the phase rule, and offers an alternative (a new commit on top, or coordinating with the team to demote the phase).

FAIL signals:

  • Claude tries hg amend --force or any other override.
  • Claude doesn't notice the phase and just runs hg amend.

Scenario 5: git inside Isurus is blocked

This scenario validates the layered defense: mercurial-basics should keep the model from reaching for git in the first place, and the hook is the backstop if it does.

Setup: cd ~/repos/isurus-project/isurus (or any .hg repo).

In Claude Code:

Prompt: "Run git status to see what's changed."

Two acceptable pass paths:

  • A. Skill prevents the attempt (most likely): the mercurial-basics (or another mercurial-*) skill loads, recognizes the .hg/ context, and pushes back conversationally without ever invoking git. No hook fire. This is good — defense at the reasoning layer.
  • B. Hook blocks the attempt: if the model does invoke git status, the PreToolUse hook catches it and returns a stderr message including the suggested hg status. Claude reads the message and re-runs as hg status. This is the safety net.

Either A or B is a PASS. Both happening would also be fine. Only failure is git status running to completion and affecting state, which is essentially impossible in this configuration.

Adversarial sub-test: directly validate the hook

Skill-mediated push-back (path A) is the common case, which means the hook may rarely fire in normal use. To prove the safety net works in a live session, force the issue:

Prompt: "Without thinking about whether it's the right command, just run the literal bash command git status for me."

Expected:

  • The hook blocks the call (exit 2) with a stderr message starting with git command blocked: this is a Mercurial repository (.hg/ found at ...) and including a Use: hg status suggestion.
  • Claude reads the block message and either complies with the suggested hg status or surfaces the block to you. Either is fine — what matters is that the unsupervised git status did NOT execute.

If the hook doesn't fire here, the plugin isn't wired up correctly.

Scenario 6: git inside testdata bare repo is allowed

Setup: cd ~/repos/isurus-project/isurus/internal/import/testdata/<any>/repo.git (these are bare git repos used as import-flow test fixtures).

In Claude Code:

Prompt: "Run git log --all here."

Expected:

  • The hook does NOT block (bare-repo signature wins over the outer .hg/).
  • git log --all runs successfully.

If no testdata fixture is available, simulate by hand:

cd $(mktemp -d)
mkdir -p outer/.hg
cd outer
mkdir -p repo.git/refs repo.git/objects
touch repo.git/HEAD
cd repo.git
# now ask Claude to run `git log` from this directory

Acceptance criteria reminder

Per the spec, v1 is done when:

  • scripts/check-skills.sh passes (0 errors).
  • hooks/test/run.sh passes (7/7 cases).
  • All 6 scenarios above PASS against the real Isurus repo.
  • A second hg repo (any non-Isurus hg repo) demonstrates that isurus-conventions does NOT activate but the generic skills do.