Committing before help rework

CLAUDE.md

@@ -5,72 +5,307 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 ## Commands
 
 ```bash
-# Run all specs
+# Run all specs. `make` / `make spec` is the same thing with -v;
+# `make spec_silent` is the bare form.
 crystal spec
 
 # Run a single spec file
-crystal spec spec/command_spec.cr
+crystal spec spec/cligen/flag_spec.cr
 
-# Type-check without running
+# Type-check without codegen — fastest way to validate macro expansion
 crystal build src/cligen.cr --no-codegen
 
-# Install dependencies
-shards install
+# End-to-end flag resolution matrix (default/env/CLI, 16 cases)
+./utils/flag_matrix.sh
+
+# API docs
+make doc && make doc_show
 ```
 
+Setting `DEBUG=1` in the environment turns on `{% debug %}` / `{% puts %}` macro tracing
+in `app/generate.cr`, `command/argument.cr`, `command/help_template.cr`, and
+`command_node.cr#help`. Very noisy, but it's the only way to see generated code.
+
 ## What This Is
 
-`cligen` is a Crystal **shard (library)** — not a standalone application. It wraps Crystal's built-in `OptionParser` with an annotation- and macro-driven system that auto-generates CLI parsers from class definitions. Users of the shard subclass `CliGen::Command` and annotate methods; the library generates the `OptionParser` wiring at compile time via Crystal macros.
+`cligen` is a Crystal **shard (library)** — not a standalone application. Consumers
+subclass `CliGen::Command`, declare flags with the `argument` macro and subcommands with
+the `subcommand` macro, and the library builds the whole CLI tree at compile time. There
+is **no runtime registration and no `OptionParser`** — `cligen` implements its own
+argument scanner.
+
+`lib/cligen` is a symlink back to the repo root so that `require "cligen"` resolves in
+this project's own test programs.
 
 ## Architecture
 
+Compile-time flow, in order:
+
+1. `Command.inherited` installs a `macro finished` hook on each subclass, which calls
+   `define_command_initializer` (and `def_init` when `@[CommandInfo(def_init: true)]`).
+2. `CliGen::App`'s `macro finished` (in `app/generate.cr`) walks `CliGen::Command.subclasses`
+   and emits `App.generate`, which constructs the runtime `Flag(T)` / `CommandNode(T)` object tree.
+3. At runtime `App.process` lazily calls `generate`, then walks `ARGV`.
+
 ### Entry point: `src/cligen.cr`
 
-Defines the `CliGen` module. The `macro finished` hook calls `define_root_parser`, which scans all `CliGen::Command` subclasses at compile time and registers each as a subcommand on the root `OptionParser`. `CliGen.parse` runs the root parser.
+Requires everything and defines `CliGen::VERSION`, `CliGen::APPNAME`
+(`File.basename(PROGRAM_NAME)`), and `override_help_template` (sets
+`CliGen::HELP_OVERRIDE_TEMPLATE` to an absolute path, checked with `file_exists?`).
 
-`ADDITIONAL_DEFAULT_FLAGS` / `add_default_flag` let users inject extra flags into every parser (root and subcommand).
+### Object model
 
-### Command definition: `src/cligen/command.cr`
+The runtime tree is built from four object families. Two of them have a **non-generic
+abstract base** so heterogeneous children can live in one array — this is load-bearing
+and comes up constantly:
 
-`CliGen::Command` is the base class. When subclassed, `macro inherited` installs a `macro finished` block that triggers four code-generation macros in order:
-
-1. `define_actions` — collects all `@[SubCommand]`-annotated class methods into `ACTIONS : Array(String)` and `@@action : String`.
-2. `define_header` — builds the `HEADER` string (banner + examples) from `@[SubCommand]` and `@[CommandSelection]` annotation metadata.
-3. `define_runner` (skipped if `@[CommandInfo(def_runner: false)]`) — generates a `self.run` method that dispatches on `@@action` or a selection variable via a `case` statement.
-4. `define_action_setter` — generates `self.action=` with bounds-checking against `ACTIONS`.
-
-#### Key macros on `Command`
-
-- **`define_argument`** — declares a class-level variable (`@@`) and an annotated setter method (`@[CommandArgument]`). Handles `String`, `Bool`, `Int32`, `Array(String|Int32)`, and `Time`. Optional `check:` proc, `logger:` method, and `def_getter:` flag.
-- **`define_selection`** — like `define_argument` but validates against a fixed list of values; annotated with `@[CommandArgument]` and participates in selection-based dispatch.
-
-### Parser generation: `src/cligen/command/parser.cr`
-
-`CliGen::Parser` is `extend`ed by `Command`. It provides `define_parser`, which generates `self.make_parser(parent_parser)`. That method:
-
-1. Creates a subparser `OptionParser` with the command's `HEADER` as banner.
-2. Wires `@[CommandSelection]`-annotated methods as `parser.on(name, description)` that set the selection variable.
-3. Wires `@[SubCommand]`-annotated methods as subcommand strings that set `@@action`.
-4. Wires `@[CommandArgument]`-annotated methods as `parser.on(short, long, description)` flag handlers.
-5. Calls `CliGen.define_default_flags` (adds `-h`/`--help`, error handlers).
-6. Registers the whole subparser on `parent_parser` under the command's lowercase class name.
-
-### Annotations
-
-| Annotation | Applied to | Purpose |
+| Generic | Base | Why the base exists |
 |---|---|---|
-| `@[CommandInfo(description:, def_runner:)]` | Command subclass | Required; provides the description shown in root help; `def_runner: false` skips auto-generating `run` |
-| `@[SubCommand(description:, examples:)]` | class method on Command | Marks a method as a dispatachable subcommand |
-| `@[CommandArgument(short:, long:, description:, type:)]` | class method on Command | Generated automatically by `define_argument`/`define_selection`; drives parser wiring |
-| `@[CommandSelection(selector:, description:, examples:)]` | class method on Command | Alternative to `SubCommand`; dispatches via a named selector variable instead of `@@action` |
-| `@[CommandPreRun]` | class method on Command | Methods run unconditionally before dispatch inside `self.run` |
-| `@[DefaultFlag]` | (reserved) | Defined but not currently used in generation |
+| `Flag(T)` | `BaseFlag` | `Array(BaseFlag)` holds flags of mixed `T` |
+| `CommandNode(T)` | `BaseCommandNode` | `Array(BaseCommandNode)` holds the command tree |
 
-### Supporting files
+- **`src/cligen/flag.cr`** — `Flag(T)`. Owns `@value`, `@default`, `@options`, `@validate`,
+  `@on_match`, `@format`. Its `process`, `coerce`, and `validate!` are giant compile-time
+  `{% if %}` chains over `T`. Supported `T`: `Bool`, `String`, `Int*` (signed and unsigned),
+  `Float*`, `Time`, `Array(Int*|Float*|String|Coercable)`, plus any type that
+  `extend`s `CliGen::Coercable` or `CliGen::Parsable`. Unsupported `T` is a `{% raise %}`.
+- **`src/cligen/flag/base.cr`** — `BaseFlag`: `var`, `short`, `long`, `long_key`,
+  `env_var`, `description`, `delimiter`, `meta`. `@long_key` is `@long` split on `\s|=`,
+  so a declaration like `long: "--help TOPIC"` still keys off `--help`.
+- **`src/cligen/flag/meta.cr`** — `FlagMeta` record (type/array/format/default/options),
+  stringified metadata used solely by the ECR help template.
+- **`src/cligen/command_node/base.cr`** — `BaseCommandNode`: the tree walk, `find_match`,
+  `get(long:)`/`get(short:)`, `all_flags`, and the duplicate checks.
+- **`src/cligen/command_node.cr`** — `CommandNode(T)`: `subcommands`, `help`, `check!`,
+  and the main `process(Array(Arg))` loop, all of which need `T`.
+- **`src/cligen/app.cr`** — `App < CommandNode(Nil)`. Singleton (`@@instance`), root of
+  the tree, adds env-var collision detection and the error boundary.
 
-- `src/cligen/format.cr` — date/datetime format strings used by `Time` argument parsing.
-- `src/cligen/regex.cr` — regexes for validating date/datetime input strings.
+### Argument scanning: `src/cligen/arg.cr`
+
+`CliGen::Arg` wraps `(value, index)` and tracks a one-way `processed?` flag. Calling
+`#processed` twice raises `ArgReprocessedError` — a deliberate fail-fast so double-consumption
+bugs surface during development rather than silently eating an argument. The parser
+never does index math; it filters with `args.reject(&.processed?)`.
+
+`Arg` also holds the class-level predicates `flag?`, `int?`, `uint?`, `float?`, which
+delegate to `CliGen::Regex`.
+
+### The dispatch loop: `CommandNode(T)#process`
+
+`find_match(token)` returns a `BaseCommandNode`, a `BaseFlag`, or a `MatchType` enum
+member. `process` `case`s over that:
+
+- **`BaseCommandNode`** — a child command matched. Hands the unprocessed args off to the
+  child and `exit 0`. The cast back to a concrete `CommandNode(T)` is done by a
+  macro-generated `case` over `CliGen::Command.subclasses`; falling through raises
+  `UnknownCommandNodeError`.
+- **`BaseFlag`** — if `requires_arg?` (i.e. `T != Bool`), it is handed the run of following
+  args that either match nothing or are in the flag's `options`; otherwise `process` with no args.
+- **`MatchType::SubCommand`** — records `matched_subcommand`; a second one raises.
+- **`MatchType::Help`** — raises `HelpRequestedError` carrying the rendered help.
+- **`MatchType::FlagWithArg`** — `--flag=value`, re-split and dispatched.
+- **`MatchType::FlagMultipleShort`** — `-abc` bundles. Only the last flag in a bundle may
+  take an argument; otherwise `FlagBundleError`. If the second char isn't a known flag,
+  raises `FlagArgumentError` (inline short args like `-n5` are not supported).
+- **`MatchType::NoMatch`** — raises `HelpRequestedError` with an "unknown token" preamble.
+
+After the loop, if no child command took over: instantiate `T`, run its
+`@[PreRunCommand]` methods, then dispatch to the matched `@[SubCommand]` method or `main`.
+`App` itself (`T == Nil`) just prints help.
+
+### Value resolution
+
+`Flag(T)#value!` resolves in strict priority order — **CLI arg → env var → default →
+raise `MissingRequiredFlagError`** — and then runs `validate!(v)` on whatever it got, so
+env-var and default values are validated on exactly the same path as CLI input.
+
+Two footguns here, both previously live bugs:
+
+- `validate!(v : T? = nil)` must use `v = value! if v.nil?`, **not** `v ||= value!`.
+  With `Flag(Bool)` and `default: false`, `||=` treats `false` as absent and recurses
+  into `value!` forever.
+- The `MissingRequiredFlagError` raise must stay **above** the `validate!(v)` call in
+  `value!`, or the same infinite recursion occurs when nothing resolved.
+
+Env vars for command arguments are **namespaced** `<COMMAND>_<VAR>` (e.g. `GREET_LEVEL`,
+not `LEVEL`) unless an explicit `env_var:` is given. Global flags are un-namespaced,
+derived as `long.gsub(/--/,"").gsub(/-/,"_").upcase`. Both macros reject an explicit
+`env_var:` containing `-`.
+
+### Validation: `check!`
+
+Run at the top of every `process`, so misconfiguration fails on first invocation:
+
+- `Flag#check!` — rejects `-h` / `--help` (`ReservedFlagError`).
+- `CommandNode#check!` — duplicate shorts/longs across `@flags + GLOBAL_FLAGS`
+  (`DuplicateFlagError`), duplicate child command names (`DuplicateCommandError`), and
+  (when `T != Nil`) requires either subcommands or a `#main` (`MissingDispatchError`).
+- `App#check!` — `super`, then env-var collisions across `all_flags + GLOBAL_FLAGS`.
+  `all_flags` recurses the whole tree; empty env vars are excluded.
+
+### Errors: `src/cligen/exceptions.cr`
+
+Everything derives from `CliGen::Error`, in three buckets plus a signal:
+
+- **`InternalError`** — framework invariant broken; should never reach a user
+  (`ArgReprocessedError`, `RegexInvariantError`, `UnknownCommandNodeError`).
+- **`ConfigurationError`** — the shard *consumer* wired something wrong
+  (`ReservedFlagError`, `DuplicateFlagError`, `DuplicateCommandError`,
+  `MissingDispatchError`, `FlagNotFoundError`, `FlagMissingArgumentError`,
+  `ParseableInvariantError`).
+- **`RuntimeError`** — bad end-user input (`MissingRequiredFlagError`, `ValidationError`,
+  `FlagArgumentError`, `InvalidFlagValueError`, `InvalidOptionError`, `UnknownFlagError`,
+  `FlagBundleError`, `TimeParseError`).
+- **`HelpRequestedError`** — not an error; carries rendered help, caught and `exit 0`.
+
+`App.handle_command_raises` is the single error boundary: `RuntimeError` and
+`ConfigurationError` `abort` with the message, `HelpRequestedError` prints and exits 0.
+Each `rescue` does `Fiber.yield` first to let buffered `Log` output flush — a known-fragile
+workaround, not a design.
+
+**Never let a non-`CliGen` exception escape.** The whole point of the typed hierarchy is
+that `handle_command_raises` catches everything; a stray stdlib exception (e.g.
+`Time::Location::InvalidTimezoneOffsetError`) reaches the user as a stack trace. Wrap and
+re-raise at the boundary — `Flag(Time)` does exactly this, translating
+`TimeParseError` into `InvalidFlagValueError`.
+
+### Time parsing: `src/cligen/timeparse.cr`
+
+`CliGen::Timeparse.parse(raw) : Time` is a single `case` over four anchored matchers from
+`CliGen::Regex`, each branching on whether `match["timezone"]?` is present (offset-aware
+vs. `parse_local`). Supported: `%Y-%m-%d %H:%M:%S [%z]`, `%Y-%m-%d [%z]`, `@<epoch> [%z]`,
+and one-or-more relative operations (`"+1 day -2 hours"`).
+
+`timeparse/relative_operation.cr` — `RelativeOperation` struct. `get_operations` `scan`s
+with `RELATIVE_OPERATION` and applies each in sequence. `apply` is macro-generated from
+`OperationUnit.constants` using `case ... in` (exhaustive, so no `else` and the return
+type collapses to `Time`).
+
+### Regex: `src/cligen/regex.cr`
+
+Two tiers, and the distinction matters:
+
+- **Components** (`TIMEZONE`, `TIME`, `DATE`, `EPOCH`, `RELATIVE`) are **unanchored** and
+  exist only to be interpolated. Interpolating a Crystal `Regex` renders it as
+  `(?-imsx:...)`, so an anchor here would end up buried mid-pattern in the composite and
+  could never match.
+- **Matchers** (`INPUT_DATE_FULL`, `INPUT_DATE_SIMPLE`, `INPUT_DATE_EPOCH`,
+  `INPUT_RELATIVE_OPERATIONS`) are fully `^...$` anchored. Match user input only against these.
+
+`TIMEZONE`'s offset is deliberately bounded to `23:59` so it can't produce an offset
+outside `Time::Location.fixed`'s ±24h limit.
+
+Also here: `FLAG_REGEX`, `FLAG_WITH_ARG`, `FLAG_MULTIPLE_SHORT`, `INT`, `UINT`, `FLOAT`.
+
+### Help output
+
+`CommandNode#help` picks a template at compile time, in priority order: the command's own
+`HELP_TEMPLATE` (set by the `help_template` macro) → `CliGen::HELP_OVERRIDE_TEMPLATE`
+(set by `CliGen.override_help_template`) → the bundled default.
+
+The default path is **hardcoded relative to the CWD**:
+`ECR.render("lib/cligen/src/cligen/template/cmd_help.ecr")`. Anything that runs a cligen
+binary must therefore run from a directory with a `lib/cligen` — which is why
+`utils/flag_matrix.sh` `cd`s to the project root.
+
+The template renders per-flag detail (type, env var, format, delimiter) only when
+`verbose?`, which reads the `--verbose` global flag.
+
+## Public macro API
+
+Called inside a `CliGen::Command` subclass:
+
+| Macro | File | Purpose |
+|---|---|---|
+| `argument(var : T, description, ...)` | `command/argument.cr` | Declares a flag-backed ivar. Options: `long`, `short`, `validation`, `on_match`, `def_setter`, `def_getter`, `options`, `delimiter`, `format`, `allow_no_verification`, `env_var` |
+| `selection(var : T, description, options, ...)` | `command/selection.cr` | Like `argument` but constrained to a fixed option list |
+| `subcommand(func, description, examples) { ... }` | `command/subcommand.cr` | Defines a `@[SubCommand]` method from a block |
+| `help_template(filepath)` | `command/help_template.cr` | Per-command ECR override |
+
+Module-level:
+
+| Macro | File | Purpose |
+|---|---|---|
+| `CliGen.add_global_flag(T, long:, description:, ...)` | `global_flag/add_global_flag.cr` | Appends to `GLOBAL_FLAGS`; visible on every command |
+| `CliGen.override_help_template(filepath)` | `cligen.cr` | Project-wide ECR override |
+
+`global_flag.cr` dogfoods `add_global_flag` for the built-in `-v/--verbose`.
+
+### Extension points
+
+| Module | Contract | Used for |
+|---|---|---|
+| `CliGen::Coercable` | `self.coerce(arg : String)` | Building `T` from a single string (also used for env vars and array elements) |
+| `CliGen::Parsable` | `self.parse_args(args : Array(CliGen::Arg))` | Multi-arg consumption; **must** mark at least one `Arg` as `processed` or `ParseableInvariantError` is raised |
+
+Both are `extend`ed, not `include`d — hence the metaclass checks `T.class < CliGen::Parsable`
+in `flag.cr`.
+
+## Annotations
+
+`src/cligen/annotations.cr` declares seven; only five are wired:
+
+| Annotation | Applied to | Status |
+|---|---|---|
+| `@[CommandInfo(description:, def_init:)]` | Command subclass | **Required.** `description` must be a `StringLiteral`; `def_init: true` generates a no-arg initializer + `self.get` singleton accessor |
+| `@[Argument(short:, long:, description:, validation:, on_match:, options:, delimiter:, format:, env_var:)]` | ivar | Emitted by `argument` **and** `selection`; read by `App.generate` and `define_command_initializer` |
+| `@[SubCommand(description:, examples:)]` | method | Emitted by `subcommand`; read by `CommandNode#subcommands` and the dispatch `case` |
+| `@[PreRunCommand]` | method | Run unconditionally before subcommand dispatch |
+| `@[Selection]` | ivar | *Read* by `generate`/`define_command_initializer`, but never emitted — `selection` emits `@[Argument]`. Effectively dead. |
+| `@[ProxyCommand]` | — | Declared only; unused |
+| `@[Trigger]` | — | Declared only; unused |
+
+## Crystal macro gotchas
+
+These have each caused real bugs in this codebase — check for them before touching a macro:
+
+- **Macro *arguments* arrive as unresolved AST** (`Path`, `Generic`), not `TypeNode`.
+  `==` silently returns false and `<` raises `undefined macro method 'Path#<'`. Call
+  `.resolve` first: `validation.return_type.resolve == Bool`, `type.resolve <= Array`.
+  Generic type *parameters* (`T` inside `Flag(T)`) are already `TypeNode` and are safe as-is.
+- **`{% verbatim do %}`** is required whenever a macro body must emit macro code that runs
+  in the *subclass's* `macro finished` context (see `command.cr`, `def_init.cr`,
+  `define_command_initializer.cr`).
+- Signed/unsigned dispatch is done by string inspection, since there's no `UInt` supertype
+  to test against: `{% int_case = T.stringify =~ /^UInt/ ? "uint?".id : "int?".id %}`.
+- `macro finished` ordering is why `App.generate` can see every `Command` subclass.
 
 ## Spec structure
 
-`spec/command_spec.cr` uses Crystal's macro system heavily: most `it` blocks are generated at compile time by inspecting `CommandSubclass` via `@type` introspection. The `macro finished` wrapper around the entire `describe` block is required because `make_parser` and `run` don't exist until all `macro finished` hooks have fired.
+- `spec/cligen/arg_spec.cr` — plain specs for `Arg`.
+- `spec/cligen/flag_spec.cr` — reopens `CliGen::Flag(T)` to expose `test_coerce` and a
+  `String`-array `process` overload, then generates most `it` blocks with `{% for int in
+  Int.subclasses %}` etc. so every numeric width is covered. `MyGoodData` / `MyBadData`
+  exercise `Coercable` / `Parsable`, including the "didn't mark anything processed" failure.
+
+`utils/flag_matrix.cr` + `utils/flag_matrix.sh` cover what specs can't: env vars must be
+set before process startup, so each of the 16 cases needs its own process. Run this after
+touching value resolution, `check!`, or help rendering.
+
+## Current state (branch `object_rework`)
+
+Green: `crystal build --no-codegen` clean, `crystal spec` 164 examples / 0 failures,
+`./utils/flag_matrix.sh` 16/16. Targeting v0.2.0 (`shard.yml` and `CliGen::VERSION` both
+still say `0.1.0`).
+
+Known gaps, all deliberate:
+
+- **Enum support** — deferred to v0.2.1; requires reworking five type-dispatch chains.
+  DESIGN.md marks it as planned.
+- **`Array(Time)`** — unsupported; the three array element chains have no `Time` case.
+- **Colon-based relative time formats** (`[-+]%H:%M:%S`) — documented in DESIGN.md as planned.
+- `@[Selection]`, `@[ProxyCommand]`, `@[Trigger]` are dead (see above).
+- The `Fiber.yield` log-flush workaround in `app.cr` is fragile; synchronous dispatch
+  would be deterministic.
+
+## Repo conventions
+
+- Every `.cr` file starts with `# SPDX-License-Identifier: MIT` and
+  `# Copyright 2026 Tristan Ancelet`. Add these to new files.
+- **`.gitignore` is a deny-all allowlist** (`*` followed by `!` exceptions). New top-level
+  files and directories are ignored silently and fail closed — add an explicit `!` entry
+  when creating one. This has bitten `DESIGN.md` and `spec/`.
+- `Log` is stdlib (`::Log.for(...)` on `BaseFlag` and `BaseCommandNode`), deliberately
+  chosen over a dependency. Always use the block form — it's zero-cost when the level is disabled.
+- Licensed MIT.