cligen/DESIGN.md

Crystal Cli Generator

Author: Tristan Ancelet Email: tristanancelet@yahoo.com

Overview

This document outlines the overall design of the CliGen shard & it's underlying classes/objects & their usecases.

How it works/High-Level overview

Using crystal macros, you define the shape (arguments/flags, work functions/subcommands, etc) and later on in src/cligen/app/generate.cr will use macros to (at compile time) generate CommandNode(T) objects & Flag(T) objects to contain your command/subcommand/arg parsing code from the data you provided in your CliGen::Command subclass.

Architecture

The desire for this project is to provide a framework for generating commandline arg-parses & command dispatch built into a class/object.

The idea is to (much like JSON::Serializable & YAML::Serializable) is to use macros to help defining a Command object and abstract the command-line away from the codebase needing input from the user at init time.

Command Objects

The core of this codebase is the CliGen::Command object.

This is the object that is meant to provide you the hook into being able to utilize the codebase. It is where you are able to define all of your expecetd arguments/flags & any selectables (aka, instance variable that you want the user to choose one value for. ex: -f|--format json|yaml|ecr).

To make this all possible we load the base class with macros that help define your variables & inform the framework how it needs to handle arguments being presented to your command object setters.

require "cligen"

module MyModule
  @[CliGen::CommandInfo(description: "My command that does cool things")]
  class MyCommand < CliGen::Command
    argument(myvar : Int32 = 23,
      long: "--myvar VAR",
      short: "-m",
      description: "This tells the utility how many times to do thing",
      validate: ->(v : Int32) : Bool { (1..23).includes?(v) }
    )

    argument(output_format : String = "ecr",
      long: "--format FORMAT",
      short: "-f",
      description: "Inform the utility what output you want the data in",
      options: %w[ json yaml ecr ]
    )

    DO_THING_EXAMPLES = [
      "myutil mycommand do_thing --myvar 5"
    ]

    subcommand do_thing, description: "Do the THING", examples: MyModule::MyCommand::DO_THING_EXAMPLES do

      output = 0

      @myvar.times do |i|
        puts "thing done %i times" % [ i + 1 ] 
        output += i
      end

      puts "Total : %i" % output
    end
  end

  CliGen::App.process
end

The examples like above provide a "DSL-esk" way of defining:

• Instance Variables,
• Short/Long flags
• Description of the flags (used in the help output as well)
• A verification proc/lambda for doing ad-hoc checks of the value provided by the user (for cases where the options: key isn't expressive enough)
• A static list of valid options via the options: key (currently compile-time; will open up to runtime collection of options later on)
• define subcommands of this current command

CliGen::CommandNode(T)

This is the "CliGen" representation of your command. Using crystal macros & generics when the CommandNode is created it will generate the internal Generic methods based on the defined command object and whether it has subcommands & main method, etc.

This is where all of the actual argument processing gets done in the CliGen::CommandNode(T)#process(args : Array(CliGen::Arg)) method.

A main-loop of scoped parsing happens in it's #process method and if a child-command is detected CliGen::Command -> MyModule::MyCommand -> MyModule::MyOtherCommand it will gather all unprocessed arguments and will call that CommandNode(T)'s #process method.

This was done because it effectively keeps arguments scoped in their own contexts without causing something like OptionParser implementation having flag/argument bleed between commands and subcommands if multiple parsers are defined (aka OptionParser does not #shift arguments off as they are processed, which end up causing arguments to bleed into child instances).

Alongside it's main duty of handling CLI parsing of arguments it also serves to handle any framework related raises from Flag(T) (invalid values, incorrect custom data class behavior, internal errors, etc).

CliGen::App

This object is the root application that handles all initial execution from bare CLI arguments. It's just a subclass of CommandNode(Nil) (Subclassed with Nil for simplicity and to not deal with code duplication between macro generated CommandNode(T) and a custom App object (with them needing almost exactly the same methods).

For the majority of the process, this object just serves as the inital entrypoint to the CLI parsing and handles raises of any Framework related error classes (help output, exceptions related directly with the CommandNode(T) objects)

CliGen::Flag(T)

This is the object that handles option/argument parsing. It's a Generic Container who's internal layout changes depending on the type it's initalized with.

The only (currently as of writing this doc) types that are supported are:

• All Int permutations (Int8, UInt8, etc)
• All Float permutations (Float32, etc)
• String
• Bool
• Enum (planned, but not implemented yet)
• Time (strict formatting allowed)
• Array(T)

Time Limitations

The framework has some strict formatting for time arguments

It only accepts data in the following static formats:

• %Y-%m-%d %H:%M:%S %z (ex: 2026-08-30 19:00:00 +0000)

  • This specifies the date, time & timezone/offset (allowing you to customize the time)

• %Y-%m-%d %H:%M:%S (ex: 2026-08-30 19:00:00)

  • This specifies the date & time and will default to the local timezone of the device it's being run on (via ::Time.parse_local) (would need much more conditionals but it gets the point accross)

• %Y-%m-%d %z (ex: 2026-08-30 +0000)

  • This specifies the date & timezone/offset, and will default to 00:00:00 for the time value

• %Y-%m-%d (ex: 2026-08-30)

  • This specifies the date, and will default to 00:00:00 for the time value and to the local timezone of the host (via ::Time.parse_local)

• @%s %z (ex: @1788122911 +0900)

  • This specifies the epoch time and the timezone you want the final Time object to be in (this gets the provided epoch time as UTC via ::Time.unix and then setting the timezone to the provided offset)

• @%s

  • This specifies the epoch time and via ::Time.unix will default to ::Time::Location::UTC for the timzone

However, minimal support for relative time has been added. Valid formats for this are:

• [-+][0-9]+ (years|months|weeks|days|hours|minutes|seconds) %z (ex: +2 hours -0900)

  • This provides a relative time to look for and what timezone offset to convert to after the relative time is calculated via ::Time#in

• [-+][0-9]+ (years|months|weeks|days|hours|minutes|seconds) (ex: -2 hours)

  • This provides a relative time to look for and will default to the local timezone via ::Time.local

• [-+]%H:[-+]?%M:[-+]?%S %z (ex: -2:3:3 -0900) (planned but not currently implemented)

  • This provides a hour, minute and second to decrease/increase by (depending on each field's sign) and a timezone to convert to afterwards with ::Time#in

• [-+]%H:[-+]?%M %z (ex: -2:3 -0900) (planned but not currently implemented)

  • This provides a hour and minute to decrease/increase by (depending on each field's sign) and a timezone to convert to afterwards with ::Time#in

• [-+]%H:[-+]?%M:[-+]?%S (ex: -2:3:3) (planned but not currently implemented)

  • This provides a hour, minute and second to decrease/increase by (depending on each field's sign)

Note: Multiple human-readable operations

The relative operations for getting time relatively to the current time on the running system +1 days can be chained and the specific operation order is important when providing your ops.

So when doing multiple you can provide them like so

mycli mycmd --time '+2 weeks -1 month +30 days -0900`

This essentially grabs the local time (via ::Time.local), setting it's timezone to the -0900 offset, and then adding 2 weeks/14 days, jumping back a full month (dependant on the actual month) and then going forward 30 days.

Note: Relative operations short format (planned but not implemented yet) For the [-+]%H:[-+]?%M:[-+]?%S formats if the sign's (+-) aren't provided for the %M or %S fields it will default to the sign provided to the %H field

Array(T) limitations

However, with Array(T) types there are a few limitations. The only supported in-place T-types are the core classes (Int*, Float*, String). However, provided you do the below extentions to your custom data class you can also use it as a data class with Arrays.

The framework (deliberately) limits the :options key to an array of individual choices that the user can provide instead of permutations of valid Array(T) values. So when providing an Array(T)

require "cligen"

module MyModule
  @[CliGen::CommandInfo(description: "Test Command")]
  class MyCommand < CliGen::Command
    argument(sources : Array(String),
             description: "The source of all your pain",
             options: %w[ tacos golang coffee ]
    )

    def main
      @sources.each_with_index do |source, index|
        puts "%d : %s" % [index + 1, source]
      end
    end
  end
end

CliGen::App.process

Providing it in the array format will have the framework collapse it into just the Array(T) values so that they can be value-checked (Array(T)#includes?(val))

Of which when an invalid value is provided the framework will abort directly if an invalid value is provided

(ins) tristan@arcanetome $: crystal run test4.cr -- mycommand --sources golang coffee pain
crystal-run-test4.tmp: 'pain' is not a valid value for --sources (valid: ["tacos", "golang", "coffee"])

However when run with valid values it has no issues

(ins) tristan@arcanetome $: crystal run test4.cr -- mycommand --sources golang coffee 
1 : golang
2 : coffee

Custom Data Types

However, there is an API defined via the CliGen::Parsable and CliGen::Coercable modules that will allow you to customize parsing of a Custom DataType.

This is limited to parsing raw strings (of any format) into whatever your data is supposed to represent

module MyModule
  class MyData
    extend CliGen::Parsable
    extend CliGen::Coercable
    
    @value : Int32
    
    def initialize(value : String)
      @value = value.to_i32
    end
    
    def self.parse_args(args : Array(CliGen::Arg))
      arg = args.first
      # Mark the argument as processed
      arg.processed
      new(arg.value)
    end
    
    def self.coerce(arg : String)
      new(arg)
    end
  end
end

Override #to_s for CLI help readablility

If you do plan to use your custom data-type with the you will need to override the #to_s(io : IO) method to ensure that the help output can render it correctly

Example of a Custom DataType use in a CliGen::Command subclass argument/flag:

require "cligen"

module MyModule
  class MyData
    extend CliGen::Coercable
    extend CliGen::Parsable
    
    @@instances = {} of String => MyData
    
    getter name : String
    getter work : Proc(Nil)
    
    def initialize(@name, &@work)
      unless @@instances.has_key?(@name)
        @@instances[@name] = self
      else
        raise "HUH?"
      end
    end

    def to_s(io : IO)
      io << @name
    end

    def self.get(name : String)
      @@instances[name]
    end
    
    def self.parse_args(args : Array(CliGen::Arg))
      arg = args.first
      arg.processed
      get(arg.value)
    end
    
    def self.coerce(arg : String)
      get(arg)
    end

    def self.all 
      @@instances.values
    end
  end

  MyData.new("test") do 
    puts "Did a test"
  end

  MyData.new("really") do 
    puts "Yes really"
  end

  MyData.new("why") do 
    puts "Just because"
  end

  @[CliGen::CommandInfo(description: "Test Command")]
  class MyCommand < CliGen::Command
    argument(source : MyModule::MyData,
             description: "The source of all your pain",
             options: MyModule::MyData.all
    )

    def main
      @source.work.call()
    end
  end
end

CliGen::App.process

Providing the overridden #to_s method makes the help output render your objects in a readable way

(ins) tristan@arcanetome $: crystal run test2.cr -- mycommand  -h
Command: mycommand
Description: Test Command


Flags:
---------------------------------------------------------------
    --source      The source of all your pain (valid: test, really, why)


Global Flags
---------------------------------------------------------------
    -v,--verbose      Enable verbose output from program & help output (default: false)



Note: When wanting help output of any command you can provide the -h/--help flags or help command

(ins) tristan@arcanetome $: crystal run test2.cr -- mycommand --source why
Just because

Without the #to_s method overridden you'll get the crystal representation of your object instead (like below)

(ins) tristan@arcanetome $: crystal run test2.cr -- mycommand  -h
Command: mycommand
Description: Test Command

Flags:
---------------------------------------------------------------
    --source      The source of all your pain (valid: #<MyModule::MyData:0x7f683ad0cc90>, #<MyModule::MyData:0x7f683ad0cc60>, #<MyModule::MyData:0x7f683ad0cc30>)


Global Flags
---------------------------------------------------------------
    -v,--verbose      Enable verbose output from program & help output (default: false)



Note: When wanting help output of any command you can provide the -h/--help flags or help command

Sill valid, but if you plan to make a registry based command/work/strategy selector you might want to override the #to_s method

CliGen::Coercable

def self.coerce(arg : String)
  new(arg)
end

The coerce method provides you the ability to parse a single string value into your class/datatype. This is generally only used when parsing from ENV VAR and when being used by parsing from an Array(T) type.

This should only be used in the case you need a simple datatype that can be learned from a single string.

CliGen::Parsable

def self.parse_args(args : Array(CliGen::Arg))
  arg = args.first
  # Mark the argument as processed (required)
  arg.processed
  new(arg.value)
end

This method is used the most and is used for when using the bare class as the generic type in the Flag(T). With this the Flag(T) will collect all provided arguments (cli arguments that weren't determined to be flags or subcommands) and pass them to your parse_args method so that you can parse them how you see fit and determine if the args provided by the user are enough and to be able to raise if data is not provided correctly/in the right format.

This gives the framework a way to allow you to extend the parser in your own custom way to allow for a "custom" format to be processed. However, it's very strict and you MUST properly mark the arguments as processed so that the mainloop won't double-process arguments passed to your custom parser. However, this won't happen as in the Flag(T) I am doing a check to ensure that args were processed after passing it to your code.

CliGen::Arg

This object serves as a wrapper around ARGV objects/strings/items and is used to keep track of argument processing to ensure that no argument is double-processed/re-processed and enforces this by the internal raise if the Arg#processed is called multiple times (mostly did it for MY sanity as doing offset math of ARGV was causing me anxiety so this was the only acceptible method in my opinion on a possibly MASSIVE set of args that could be provided via the CLI).

Planned Features

Markdown Documentation Generation

Since all command metadata is present in annotations at compile time (@[CliGen::CommandInfo], @[CliGen::SubCommand], @[CliGen::Argument]), the framework can walk the same structures that generate.cr already walks and render them into a Markdown document instead of a CommandNode tree.

The generation would be driven by a macro finished block (similar to generate.cr) that emits a self.generate_docs class method on App. This method walks every Command subclass and its annotations to produce a structured document.

Proposed output structure:

# <AppName>

## Commands

### mycommand
<CommandInfo description>

#### Help
<CommandNode help output>

#### Flags
| Flag           | Type  | Default | Description |
|----------------|-------|---------|-------------|
| -m,--myvar VAR | Int32 | 23      | ...         |

#### Subcommands
- `do_thing` — <description>
  - Examples: ...

Implementation notes:

• Driven by a --generate-docs global flag or a dedicated class method (if the utility was compiled with the INCLUDE_DOC=true ENV VAR set)
• ECR templates (already pulled in) are the natural rendering mechanism
• The same annotation data powers both runtime help output and the doc generator, keeping them in sync automatically

Bash Autocompletion Generation

Since all command names and flag names are known at compile time, a complete bash completion script can be generated statically — no runtime --completions endpoint needed.

The approach is a hidden --generate-completion bash flag (potentially extended to zsh/fish later) that prints a ready-to-install completion script to stdout.

Proposed completion script shape:

_myapp() {
  local cur="${COMP_WORDS[COMP_CWORD]}"
  local prev="${COMP_WORDS[COMP_CWORD-1]}"

  case "${COMP_WORDS[1]}" in
    mycommand)
      COMPREPLY=($(compgen -W "--myvar -m --format -f" -- "$cur"))
      ;;
    *)
      COMPREPLY=($(compgen -W "mycommand myothercommand" -- "$cur"))
      ;;
  esac
}
complete -F _myapp myapp

Implementation notes:

• Script body generated at compile time via a macro finished walk of Command.subclasses
• @[CliGen::Argument] options (%w[json yaml ecr]) can be included as valid completions for their flag
• Install path: myapp --generate-completion bash > ~/.bash_completion.d/myapp or printed with instructions
• Same annotation data used by the doc generator, so both stay in sync with the command definition

Converter Objects

Currently a custom type is wired in by reopening it and extending CliGen::Coercable (or CliGen::Parsable). That works, but it requires owning the type — you cannot use Flag(SomeShardType) without monkey-patching another shard. A converter object, modelled on JSON::Serializable's converter:, moves the conversion logic outside the type.

abstract class CliGen::Converter(T)
  abstract def convert(raw : String) : T
  abstract def to_s(value : T) : String
end

class PointConverter < CliGen::Converter(Point)
  def convert(raw : String) : Point
    a, b = raw.split("x", 2)
    Point.new(a.to_i, b.to_i)
  end

  def to_s(value : Point) : String
    "#{value.x}x#{value.y}"
  end
end

argument(origin : Point, description: "origin", converter: PointConverter)

Implementation notes:

• Threading is the usual four hops: converter: on argument/add_global_flag → validated in check_flag_vars → @[CliGen::Argument(converter:)] → generate.cr → Flag.new
• The converter/flag type match is checkable at compile time — PointConverter.ancestors contains the instantiated CliGen::Converter(Point), so a mismatched converter is rejected with a real message rather than failing inside array construction
• Converter(T)#convert has a declared return type, which Coercable#coerce does not — this turns a class of user error into a definition-site compile error
• Being bidirectional, a converter supplies the rendering half too, so the T.class.has_method? :to_s constraint in Flag(T)#initialize can be dropped for converted types
• A subclass instance (rather than a module) can carry state — a date format, a locale, a delimiter — and is what makes the generic parameter available for the ancestor check above
• This does not replace CliGen::Parsable, which has a different shape (Array(Arg) -> T plus the mark-something-processed invariant). Converter is the String -> T path; Parsable stays the multi-arg path. An explicit converter should take precedence over an implicit Coercable
• Open question for Array(T): whether converter: means a Converter(T) applied per-element after the delimiter split, or a Converter(Array(T)) that splits itself. Per-element composes with delimiter: and format: and is more reusable; the ancestor check can distinguish the two, so both could be supported
• Payoff beyond the feature itself: flag.cr has three separate compile-time chains over T (process, coerce, and the array-element branches). A converter short-circuits all three with a single if conv = @converter. It is also a route to Enum support — a macro-generated EnumConverter(T) calling T.parse(raw), with options derived from T.names, would land enums without touching those dispatch chains

JSON-RPC like execution

This is a LATE (post v1.0) feature. It takes the same format and instead of a CLI argument

{
  "method": "myclass.mysubcommand",
  "params": {
    "myvar": 3,
    "format": "json"
  }
}

Essentially, this would (instead of tying directly into a command execution module would essentially wrap around App.process and provide them as Array(String) args to App.process(args : Array(String)) to allow the entire app execution to happen.

The json snippet above would essentially be the same as

App.process(["--myvar", "3", "--format" , "json", "myclass", "mysubcommand"])

However, the design is bound to change as the complexity alone to make the command execution be "similar but different" as any responses would like need to be returned back in JSON-RPC style back to the calling process.

Alongside to generating the JSON-RPC style documenatation for a calling process to know what methods are defined and what type of params are accessible and what types they accept.

This is a LONG TERM goal as it would be pretty neat to implement.