Modified a few things:
- Finished implementing relative date parsing and migrated date parsing from Flag(T) to it's own dedicated module. Additionally added a the ability to do "recursive" relative operations for time searching. - Additionally made a Struct & Enum for containing relative operations around dates. - Added additional checks in App & CommandNode(T) around checking env_vars of flags & checking for duplicate commands - Added a CommandMeta record to contain the classname of the CommandNode(T) - Seperated out all of the monolitic files (command_node.cr & file.cr). Moved the base classes and records into their own files in the dir with the same name of their generic counterparts. - Fixed a number of macro related bugs around Command.argument & CliGen.add_global_flag - My ass hurts from sitting here for hours and doing these changes and arguing with claude over what needs to be done. Fun tho.
This commit is contained in:
@@ -0,0 +1,501 @@
|
||||
# Crystal Cli Generator
|
||||
|
||||
**Author:** Tristan Ancelet
|
||||
**Email:** tristanancelet@yahoo.com
|
||||
|
||||
## Overview
|
||||
|
||||
This document outlines the overall design of the CliGen shard & it's underlying classes/objects & their usecases.
|
||||
|
||||
## How it works/High-Level overview
|
||||
|
||||
Using crystal macros, you define the shape (arguments/flags, selections, work functions/subcommands, etc) and later on in `src/cligen/app/generate.cr` will use macros to (at compile time) generate `CommandNode(T)` objects & `Flag(T)` objects to contain your command/subcommand/arg parsing code from the data you provided in your `CliGen::Command` subclass.
|
||||
|
||||
|
||||
## Architecture
|
||||
|
||||
The desire for this project is to provide a framework for generating commandline arg-parses & command dispatch built into a class/object.
|
||||
|
||||
The idea is to (much like `JSON::Serializable` & `YAML::Serializable`) is to use macros to help defining a Command object and abstract the command-line away from the codebase needing input from the user at init time.
|
||||
|
||||
|
||||
### Command Objects
|
||||
|
||||
The core of this codebase is the `CliGen::Command` object.
|
||||
|
||||
This is the object that is meant to provide you the hook into being able to utilize the codebase. It is where you are able to define all of your expecetd arguments/flags & any selectables (aka, instance variable that you want the user to choose one value for. `ex: -f|--format json|yaml|ecr`).
|
||||
|
||||
To make this all possible we load the base class with macros that help define your variables & inform the framework how it needs to handle arguments being presented to your command object setters.
|
||||
|
||||
```crystal
|
||||
require "cligen"
|
||||
|
||||
module MyModule
|
||||
@[CliGen::CommandInfo(description: "My command that does cool things")]
|
||||
class MyCommand < CliGen::Command
|
||||
argument(myvar : Int32 = 23,
|
||||
long: "--myvar VAR",
|
||||
short: "-m",
|
||||
description: "This tells the utility how many times to do thing",
|
||||
validate: ->(v : Int32) : Bool { (1..23).includes?(v) }
|
||||
)
|
||||
|
||||
argument(output_format : String = "ecr",
|
||||
long: "--format FORMAT",
|
||||
short: "-f",
|
||||
description: "Inform the utility what output you want the data in",
|
||||
options: %w[ json yaml ecr ]
|
||||
)
|
||||
|
||||
DO_THING_EXAMPLES = [
|
||||
"myutil mycommand do_thing --myvar 5"
|
||||
]
|
||||
|
||||
subcommand do_thing, description: "Do the THING", examples: MyModule::MyCommand::DO_THING_EXAMPLES do
|
||||
|
||||
output = 0
|
||||
|
||||
@myvar.times do |i|
|
||||
puts "thing done %i times" % [ i + 1 ]
|
||||
output += i
|
||||
end
|
||||
|
||||
puts "Total : %i" % output
|
||||
end
|
||||
end
|
||||
|
||||
CliGen::App.process
|
||||
end
|
||||
```
|
||||
|
||||
The examples like above provide a "DSL-esk" way of defining:
|
||||
- Instance Variables,
|
||||
- Short/Long flags
|
||||
- Description of the flags (used in the help output as well)
|
||||
- A verification proc/lambda for doing ad-hoc checks of the value provided by the user (essentially allowing you to implement your own option: key like in selection)
|
||||
- Selections (currently compile-time and will open it up to runtime collecting of options later on based on defined annotations in the class)
|
||||
- define subcommands of this current command
|
||||
|
||||
|
||||
#### CliGen::CommandNode(T)
|
||||
|
||||
This is the "CliGen" representation of your command. Using crystal macros & generics when the CommandNode is created it will generate the internal Generic methods based on the defined command object and whether it has subcommands & main method, etc.
|
||||
|
||||
This is where all of the actual argument processing gets done in the `CliGen::CommandNode(T)#process(args : Array(CliGen::Arg))` method.
|
||||
|
||||
A main-loop of scoped parsing happens in it's `#process` method and if a child-command is detected `CliGen::Command -> MyModule::MyCommand -> MyModule::MyOtherCommand` it will gather all unprocessed arguments and will call that `CommandNode(T)`'s `#process` method.
|
||||
|
||||
This was done because it effectively keeps arguments scoped in their own contexts without causing something like OptionParser implementation having flag/argument bleed between commands and subcommands if multiple parsers are defined (aka OptionParser does not `#shift` arguments off as they are processed, which end up causing arguments to bleed into child instances).
|
||||
|
||||
Alongside it's main duty of handling CLI parsing of arguments it also serves to handle any framework related raises from `Flag(T)` (invalid values, incorrect custom data class behavior, internal errors, etc).
|
||||
|
||||
|
||||
#### CliGen::App
|
||||
|
||||
This object is the root application that handles all initial execution from bare CLI arguments. It's just a subclass of `CommandNode(Nil)` (Subclassed with `Nil` for simplicity and to not deal with code duplication between macro generated `CommandNode(T)` and a custom `App` object (with them needing almost exactly the same methods).
|
||||
|
||||
For the majority of the process, this object just serves as the inital entrypoint to the CLI parsing and handles raises of any Framework related error classes (help output, exceptions related directly with the `CommandNode(T)` objects)
|
||||
|
||||
|
||||
#### CliGen::Flag(T)
|
||||
|
||||
This is the object that handles option/argument parsing. It's a Generic Container who's internal layout changes depending on the type it's initalized with.
|
||||
|
||||
The only (currently as of writing this doc) types that are supported are:
|
||||
- All Int permutations (Int8, UInt8, etc)
|
||||
- All Float permutations (Float32, etc)
|
||||
- String
|
||||
- Bool
|
||||
- Enum (*planned, but not implemented yet*)
|
||||
- Time (strict formatting allowed)
|
||||
- Array(T)
|
||||
|
||||
|
||||
##### Time Limitations
|
||||
|
||||
The framework has some strict formatting for time arguments
|
||||
|
||||
It only accepts data in the following static formats:
|
||||
- `%Y-%m-%d %H:%M:%S %z` (ex: `2026-08-30 19:00:00 +0000`)
|
||||
* This specifies the date, time & timezone/offset (allowing you to customize the time)
|
||||
|
||||
- `%Y-%m-%d %H:%M:%S` (ex: `2026-08-30 19:00:00`)
|
||||
* This specifies the date & time and will default to the local timezone of the device it's being run on (via `::Time.parse_local`) (would need much more conditionals but it gets the point accross)
|
||||
|
||||
- `%Y-%m-%d %z` (ex: `2026-08-30 +0000`)
|
||||
* This specifies the date & timezone/offset, and will default to 00:00:00 for the time value
|
||||
|
||||
- `%Y-%m-%d` (ex: `2026-08-30`)
|
||||
* This specifies the date, and will default to 00:00:00 for the time value and to the local timezone of the host (via `::Time.parse_local`)
|
||||
|
||||
- `@%s %z` (ex: `@1788122911 +0900`)
|
||||
* This specifies the epoch time and the timezone you want the final `Time` object to be in (this gets the provided epoch time as UTC via `::Time.unix` and then setting the timezone to the provided offset)
|
||||
|
||||
- `@%s`
|
||||
* This specifies the epoch time and via `::Time.unix` will default to `::Time::Location::UTC` for the timzone
|
||||
|
||||
However, minimal support for relative time has been added. Valid formats for this are:
|
||||
- `[-+][0-9]+ (years|months|weeks|days|hours|minutes|seconds) %z` (ex: `+2 hours -0900`)
|
||||
* This provides a relative time to look for and what timezone offset to convert to after the relative time is calculated via `::Time#in`
|
||||
|
||||
- `[-+][0-9]+ (years|months|weeks|days|hours|minutes|seconds)` (ex: `-2 hours`)
|
||||
* This provides a relative time to look for and will default to the local timezone via `::Time.local`
|
||||
|
||||
- `[-+]%H:[-+]?%M:[-+]?%S %z` (ex: `-2:3:3 -0900`) (*planned but not currently implemented*)
|
||||
* This provides a hour, minute and second to decrease/increase by (depending on each field's sign) and a timezone to convert to afterwards with `::Time#in`
|
||||
|
||||
- `[-+]%H:[-+]?%M %z` (ex: `-2:3 -0900`) (*planned but not currently implemented*)
|
||||
* This provides a hour and minute to decrease/increase by (depending on each field's sign) and a timezone to convert to afterwards with `::Time#in`
|
||||
|
||||
- `[-+]%H:[-+]?%M:[-+]?%S` (ex: `-2:3:3`) (*planned but not currently implemented*)
|
||||
* This provides a hour, minute and second to decrease/increase by (depending on each field's sign)
|
||||
|
||||
**Note:** Multiple human-readable operations
|
||||
|
||||
The relative operations for getting time relatively to the current time on the running system `+1 days` can be chained and the specific operation order is important when providing your ops.
|
||||
|
||||
So when doing multiple you can provide them like so
|
||||
```
|
||||
mycli mycmd --time '+2 weeks -1 month +30 days -0900`
|
||||
```
|
||||
|
||||
This essentially grabs the local time (via `::Time.local`), setting it's timezone to the `-0900` offset, and then adding 2 weeks/14 days, jumping back a full month (dependant on the actual month) and then going forward 30 days.
|
||||
|
||||
|
||||
**Note:** Relative operations short format (*planned but not implemented yet*)
|
||||
For the `[-+]%H:[-+]?%M:[-+]?%S` formats if the sign's (`+-`) aren't provided for the `%M` or `%S` fields it will default to the sign provided to the `%H` field
|
||||
|
||||
|
||||
##### Array(T) limitations
|
||||
|
||||
However, with Array(T) types there are a few limitations. The only supported in-place T-types are the core classes (Int*, Float*, String). However, provided you do the below extentions to your custom data class you can also use it as a data class with Arrays.
|
||||
|
||||
The framework (deliberately) limits the `:options` key to an array of individual choices that the user can provide instead of permutations of valid `Array(T)` values. So when providing an `Array(T)`
|
||||
|
||||
```crystal
|
||||
require "cligen"
|
||||
|
||||
module MyModule
|
||||
@[CliGen::CommandInfo(description: "Test Command")]
|
||||
class MyCommand < CliGen::Command
|
||||
argument(sources : Array(String),
|
||||
description: "The source of all your pain",
|
||||
options: %w[ tacos golang coffee ]
|
||||
)
|
||||
|
||||
def main
|
||||
@sources.each_with_index do |source, index|
|
||||
puts "%d : %s" % [index + 1, source]
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
CliGen::App.process
|
||||
```
|
||||
|
||||
Providing it in the array format will have the framework collapse it into just the Array(T) values so that they can be value-checked (`Array(T)#includes?(val)`)
|
||||
|
||||
Of which when an invalid value is provided the framework will abort directly if an invalid value is provided
|
||||
```
|
||||
(ins) tristan@arcanetome $: crystal run test4.cr -- mycommand --sources golang coffee pain
|
||||
crystal-run-test4.tmp: 'pain' is not a valid value for --sources (valid: ["tacos", "golang", "coffee"])
|
||||
```
|
||||
|
||||
However when run with valid values it has no issues
|
||||
```
|
||||
(ins) tristan@arcanetome $: crystal run test4.cr -- mycommand --sources golang coffee
|
||||
1 : golang
|
||||
2 : coffee
|
||||
```
|
||||
|
||||
|
||||
##### Custom Data Types
|
||||
|
||||
However, there is an API defined via the `CliGen::Parsable` and `CliGen::Coercable` modules that will allow you to customize parsing of a Custom DataType.
|
||||
|
||||
This is limited to parsing raw strings (of any format) into whatever your data is supposed to represent
|
||||
|
||||
```crystal
|
||||
module MyModule
|
||||
class MyData
|
||||
extend CliGen::Parsable
|
||||
extend CliGen::Coercable
|
||||
|
||||
@value : Int32
|
||||
|
||||
def initialize(value : String)
|
||||
@value = value.to_i32
|
||||
end
|
||||
|
||||
def self.parse_args(args : Array(CliGen::Arg))
|
||||
arg = args.first
|
||||
# Mark the argument as processed
|
||||
arg.processed
|
||||
new(arg.value)
|
||||
end
|
||||
|
||||
def self.coerce(arg : String)
|
||||
new(arg)
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
##### Override #to_s for CLI help readablility
|
||||
|
||||
If you do plan to use your custom data-type with the you will need to override the `#to_s(io : IO)` method to ensure that the help output can render it correctly
|
||||
|
||||
Example of a Custom DataType use in a `CliGen::Command` subclass argument/flag:
|
||||
```crystal
|
||||
require "cligen"
|
||||
|
||||
module MyModule
|
||||
class MyData
|
||||
extend CliGen::Coercable
|
||||
extend CliGen::Parsable
|
||||
|
||||
@@instances = {} of String => MyData
|
||||
|
||||
getter name : String
|
||||
getter work : Proc(Nil)
|
||||
|
||||
def initialize(@name, &@work)
|
||||
unless @@instances.has_key?(@name)
|
||||
@@instances[@name] = self
|
||||
else
|
||||
raise "HUH?"
|
||||
end
|
||||
end
|
||||
|
||||
def to_s(io : IO)
|
||||
io << @name
|
||||
end
|
||||
|
||||
def self.get(name : String)
|
||||
@@instances[name]
|
||||
end
|
||||
|
||||
def self.parse_args(args : Array(CliGen::Arg))
|
||||
arg = args.first
|
||||
arg.processed
|
||||
get(arg.value)
|
||||
end
|
||||
|
||||
def self.coerce(arg : String)
|
||||
get(arg)
|
||||
end
|
||||
|
||||
def self.all
|
||||
@@instances.values
|
||||
end
|
||||
end
|
||||
|
||||
MyData.new("test") do
|
||||
puts "Did a test"
|
||||
end
|
||||
|
||||
MyData.new("really") do
|
||||
puts "Yes really"
|
||||
end
|
||||
|
||||
MyData.new("why") do
|
||||
puts "Just because"
|
||||
end
|
||||
|
||||
@[CliGen::CommandInfo(description: "Test Command")]
|
||||
class MyCommand < CliGen::Command
|
||||
argument(source : MyModule::MyData,
|
||||
description: "The source of all your pain",
|
||||
options: MyModule::MyData.all
|
||||
)
|
||||
|
||||
def main
|
||||
@source.work.call()
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
CliGen::App.process
|
||||
```
|
||||
|
||||
Providing the overridden `#to_s` method makes the help output render your objects in a readable way
|
||||
|
||||
```
|
||||
(ins) tristan@arcanetome $: crystal run test2.cr -- mycommand -h
|
||||
Command: mycommand
|
||||
Description: Test Command
|
||||
|
||||
|
||||
Flags:
|
||||
---------------------------------------------------------------
|
||||
--source The source of all your pain (valid: test, really, why)
|
||||
|
||||
|
||||
Global Flags
|
||||
---------------------------------------------------------------
|
||||
-v,--verbose Enable verbose output from program & help output (default: false)
|
||||
|
||||
|
||||
|
||||
Note: When wanting help output of any command you can provide the -h/--help flags or help command
|
||||
|
||||
(ins) tristan@arcanetome $: crystal run test2.cr -- mycommand --source why
|
||||
Just because
|
||||
```
|
||||
|
||||
Without the `#to_s` method overridden you'll get the crystal representation of your object instead (like below)
|
||||
```
|
||||
(ins) tristan@arcanetome $: crystal run test2.cr -- mycommand -h
|
||||
Command: mycommand
|
||||
Description: Test Command
|
||||
|
||||
Flags:
|
||||
---------------------------------------------------------------
|
||||
--source The source of all your pain (valid: #<MyModule::MyData:0x7f683ad0cc90>, #<MyModule::MyData:0x7f683ad0cc60>, #<MyModule::MyData:0x7f683ad0cc30>)
|
||||
|
||||
|
||||
Global Flags
|
||||
---------------------------------------------------------------
|
||||
-v,--verbose Enable verbose output from program & help output (default: false)
|
||||
|
||||
|
||||
|
||||
Note: When wanting help output of any command you can provide the -h/--help flags or help command
|
||||
```
|
||||
|
||||
Sill valid, but if you plan to make a registry based command/work/strategy selector you might want to override the `#to_s` method
|
||||
|
||||
|
||||
##### CliGen::Coercable
|
||||
|
||||
```crystal
|
||||
def self.coerce(arg : String)
|
||||
new(arg)
|
||||
end
|
||||
```
|
||||
|
||||
The coerce method provides you the ability to parse a single string value into your class/datatype. This is generally only used when parsing from ENV VAR and when being used by parsing from an `Array(T)` type.
|
||||
|
||||
This should only be used in the case you need a simple datatype that can be learned from a single string.
|
||||
|
||||
|
||||
##### CliGen::Parsable
|
||||
|
||||
```crystal
|
||||
def self.parse_args(args : Array(CliGen::Arg))
|
||||
arg = args.first
|
||||
# Mark the argument as processed (required)
|
||||
arg.processed
|
||||
new(arg.value)
|
||||
end
|
||||
```
|
||||
|
||||
This method is used the most and is used for when using the bare class as the generic type in the `Flag(T)`. With this the `Flag(T)` will collect all provided arguments (cli arguments that weren't determined to be flags or subcommands) and pass them to your parse_args method so that you can parse them how you see fit and determine if the args provided by the user are enough and to be able to raise if data is not provided correctly/in the right format.
|
||||
|
||||
This gives the framework a way to allow you to extend the parser in your own custom way to allow for a "custom" format to be processed. However, it's very strict and you MUST properly mark the arguments as processed so that the mainloop won't double-process arguments passed to your custom parser. However, this won't happen as in the Flag(T) I am doing a check to ensure that args were processed after passing it to your code.
|
||||
|
||||
|
||||
#### CliGen::Arg
|
||||
|
||||
This object serves as a wrapper around ARGV objects/strings/items and is used to keep track of argument processing to ensure that no argument is double-processed/re-processed and enforces this by the internal raise if the `Arg#processed` is called multiple times (mostly did it for MY sanity as doing offset math of ARGV was causing me anxiety so this was the only acceptible method in my opinion on a possibly MASSIVE set of args that could be provided via the CLI).
|
||||
|
||||
|
||||
## Planned Features
|
||||
|
||||
### Markdown Documentation Generation
|
||||
|
||||
Since all command metadata is present in annotations at compile time (`@[CliGen::CommandInfo]`, `@[CliGen::SubCommand]`, `@[CliGen::Argument]`, `@[CliGen::Selection]`), the framework can walk the same structures that `generate.cr` already walks and render them into a Markdown document instead of a `CommandNode` tree.
|
||||
|
||||
The generation would be driven by a `macro finished` block (similar to `generate.cr`) that emits a `self.generate_docs` class method on `App`. This method walks every `Command` subclass and its annotations to produce a structured document.
|
||||
|
||||
Proposed output structure:
|
||||
|
||||
```
|
||||
# <AppName>
|
||||
|
||||
## Commands
|
||||
|
||||
### mycommand
|
||||
<CommandInfo description>
|
||||
|
||||
#### Help
|
||||
<CommandNode help output>
|
||||
|
||||
#### Flags
|
||||
| Flag | Type | Default | Description |
|
||||
|----------------|-------|---------|-------------|
|
||||
| -m,--myvar VAR | Int32 | 23 | ... |
|
||||
|
||||
#### Subcommands
|
||||
- `do_thing` — <description>
|
||||
- Examples: ...
|
||||
```
|
||||
|
||||
Implementation notes:
|
||||
|
||||
* Driven by a `--generate-docs` global flag or a dedicated class method (if the utility was compiled with the INCLUDE_DOC=true ENV VAR set)
|
||||
* ECR templates (already pulled in) are the natural rendering mechanism
|
||||
* The same annotation data powers both runtime help output and the doc generator, keeping them in sync automatically
|
||||
|
||||
### Bash Autocompletion Generation
|
||||
|
||||
Since all command names and flag names are known at compile time, a complete bash completion script can be generated statically — no runtime `--completions` endpoint needed.
|
||||
|
||||
The approach is a hidden `--generate-completion bash` flag (potentially extended to `zsh`/`fish` later) that prints a ready-to-install completion script to stdout.
|
||||
|
||||
Proposed completion script shape:
|
||||
|
||||
```bash
|
||||
_myapp() {
|
||||
local cur="${COMP_WORDS[COMP_CWORD]}"
|
||||
local prev="${COMP_WORDS[COMP_CWORD-1]}"
|
||||
|
||||
case "${COMP_WORDS[1]}" in
|
||||
mycommand)
|
||||
COMPREPLY=($(compgen -W "--myvar -m --format -f" -- "$cur"))
|
||||
;;
|
||||
*)
|
||||
COMPREPLY=($(compgen -W "mycommand myothercommand" -- "$cur"))
|
||||
;;
|
||||
esac
|
||||
}
|
||||
complete -F _myapp myapp
|
||||
```
|
||||
|
||||
Implementation notes:
|
||||
|
||||
* Script body generated at compile time via a `macro finished` walk of `Command.subclasses`
|
||||
* `@[CliGen::Selection]` options (`%w[json yaml ecr]`) can be included as valid completions for their flag
|
||||
* Install path: `myapp --generate-completion bash > ~/.bash_completion.d/myapp` or printed with instructions
|
||||
* Same annotation data used by the doc generator, so both stay in sync with the command definition
|
||||
|
||||
|
||||
|
||||
### JSON-RPC like execution
|
||||
|
||||
This is a LATE (post v1.0) feature. It takes the same format and instead of a CLI argument
|
||||
|
||||
```json
|
||||
{
|
||||
"method": "myclass.mysubcommand",
|
||||
"params": {
|
||||
"myvar": 3,
|
||||
"format": "json"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Essentially, this would (instead of tying directly into a command execution module would essentially wrap around App.process and provide them as Array(String) args to App.process(args : Array(String)) to allow the entire app execution to happen.
|
||||
|
||||
The json snippet above would essentially be the same as
|
||||
|
||||
```crystal
|
||||
App.process(["--myvar", "3", "--format" , "json", "myclass", "mysubcommand"])
|
||||
```
|
||||
|
||||
However, the design is bound to change as the complexity alone to make the command execution be "similar but different" as any responses would like need to be returned back in JSON-RPC style back to the calling process.
|
||||
|
||||
Alongside to generating the JSON-RPC style documenatation for a calling process to know what methods are defined and what type of params are accessible and what types they accept.
|
||||
|
||||
This is a LONG TERM goal as it would be pretty neat to implement.
|
||||
Reference in New Issue
Block a user