CLI Principles
CLI Principles Section 10 Draft

Compatibility and testing

Source 10-compatibility-testing.md

Back to the spec index

  • Black-box and language agnostic. A conformance check runs a CLI as a program from the outside: it passes parameters, feeds stdin, and looks at stdout, stderr, the exit code, and files. It never looks at source code. Any implementation in any language can be checked the same way.
  • Golden tests for -help, describe, and every JSON shape.
  • Conformance checks: exit codes, query commands never write, stdout is valid JSON in JSON mode, json and yaml carry equal data, text equals md apart from marks and colour, mutations are idempotent (section 1).
  • Lint CLIs against this spec, possibly with an LLM-based checker as Arcjet did.