Multiple options before subcommands

[sh54]

I want to run a program via:

my-program --config config.edn --config other-config.edn dothis --hello world

If I saw this in the wild I would expect this to run my-program with subcommand dothis with the options {:config ["config.edn" "other-config.edn"], :hello "world"}.

However if I setup a dispatch table with the root spec resembling {:config {:coerce []}} then the bit dothis gets interpreted as another value to add to option :config rather than getting recognized as the subcommand to run.

Here is an example program. The comments have examples of dispatch calls and their actual results, along with what I would expect them to return if I think the result is incorrect.

(ns example
  (:require [babashka.cli :as cli]))

(def global-spec {:config  {:desc "Config edn file to use"
                            :coerce []}
                  :verbose {:coerce :boolean}})
(def dns-spec {})
(def dns-get-spec {})

(def table
  [{:cmds []            :fn identity :spec global-spec}
   {:cmds ["dns"]       :fn identity :spec dns-spec}
   {:cmds ["dns" "get"] :fn identity :spec dns-get-spec}])

(defn -main
  [& args]
  (prn (cli/dispatch table (vec args))))

(when (= *file* (System/getProperty "babashka.file"))
  (apply -main *command-line-args*))

(comment
  ;; Correct
  (cli/dispatch table ["--config" "config-dev.edn"])
  #_ {:dispatch [], :opts {:config ["config-dev.edn"]}, :args nil}

  ;; Correct
  (cli/dispatch table ["--config" "config-dev.edn" "--config" "other.edn"])
  #_ {:dispatch [],
      :opts {:config ["config-dev.edn" "other.edn"]},
      :args nil}

  ;; Correct
  (cli/dispatch table ["--config" "config-dev.edn" "--config" "other.edn" "--verbose"])
  #_ {:dispatch [],
      :opts {:config ["config-dev.edn" "other.edn"], :verbose true},
      :args nil}

  ;; Incorrect
  (cli/dispatch table ["--config" "config-dev.edn" "--config" "other.edn" "dns"])
  #_ {:dispatch [], :opts {:config ["config-dev.edn" "other.edn" "dns"]}, :args nil}
  ;; Expected
  #_ {:dispatch ["dns"], :opts {:config ["config-dev.edn" "other.edn"]}, :args nil}

  ;; Incorrect
  (cli/dispatch table ["--config" "config-dev.edn" "--config" "other.edn" "dns" "get"])
  #_ {:dispatch [],
      :opts {:config ["config-dev.edn" "other.edn" "dns" "get"]},
      :args nil}
  ;; Expected
  #_ {:dispatch ["dns" "get"],
      :opts {:config ["config-dev.edn" "other.edn"]},
      :args nil}

  ;; Correct - the --verbose flag seems to break up everything getting dragged into :config
  (cli/dispatch table ["--config" "config-dev.edn" "--config" "other.edn" "--verbose" "dns" "get"])
  #_ {:dispatch ["dns" "get"],
      :opts {:config ["config-dev.edn" "other.edn"], :verbose true},
      :args nil}

  ;; Incorrect
  (cli/dispatch table ["--config" "config-dev.edn" "--verbose" "--config" "other.edn" "dns" "get"])
  #_ {:dispatch [],
      :opts {:config ["config-dev.edn" "other.edn" "dns" "get"], :verbose true},
      :args nil}
  ;; Expected
  #_ {:dispatch ["dns" "get"],
      :opts {:config ["config-dev.edn" "other.edn"], :verbose true},
      :args nil}

  ;; Incorrect
  (cli/dispatch table ["--config" "config-dev.edn" "--config=other.edn" "dns" "get"])
  #_ {:dispatch [],
      :opts {:config ["config-dev.edn" "other.edn" "dns" "get"]},
      :args nil}
  ;; Expected
  #_ {:dispatch ["dns" "get"],
      :opts {:config ["config-dev.edn" "other.edn"]},
      :args nil}

  ;; Correct
  (cli/dispatch table ["dns" "get"])
  #_ {:dispatch ["dns" "get"], :opts {}, :args nil}

  )

This is tested with 0.8.60

[borkdude]

I'll reply in an hour or so

[sh54]

For some precedence here is an example from docker:

docker compose --file docker-compose.yml --file docker-compose.prod.yml up --build --detach

The compose bit gets {:file ["docker-compose.yml" "docker-compose.prod.yml"}.

The up bit gets {:build true, :detach true}.

[sh54]

Did a little fiddling with the code.

This version fixes my particular issue and handles various edge cases.

So given:

(def global-spec {:config  {:desc "Config edn file to use"
                            :coerce []}
                  :something {:coerce :keyword}
                  :verbose {:coerce :boolean}})
(def dns-spec {})
(def dns-get-spec {})

(def table
  [{:cmds []            :fn identity :spec global-spec}
   {:cmds ["dns"]       :fn identity :spec dns-spec}
   {:cmds ["dns" "get"] :fn identity :spec dns-get-spec}])

Then target case works

(cli/dispatch table ["--config" "config-dev.edn" "--config" "another.edn" "dns" "get"])
{:dispatch ["dns" "get"],
 :opts {:config ["config-dev.edn" "another.edn"]},
 :args nil}

dns is still possible to use as the value for an option.

(cli/dispatch table ["--config" "config-dev.edn" "--something" "dns" "dns" "get"])
{:dispatch ["dns" "get"],
 :opts {:config ["config-dev.edn"], :something :dns},
 :args nil}

But not if the option takes multiple values.

(cli/dispatch table ["--config" "config-dev.edn" "--config" "dns" "get"])
{:dispatch ["dns" "get"], :opts {:config ["config-dev.edn"]}, :args nil}

Unless you use the workaround --config=dns.

(cli/dispatch table ["--config" "config-dev.edn" "--config=dns" "dns" "get"])
{:dispatch ["dns" "get"], :opts {:config ["config-dev.edn" "dns"]}, :args nil}

The code changes are not lovely though. It adds an extra binding to a loop and adds some funky looking logic to the the-end? variable.

[borkdude]

I think the problem might be caused by allowing --foo 1 2 3 as multiple options under one flag. If we prohibit this (via an option) which is probably more aligned with some standards out there, then the parsing becomes simpler.

[borkdude]

As a workaround you could first parse your arguments using parse-args and then pull out the :args from that and feed that to cli/dispatch

[borkdude]

@sh54 Fixed. Do you need another release?

[sh54]

Ah fantastic! I would much appreciate a release.

[borkdude]

Published as 0.8.61

[sh54]

So right now it is running differently under clojure and babashka. Under clojure it does the new correct behavior whilst under babashka it does the old thing. It was rather surprising but I have tried it on a few machines and I get the same result. Here is a repo that reproduces this. All I can guess is that under babashka it somehow hits an old version of cli. I did a sanity check with a new install bb v1.12.195 in a fresh ubuntu vm and got the same results.

[borkdude]

@sh54 This is because babashka.cli is a built-in library. To use the library from source try this:

(require '[babashka.cli] :reload)

[sh54]

Thanks that fixes it.