Explore eliminating POSIX_SHLEX and STRIP_QUOTES_FOR_NON_POSIX

[kmvanbrunt]

cmd2 currently defaults to non-POSIX parsing with shlex and strips quotes before passing args to functions.

The basic difference between POSIX and non-POSIX parsing rules in shlex deals with how quotes are processed. See https://docs.python.org/3/library/shlex.html#parsing-rules for more details.

Previously it was decided to default to non-POSIX since the quoting rules are more intuitive.
From a maintenance perspective, it would be nice to have 1 parsing behavior for our shell and not try to emulate multiple shells. This is especially true when trying to parse tokens for tab-completion.

Are you OK if I remove these flags and preserve the default behavior?

Thoughts? @tleonhardt @kotfu

[tleonhardt]

I am happy with the default behavior and would be fine with simplifying maintenance by removing these options. If we do, just make sure to flag it as a potentially breaking change in the CHANGELOG.

I don't know if anyone is using these to change the default behavior.

[rowdyrotifer]

Sorry if this isn't directly related, but is this non-POSIX option why backslashes are not recognized? This seems to cause path completion to fail for directories with spaces.

[tleonhardt]

@MarkLalor
Hey Mark, @kmvanbrunt has been putting a TON of work recently into dramatically improved tab-completion of paths with spaces and other things like that. His changes are in the new_quoted_completion branch and I suspect he will create a PR for this in another week or so.

He could do a better job than I could of explaining exactly why it doesn't work in the current incarnation ...

[kmvanbrunt]

@MarkLalor
That isn't the exact reason we currently don't tab complete strings with spaces.
However, in my current branch where I am adding support for spaces and quotes in completion, I need to settle on 1 way to parse. The non-POSIX rules of just using quotes as opposed to escaped space characters is a lot more intuitive to me.

So it will work like this. If you know the item you are completing has a space, then you can start your string with a quote (single or double is fine), and the completion will do the rest.

Also, I've added support to dynamically add an opening quote if the completed text has a space. Closing quotes are added when the item completes (unless its a directory).

[rowdyrotifer]

@kmvanbrunt That sounds awesome, the way you've described it makes it sound nice and easy to use, just like normal POSIX commands, with the benefit of the intuitive quotes. Thanks for explaining. I think it's a great area to improve the package.

[kmvanbrunt]

@MarkLalor
Thanks, I hope the final product is helpful.

[kotfu]

Been working on the ply branch to replace pyparsing and agree that supporting multiple options here is kind of a pain.

Here's my ideal scenario:

• user types in their input
• cmd2 parses the input, stripping comments, looking for pipe redirection, quoted strings, etc
• cmd2 calls the appropriate do_command method, passing a ParsedString object (cmd2 already does this) which:
  • looks like a string containing the arguments. This maintains backwards compatibility with cmd. Comments have been stripped, as have any redirector and pipe clauses.
  • has a .command method which contains the name of the command
  • has a .arglist method which gives you a list of arguments just like sys.argv does. In this list, quoted arguments are a single element in the list, and the quotes have been stripped.
  • has a .rawinput method which gives you exactly what the user typed: comments, redirectors, and everything

This would replace (or diminish the need for) the @with_arglist decorator.

I think that removing POSIX_SHLEX option and having it always be False, and removing the STRIP_QUOTES_FOR_NON_POSIX option and having it always be True gets me exactly what I want.

[kotfu]

I'm going to add this to the ply branch, which replaces pyparsing with shlex (#37).