PyScript revamp

[anselor]

Use reflection to present a Python API for all argparse commands inside of the pyscript script. For non-argparse commands, allow commands to be called with a string similar to through the application itself.

[tleonhardt]

I am intrigued by this. This sounds like a great idea.

[anselor]

@tleonhardt It's still just an idea at the moment. I'm going to play with the idea a bit and see how user-friendly it is.

[anselor]

I have an initial stab at what I'm thinking with the revamp of embedded python scripting. You can see it in the pyscript branch.

You can try it out with the subcommand and tab completion examples.

Run the example of your choice, and enter the interactive python shell (py)
Everything is accessible through app or cmd
Every top-level command is accessible as a function on app or cmd. Each layer of sub-command can be called as a function on the return value of the previous command function. Each of the arguments or flags can be called as function arguments.

For example, if you try tab_autocompletion.py each of these is a valid way to execute a command:
app('media movies list')
app.media().movies().list()
app.media.movies.list()
app('media movies list -a "Mark Hamill"')
app.media().movies().list(actor="Mark Hamill")
app.media.movies.list(actor="Mark Hamill")

Previously the embedded python had to acess self._last_result to get the result of a command. In the new incarnation self._last_result will be returned from the function.

@tleonhardt @kmvanbrunt

[anselor]

I ran into an interesting problem testing this set of changes. If I trigger argparse help output it kicks me out of the interactive python shell. It seems when argparse sees -h/--help it prints help and then calls sys.exit. When I do this from the interactive python shell it ends the shell and returns me back to the cmd2 prompt.
Somehow in cmd2 we're preventing the application from exiting and instead just loops around to the prompt again. I'd like to replicate this behavior to prevent argparse help from ending the interactive python shell. Do you have any ideas?

To reproduce:
run an example
py
app.help('set')

@kotfu @tleonhardt

[tleonhardt]

cmd2.Cmd._in_py is a boolean instance variable which is used to keep track of whether or not cmd2 is currently executing within either the py or pyscript command.

Currently I believe it is just used by the cmd2.Cmd.ppaged method so that it never attempts to display output using a pager when running inside an embedded Python interpreter or interactive Python script.

But the overall intention is to use _in_py wherever it makes sense.

[tleonhardt]

Possibly of relevance, I've thought for awhile now that we may need to effectively override the default manner in which commands using one of the two cmd2 argparse decorators display their help.

Currently this is purely delegated to argparse. And there is a cheap hack within cmd2.Cmd.do_help to temporarily redirect both sys.stdout and sys.stderr while argparse is doing its thing for displaying the help.

Previously when we had the optparse decorator, there was a class which overrode the default optparse function for displaying help. I think we might need to do something similar for argparse.

[anselor]

@tleonhardt
I'm just adding a couple things to the python session inside of do_py() so any state tracking we're doing will remain the same as before.
As for a fully customizing the argparse help output - that's a surprisingly complicated piece of code. I managed to figure out how to hijack some of the behavior to improve some of the formatting but to fully replace it would be a significant effort.

[tleonhardt]

@anselor
Yeah, the reason I didn't already add the code to fully customize the argparse help output is because I realized that it was going to be exponentially more difficult than it was for optparse. So I figured that what we have is "good enough for now" and put the idea on the back burner. I didn't think about the possibility of someone trying to get help within an interactive py session. I guess my response would be "don't do that".

[kotfu]

I'm digging deep into my memory for this one, but I think argparse calls sys.exit() after it prints the help when it parses a -h option. sys.exit() raises a SystemExit exception. If you catch that exception and don't let it propagate up to the python interpreter, you can prevent the program from exiting.

[kotfu]

@tleonhardt You may also find the argparser.format_help() function useful, which returns the formatted from the argparser, but doesn't do any of the other system exit stuff or try and output the help anywhere.