Convert to a package directory structure

[tleonhardt]

cmd2 has grown to the point where the cmd2.py file is over 4000 lines and with new features being added the overall code size is likely to grow substantially.

It has probably reached the point that cmd2 would benefit by becoming a real Python package with multiple source code files.

For this to happen, we will need a cmd2 directory in which all of the source code lives and there is a dunder init.py file to make it a package. This article on Structuring Your Project has good basic advice on how to approach this.

[tleonhardt]

It is debatable how the package structure should be broken up, but with the new AutoCompleter feature being added I potentially see this breakdown of the source code within the cmd2 directory:

• __init__.py
  • Perhaps empty just to make it a package
• cmd2.py
  • Mostly as it exists now with a little bit of readline related stuff moved to rl_utils.py
• autocompleter.py (or alternatively auto_completer.py)
  • New code to support autocompletion of argparse argument names and values
• rl_utils.py
  • Code which supports cross-platform readline usage regardless of underlying library
  • Used by both cmd2.py and autocompleter.py and doesn't depend on either of those

I'm sure there are other ways to break it down, but that is a potential start.

[kotfu]

I like this idea a lot. If we head this direction, I'd recommend we put the parsing classes in a separate file too. They are already separate classes in cmd2.py.

[tleonhardt]

@kotfu Good call on separating out the parsing classes.

I'm not very familiar with making Python packages, are you?

What will the implications be for setup.py and the unit tests, for example?

[kotfu]

There seems to be no standard best practice for this, but here's how I've done it:

cmd2
| cmd2
| | __init__.py
| | cmd2.py
| tests
| |  __init__.py
| | test_cmd2.py
| | test_completion.py
| setup.py
| pytest.ini
| tox.ini

So inside the cmd2 source code there are two packages, one called cmd2 and another called tests. A few minor tweaks to pytest.ini and setup.py and we are in business.

[tleonhardt]

@kotfu I am hesitant to ask you to do anything which will distract you from the shlex work, but if this is something you think you could knock out in a few minutes, would you like to take a crack at it?

[kotfu]

On it.

[anselor]

Everyone that's using Cmd2 is probably doing:
from Cmd2 import Cmd

We probably need something like this in init.py:
from Cmd2 import Cmd

That way when they import from the new Cmd2 directory it'll provide Cmd in the location they expect.

Otherwise everything will break for everyone when we release the next version.

[anselor]

Ok, just looked at your branch and looks like you're already on top of this.

[kotfu]

The package branch is mostly working now. There are lots of functions and static variables that aren't part of a class, so the __init__.py file is pretty messy, but that's probably OK. I haven't tried building a package, and this would need lots more testing before merging, but it's a start. I guess we need to decide if we really wanna do this now; if so, let's get this finished up and merge it.

[tleonhardt]

I think now is a good time to convert to a multi-file package.

The 0.9.0 release is going to be a major release with huge backwards incompatible changes. So I feel like we might was well just rip that bandaid off now. Given that now is when we have given up supporting Python 2.7 in interests of supporting enhanced ability to refactor and add new features, it feels like an appropriate time. And I think as we add features to cmd2 it would eventually become inevitable that we would want to split it into multiple files.

Also @anselor looks to be nearly done with the AutoCompleter features which adds over another 800 lines of source.

And @kotfu if you manage to get done with the changes where shlex replaces pyparsing, that will be THE biggest change in cmd2's history.