legate.py steals command-line arguments from target applications

[piyueh]

Problem

Lunching jacobi.py using legate with the flag --num does not change the size of the matrix A in the Jacobi example.

Steps to reproduce

1. Install both legate.core and legate.numpy
2. Go to the example folder of legate.numpy
3. Run the jacobi.py code with legate with legate --cpus 1 --omps 1 --ompthreads 1 ./jacobi.py --num 1234

Expected and actual results

The log message should say "Generating 1234x1234 system...", However, the actual message says "Generating 100x100 system..."

Diagnosis

When running the example code of the Jacobi solver, the jacobi.py allows a command-line argument called --num. However, if launching jacobi.py using legate, legate steals the value of --num, saves it to numamem, and never actually passes it to jacobi.py.

This is because, by default, an ArgumentParser allows implicit abbreviations for long arguments. When executing legate <some flags> ./jacobi.py --num 1234, the argument parser of legate.py (line 549-807 here) sees --num as a abbreviation of the flag --numamem.

Solution

A quick solution is to disable the implicit abbreviation by changing line 549 in legate.py from

parser = argparse.ArgumentParser(description="Legate Driver.")

to

parser = argparse.ArgumentParser(description="Legate Driver.", allow_abbrev=False)

But I believe eventually there should be a better solution to separate the arguments of legate.py from a target application. Otherwise, if a target application has some overlapped arguments with legate.py (i.e., the same argument names), this conflict can not be resolved by simply disabling allow_abbrev.

[manopapad]

Thanks for triaging this issue! I followed your suggestion and disabled abbreviations in the launcher script.

The practice of having each layer consume the arguments it recognizes is common across the Legate/Legion/Realm codebases, but it's not as big of a problem with other libraries, because most of their arguments have a library-specific prefix (e.g. -ll:show_rsrv). We could do the same with arguments accepted by the Legate launcher, or for the launcher in particular we could require that all launcher arguments (--cpus, --gpus etc.) come before the executable.

I'll leave this issue open for other people to chime in.

[marcinz]

I think that we could change to the convention:

legate [legate flags] application [application flags]

This would be consistent with what everyone is used to from MPI launchers. In that convention, the arguments would be split by position and not by filtering.

[lightsighter]

I don't think I want to enforce that formulation unless we have to. I 'm happy with the @manopapad solution to just prevent abbreviations from parsing.

[manopapad]

Another argument for separating arguments per layer is so we can recognize typos in the arguments meant for a specific layer. If arguments can appear in any order then the best a layer can do is consume the arguments it recognizes exactly, and assume any unrecognized arguments (even if they look suspiciously like something it accepts, e.g.. --proofile) will be consumed by another layer.

[piyueh]

From the viewpoint of an application developer, preventing abbreviation is not enough. Some commonly seen flags/arguments, such as --verbose and --help, may exist at the same time in legate launcher and an application. For example, many examples in legate.numpy/examples support the flag --verbose for detailed output. None of them is working when using legate launcher because legate consumes that flag first.