E126 on `with` statement line continuation

[darkfeline]

E126 (continuation line over-indented for hanging indent) is reported on what I think should be correct formatting on line continuation in multiline with statements:

with open('foo') as foo, \
     open('bar') as bar, \
     open('baz') as baz:
    pass

PEP8 doesn't specify this explicitly, but I believe this is a perfectly reasonable and natural way to indent such a statement. Also, IDLE automatically indents this the same way.

[sigmavirus24]

PEP8 does specify how hanging-indents should be treated and pep8.py does exactly what the document says it should.

Also, IDLE automatically indents this the same way.

Editor behaviour is not what defines the rules in the document or the behaviour of this tool.

[darkfeline]

Actually, upon careful reading of PEP8, it says that "The 4-space rule is optional for continuation lines" and "Hanging indents may be indented to other than 4 spaces".

The point I was trying to make was that, for example with while statements, PEP8 specifically allows for aligning with the opening delimiter:

while (foo == foo and
       bar == bar and
       baz == baz):
    pass

with statements cannot take parentheses for implicit line continuation, so the only option, given pep8 (this program) as it is now, is a double indent, which looks ugly:

with open('foo') as foo, \
        open('bar') as bar:
    pass

or simulate a "hanging indent", as described in PEP8 (see the footnote on the PEP8 page) (even uglier):

with \
        open('foo') as foo, \
        open('bar') as bar:
    pass

What I was requesting is simulating the alignment with the opening delimiter for with statements:

with open('foo') as foo, \
     open('bar') as bar:
    pass

However, your post made me go back and re-examine PEP8, and I think that pep8's checking of continuation lines should be even looser, because according to PEP8, any level of indentation is acceptable, as long as it isn't indistinguishable from the indentation of a following suite:

if (foo == foo and
    foo == bar):  # same as below; bad
    pass

Futhermore, while I agree that editor behaviour is not prescriptive, IDLE is officially bundled with Python and its auto-indentation otherwise follows PEP8 accurately, so I believe its behaviour should be given some weight.

[sigmavirus24]

@darkfeline the section you're referring to says:

Use 4 spaces per indentation level.

And further the initial examples all show that pep8's current behaviour is correct. It says you may indent to other indentation levels but that doesn't mean the former examples are to be thrown out. How we interpret that means you are free to use --ignore=E126 to indent amounts that are not a multiple of 4 or visual.

Your examples that use opening delimiters (namely parentheses) work because they are allowed to use delimiters like that. A with_statement has no such allowance in the grammar of the language. Further if it did your statement would look like:

with (open('file0') as file0,
      open('file1') as file1):
    pass

Which means that to simulate that behaviour you would actually want to indent like so:

with open('file0') as file0, \
      open('file1') as file1:
    pass

That would be visual if the grammar allowed for parentheses, but it doesn't so that is a moot argument. In the absence of that, it says to indent a multiple of 4. Whether IDLE is officially bundled with Python or not is not a concern and how it treats other indentation is not either.

As I see it you have a few options:

• Fork pep8 to allow your behaviour and use your fork
• Indent a multiple of 4 spaces
• Ignore E127

[darkfeline]

Further if it did your statement would look like:

with (open('file0') as file0,
      open('file1') as file1):
    pass

Which means that to simulate that behaviour you would actually want to indent like so:

with open('file0') as file0, \
      open('file1') as file1:
    pass

I'm a little surprised by how you don't see that the point is that the two opens are aligned in the first example and not in the second. I actually want to indent like so:

with open('file0') as file0, \
     open('file1') as file1:
    pass

Notice the aligned expressions (the open function calls in this example), this is visually pleasing.

And further the initial examples all show that pep8's current behaviour is correct. It says you may indent to other indentation levels but that doesn't mean the former examples are to be thrown out.

In that case shouldn't E126 instead be a W1 warning? W191 (indentation with tabs) is a warning, while it is specifically banned by PEP8. On the flip side, non-4-space continuation lines is explicitly allowed by PEP8 (though, as you say, the examples all follow 4-space indents), yet it is reported as an error.

I can see why you would want to keep pep8's current behavior, but I think that if pep8 isn't going to ignore it (which I still think it should), it should at least be reduced to a warning for consistency's sake.

I would like one more chance to convince you that removing E126 is consistent with the PEP8 style guide.

Specifically, PEP8 says the following:

Use 4 spaces per indentation level.

Note that this applies to indentation levels (for suites), and not continuation lines, which is the subject of E126. Continuation lines are described thusly.

Continuation lines should align wrapped elements either vertically using Python's implicit line joining inside parentheses, brackets and braces, or using a hanging indent [5]. When using a hanging indent the following considerations should be applied; there should be no arguments on the first line and further indentation should be used to clearly distinguish itself as a continuation line.

Here we see that there are two options:

1. vertical alignment using implicit line joining in parentheses
2. hanging indent

The first doesn't apply in this situation, so let's look at the second. PEP8 describes a hanging ident as indenting the lines after the first line further than the first line to distinguish it from the first line, and specifically states that the 4-space indentation rule does not apply to continuation lines. Again recall that continuation lines are not the same as regular indentation of code in suites.

Thus, the only error PEP8 should report for the indentation of continuation lines is if the continuation line is indented the same amount as the first line, and any other style "errors", if they are kept in pep8, should be reduced to warnings at most.

[sigmavirus24]

I'm a little surprised by how you don't see that the point is that the two opens are aligned in the first example and not in the second.

I do understand that. I'm telling you I disagree. I could feign surprise but I won't.

this is visually pleasing

That argument doesn't fly. Many people, for example, think the following is visually pleasing:

varone    = 1
vartwo    = 2
longervar = 3

But that violates PEP8.

I can see why you would want to keep pep8's current behavior, but I think that if pep8 isn't going to ignore it (which I still think it should), it should at least be reduced to a warning for consistency's sake.

I agree that because of the wording of the document it should be a warning but there are problems with that:

1. The document was worded that way when this check (error) was created.
2. Many people [1] rely on the error and warning codes not changing between non major versions (e.g., 1.5 and 1.6).

So while it would be nice to change this to a warning, we cannot. Or at least, we can't do that until version 2.

specifically states that the 4-space indentation rule does not apply to continuation lines.

Again, were only the document immutable, then this wouldn't be an issue because when the check was written this would have been accounted for.

[1]: OpenStack, NASA, and many other extremely large projects

[darkfeline]

@sigmavirus24
I see, that is very reasonable. Shall this bug be kept open and marked for if/when the next major version of pep8 is released?

[sigmavirus24]

@darkfeline I can't close issues so the answer is maybe. That's entirely up to @florentx

[akitada]

PEP 8 was updated and now it has a multiline with-statement example
that is visually aligned with the first item.

So, I think @darkfeline's request is a reasonable and E126 should not be reported on those code.

[sigmavirus24]

@akitada submit a pull request then and we'll see how it is received.

[akitada]

@sigmavirus24 How about just adding E126 to DEFAULT_IGNORE?
I think this issue is similar to #256 and @florentx is proposing to make E121 ignored by default.
Why not add E126 and others reported on PEP 8 examples to thet list?
Any concerns?

[sigmavirus24]

@akitada because E126 is also responsible for checking hanging indents like these if I remember correctly, so it is still valid.

[akitada]

@sigmavirus24 I ran pep8.py over those code but E126 didn't pop up.

$ cat no.py
# Arguments on first line forbidden when not using vertical alignment.
foo = long_function_name(var_one, var_two,
    var_three, var_four)

# Further indentation required as indentation is not distinguishable.
def long_function_name(
    var_one, var_two, var_three,
    var_four):
    print(var_one)
$ python pep8.py no.py
no.py:3:5: E128 continuation line under-indented for visual indent
no.py:6:1: E302 expected 2 blank lines, found 1
no.py:8:5: E125 continuation line with same indent as next logical line

I found another issue with E126, #167 and I think @methane's assertion is correct.