Mypy doesn't handle namedtuples with dynamic fields

[agentydragon]

Given:

A = ['x']
T = namedtuple('T', A)

We get: error: List literal expected as the second argument to namedtuple(). Mypy should be able to handle this somehow, e.g., by allowing any operation on T instances if the second argument to namedtuple is not a list literal.

[JukkaL]

It would be reasonable to fall back to a "namedtuple with unknown item names" in a case like above in a weak type checking mode. I'm still unsure about how this weak mode could be enabled for particular files.

In the normal type checking mode we'd probably still want to give a warning about the above code.

[o11c]

I actually think the long-term correct approach will be to track the values of constants.

[elazarg]

@JukkaL shouldn't there be basic constant propagation, just before the semantic analysis? It will be most helpful, with ints and strings. And it shouldn't do anything tricky - no fixpoint or anything; just a single pass. At least it can be another optional feature.

[JukkaL]

Yeah, it seems that this should be possible for ints, strings and booleans and maybe tuples and lists, at least within a single module. So these could be okay:

x = ['x']
X = namedtuple('X', x)
y = 'a b c'
Y = namedtuple('Y', y)

We know with certainty the values of x and y, because nothing can change them after the initialization.

However, this is trickier:

# a.py
y = 'a b c'

# b.py
from a import y
Y = namedtuple('Y', y)   # but maybe another module has modified y?

And what about this:

from m import f
y = 'a b c'
f()  # hmm could this modify y if it imports the current module?
Y = namedtuple('Y', y)

Constant propagation could work somewhat like this:

• On each assignment, keep track of the constant value of a variable, if the right hand side is a constant.
• If we encounter a destructive operation, change (some) constants back to variables. Examples:
  • Referring to a variable using global should mark a variable as not constant.
  • The second assignment to a constant marks it as a variable (except perhaps if this replaces the original value on all code paths).
  • Calling a user-defined function should likely mark at least all currently known constants that refer to mutable objects such as lists as not constants.
• Outside a module mark everything in the module as not constant, because we can't easily know whether another module might have touched something.

Before spending more time on this I'd like to see more use cases where this would be useful. Named tuples are clearly a potential example, but that is a pretty narrow use case.

[elazarg]

Tracking types is a form of constant propagation, and the questions you raise apply to class names, function definitions etc. We already assume the code behaves nicely.

(If I understand your description correctly, you want the join operation on different constants to result in Top. That's the sensible thing to do; but maybe there's a place for a plugin here too?).

[gvanrossum]

An example like the OP's here is usually a proxy for a more dynamic source of field names, which won't work even if we implement constant propagation.

[elazarg]

I used code similar to the example just to separate the fields part, which is very noisy, from the inheriting class, without introducing a temporary type name.

[sjaensch]

Note that this error is also triggered when passing a tuple instead of a list literal:

T = namedtuple('T', ('a', 'b'))

Is this within the scope of this issue or should I open a separate one?

[elazarg]

This has been fixed in #2262, in the repo version. It is unrelated to this issue.

[gvanrossum]

This just never will happen (the dynamic version).