PEP 558: Remove dynamic frame semantics proposal

[ncoghlan]

Changing the frame API semantics based on whether or not a
tracing function is active is tricky to implement and hard
to document clearly, so this simplifies the proposal by
instead having the frame API always expose a write-through
proxy at function scope, and restricting the dynamic
snapshot behaviour to the locals() builtin.

[ncoghlan]

@njsmith attempting to implement the dynamic frame semantics for PEP 558 convinced me that you were right and it wasn't a good idea in the first place, so this switches to the simpler design with more consistent API behaviour.

[njsmith]

The other big question for me is whether it makes more sense to keep locals() acting differently from f_locals, or to always return the proxy object no matter how you're accessing the locals.

The main arguments I see for potentially changing locals() would be

• the current semantics are pretty confusing and not particularly useful
• if we stop exposing the underlying storage dict at the python level, we could potentially skip allocating it entirely for most frames (except in very rare cases like someone writing to a key that's not in the locals array), and delete the whole LocalsToFast/FastToLocals machinery.

I think this could also address the concerns the PEP raises about the cost of allocating the proxy object. I guess right now the proposal is that references go:

proxy -> frame -> backing dict

Suppose we stop allocating the backing dict except in special cases. Then we have a free pointer field in the frame object. We could repurpose that pointer to hold a cached, lazily allocated proxy object. So a frame starts out:

frame -> NULL

and then in the rare cases where someone accesses f_locals or locals(), we allocate the proxy and cache it in the frame:

frame -> proxy

and in the even rarer case where someone writes to a proxy key that doesn't have a corresponding fast-local entry, then we need a backing store:

frame -> proxy -> backing dict

The key trick here is that since the proxy is cached, that not only saves on allocations if it's being accessed repeatedly, it also makes it ok for it to hold state, because we know that repeated accesses will hit the same proxy object.

A downside is that this would introduce a circular reference, since the proxy also needs to hold a reference to the frame. Maybe this would be fine because it would only affect frames where the locals were actually introspected, which is rare, and we have a GC? But having locals introspection delay deallocation would be user visible, and people do rely on prompt deallocation no matter how often we say that it's not guaranteed by the language. Maybe there's some way to break the reference? Or I guess we could continue to keep the backing store in the frame object (but lazily allocate it to get the speedup on most frame creation), and just use a freelist to make proxy object access cheap, if it's really a problem?

[ncoghlan]

Changing locals() would be a backwards incompatible change, so it's a complete non-starter.

That isn't at all obvious though, so I'll add a new design discussion section about that.

[njsmith]

I'm going to need more details on why you think that :-). Did you try it? What broke? Currently mutating locals() is widely documented to be a major footgun with inconsistent behavior, and all the major linters prohibit it...

…

On Tue, May 21, 2019, 05:42 Nick Coghlan ***@***.***> wrote: Changing locals() would be a backwards incompatible change, so it's a completely non-starter. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1051?email_source=notifications&email_token=AAEU42EZFQYD7WMXMZSVEJDPWPU2VA5CNFSM4HMZSAV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV3Y57Q#issuecomment-494374654>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEU42FAD4GAUBRSEDJEPBLPWPU2VANCNFSM4HMZSAVQ> .

[ncoghlan]

Folks do it all the time by calling exec() and eval() without explicit namespaces at function scope - that implicitly passes globals() and locals() to the executed code, and the presence of locals() keeps the module globals from being mutated, while the fact locals() is a dynamic snapshot keeps the function locals from being mutated.