PyAML device identification (name uniqueness?)

[JeanLucPons]

Hello,

There were already several discussions around the way to identity a device in PyAML and in pyAT.
It would be nice to find an agreement as it will impact the implementation.
Today the identification is done using the couple type,name (i.e. Quadrupole("QF1")).
The main advantage is to be able to use a unique name for all multipoles of a combined function magnet, ease its configuration and be compatible with FamName attribute of an AT Multipole element. FamName AT attribute gives also access to AT element selection function.

The limitation is that, at the configuration level, the type has to be specified for array definitions.

type: pyaml.arrays.hcorrector  # Can contains only HCorrector
name: HCORR
elements:
   - SH1A-C01-H
   - SH1A-C02-H

type: pyaml.arrays.element
name: ARBITRARY
elements:
  - HCorrector(SH1A-C01-H)
  - BPM(BPM-C01-2)
  - RFCavity(CAV6)

There are of course alternatives to make PyAML device names unique.
There are also proposal to allow external modules to manage the PyAML device identification and link to pyAT elements.
Please make proposal if you disagree with the above, so we can find an agreement in order to minimize modification of existing code.
Keep in mind that PyAML is supposed to work on control system without necessarily internal simulator configured.

Sum up (can change):

• Having unique element name in PyAML
• The PyAML element name maps to lattice element name (the python attribute containing the element name is configurable, by default FamName)
• When lattice element name is not unique:
  • An auto increment is added when lattice is loaded allowing specific configurations
  • If the PyAML element name is the name, for a magnet, the strength is applied proportionally to the length of each sub items (split or serialized magnet)

[TeresiaOlsson]

I think there should be an unique name. In MML there are three different ways (family, element), (family, [cell, element]) and common name (including utility functions to translate between them). For the first two you always need to specify two things. It's annoying and at HZB we therefore use the common name which is an unique identifier since we have a naming convention which supports that. But at least the (family, [cell, element]) convention I think will also be required since many labs currently use that and only knows their devices by that naming convention.

In MML the names between the MML "devices" and the lattice are completely separate. At the point where you want to add a simulator to your configuration, there is a special part to set up the mapping that you need to configure. I think that is a more flexible approach than relying on pyAML devices having the same name as in the lattice file but maybe there could be some default behaviour for those who are lucky and have the situation where the naming is the same.

[JeanLucPons]

If we want a complete separation between AT name and PyAML name. we need to be clear on how to configure the mapping and how to handle combined function magnet without necessarily separated multipoles including serialized one (i.e. Sextupole with serialized main current and correction coils). In case of non separated multipole, we need also to ensure that the design allow computations of hardware setpoint in a serialized manner to avoid calculation conflicts.

[TeresiaOlsson]

At HZB we like the functionality in pyAT where you can find devices using wildcards. We want to have that in pyAML too since we use it a lot. I guess that will only work if we have unique identifiers?

[kparasch]

From a user's point of view, I think it would be much more convenient to not have to know the type to access a magnet, especially if we start having external types, e.g. complicated sextupoles/id devices. This would imply that the pyaml name is unique. I am ok if we end up deciding that the unique id is (type,name) though.

DESY lattices:
PetraIII and IV have FODO-like straight sections between achromat arcs and so the MML [cell, element] way of accessing things becomes very awkward, even though PetraIII was eventually configured for MML (but to my understanding this was not an easy thing to do).

• Petra III uses CommonName with unique names in the AT lattice. (Dipoles are in series with back-leg windings for orbit correction. All dipoles and all windings have unique names)
• Petra IV at the moment uses FamName with non-unique names in the AT lattice. At present, there is no unique identifier apart from the element index. However, we plan to change to have unique names in the future. E.g. each individual component of the combined function sextupoles will have unique names for the control system (h. corr., v. corr., skew quad., sextupole), but they may not be necessarily found in the AT lattice as that will likely have only the name of the combined function sextupole.

I think we should be able to specify the attribute (CommonName, FamName, index) of the simulator when we want to link to a model.

[JeanLucPons]

In any case a unique identifier is required.
What is index in your case ? The index [A|B,idx]of the polynomial ? The index of the element ?

[kparasch]

The index of the element in the AT lattice ring[index]. In the AT model, the individual components don't have unique identifiers. For the control system we generate unique names but they are not linked to the AT model.

[JeanLucPons]

We already discussed the problem of identifying with the element index and this does not seem to be a good idea as you may want to dynamically add a Marker, split an element, ... in your lattice. At least it is possible.
The fact that FamName is not unique can be handle by an additional index concatenated during the loading phase as previously discussed.

[TeresiaOlsson]

I think this is a good illustration of the differences between existing/design machine. We have the same situation for BESSY II and BESSY III. I think pyAML needs to be able to support both situations since we want to also use it for simulated commissioning and at that point there is likely no unique names in the lattice file.

From a user's point of view, I think it would be much more convenient to not have to know the type to access a magnet, especially if we start having external types, e.g. complicated sextupoles/id devices. This would imply that the pyaml name is unique. I am ok if we end up deciding that the unique id is (type,name) though.

DESY lattices: PetraIII and IV have FODO-like straight sections between achromat arcs and so the MML [cell, element] way of accessing things becomes very awkward, even though PetraIII was eventually configured for MML (but to my understanding this was not an easy thing to do).

* Petra III uses CommonName with unique names in the AT lattice. (Dipoles are in series with back-leg windings for orbit correction. All dipoles and all windings have unique names)

* Petra IV at the moment uses FamName with non-unique names in the AT lattice. At present, there is no unique identifier apart from the element index. However, we plan to change to have unique names in the future. E.g. each individual component of the combined function sextupoles will have unique names for the control system (h. corr., v. corr., skew quad., sextupole), but they may not be necessarily found in the AT lattice as that will likely have only the name of the combined function sextupole.

I think we should be able to specify the attribute (CommonName, FamName, index) of the simulator when we want to link to a model.

[kparasch]

I agree, identifying by index is a nightmare. That's why will change the lattice in the future (within the next 5 years :) ) to have unique names, whether it is in the FamName, CommonName, or any other attribute.

Using an additional index for each non-unique FamName is an even bigger nightmare for us. The first three magnets of the first cell in each arc is different than the rest of the cells. For example, QF8[0] belongs in the first cell, QF8[1] in the second but QF1[0] belongs in the second cell.

[TeresiaOlsson]

One option for the mapping is to not add any automatic part but instead make the user implement this themselves. Potentially following the CATAP idea where jinja is used to generate a starting point and the user needs to fill in the missing code. I would say this is the way MML does it.

It is a bit annoying and means more work when configuring the simulator but it also gives full flexibility. So pros/cons :/

[JeanLucPons]

An other proposal could be also to define an additional structured python attribute for an AT element.
Including:

• available functions for combined function magnet with unique name,
• family name,
• serialization information (a family is not necessarily linked to a serialized main current)
• ...

[kparasch]

I think it is a viable solution but I am not a big fan of it. This way, what is originally the task of the person configuring pyaml for their accelerator, now becomes the problem of the person producing the lattice file, otherwise you end up with different lattice models, produced by different people that need to be maintained.

The work to provide all this information somewhere has to be done in any way. I would prefer to write/read yaml files than having to open and modify a binary file.

[JeanLucPons]

The person who produces the lattice file should also inform what a Multipole AT element is supposed to perform.
He/she should also tell which magnets are serialized or not.
When you get a lattice file that you don't know, these information are always crucially missing.
We can imagine to have these information in a separate file and add a UniqueName field in AT.

[simoneliuzzo]

Unique names could be a requirement for pyAML. In any case we need to do it at some point in any facility.
I would then keep the proposal of Jean-Luc but assuming that the field that we refer to in the model lattice is not FamName but a new field named for example UniqueName.

Also a yaml file listing the UniqueNames would work for me: Every line is a lattice element; A line with multiple unique names is a CFM magnet.

[TeresiaOlsson]

Why do you want to add a field to the lattice? I thought the lattice would just be loaded into pyAML (using whatever format is best when people no longer have .mat files) into internal data models where the field could just be name and from those data models a pyAT lattice object is generated with unique names and that's the one used for simulations?

This is ok for me. The user gives a lattice and pyAML is assigning the UniqueNames as additional fields to the lattice used internally, following a convention as the ones shown above by @JeanLucPons and @kparasch . So the user is not obliged to write the unique name and pyAML can decide/use a uniform internal convention.

I think this will also make it easier to add abstraction for other simulation codes later if someone wants it. Then pyAML has internally a clear standard which is independent of the format of the lattice file.

One option could be that this standard is automatically generated if the user does nothing to automatically be able to handle easy cases. But then there is an option to overwrite that behaviour for cases when it doesn't work. For example:

• I have unique names in the lattice file and I want to use the same names in pyAML: mapping is automatic.
• There is a few elements where the names I want to use in pyAML don't exist in the lattice file, for example corrector and skews on a sextupole: I need to overwrite the default behaviour and add a mapping myself.
• I don't have unique names in the lattice file but just family names: pyAML automatically adds the unique names as QF_1, QF_2 etc and then in the pyAML config I need to say which one to map to.

[JeanLucPons]

Thanks, i edited the first post to try to sum up the discussion.

[GamelinAl]

I agree with the conclusion reached so far (as sum up by @JeanLucPons).

I would just add that we can keep the idea of an optional user-defined mapping as suggested by @TeresiaOlsson as last possibility.
This was more or less proposed by @gupichon by structuring this part of the code as modules in the other discussion:

To improve this, I propose delegating the linking logic between PyAML elements (referenced by name) and PyAT elements to a set of dedicated modules. A default implementation would reproduce the current behavior for backward compatibility, while additional strategies could allow:

• Linking by a user-defined attribute name (which could also be a custom property).

The module to use (including custom implementations defined outside pyaml, if needed) would be specified in the simulator section of the configuration.

This modular approach would make the mapping mechanism more flexible and better aligned with different use cases and naming conventions.

[TeresiaOlsson]

I like the suggestion from @gupichon. Then if we discover during the testing that there are many labs where the default doesn't work we already have the architecture to add some customization possibility.

[JeanLucPons]

Custom element linker will be used to associate a PyAML name (or other field, if i correctly understood the implementation proposed by @gupichon) to a set of lattice elements.