Allow blocking or explicitly selecting PV sources

[smarsching]

At the moment, the choice of data source implementations for certain protocols is hard-coded. For example, the org.phoebus.pv.ca.JCA_PVFactory will always be registered for PVs with the type ca.

I am in the process of preparing a custom product of Phoebus, and due to having experienced issues with JCA in the past, we want to use EPICS Jackie instead of JCA. Some time ago, we did the same for a custom product of CSS. There, this was easy because the relevant bundle could simply be excluded from the built and be replaced with a different bundle.

In Phoebus, however, the CA support that is based on JCA is part of the core-pv module. So, in order to replace JCA_PVFactory with a different implementation we would have to maintain a fork of core-pv, which is bothersome in the long run.

I see three ways how this could be avoided:

1. The contents of the org.phoebus.pv.ca package could be moved from the core-pv to a separate module (e.g. core-pv-ca).
2. A block_pv_sources preference could be added to the org.phoebus.pv.PVPool class. This preference would specify a comma-separated list of classes that should be ignored when discovering PVFactory implementations.
3. A pv_sources_map preference could be added to the org.phoebus.pv.PVPool class. This prerence would specify a comma-separated list of type to class mappings (e.g. ca:com.example.MySource,pva:com.example.OtherSource) which would override the discovery mechanism for the specified types.

Option 1 upsides:

• Clear separation of interface and implementation.
• Code which is not used does not have to be distributed with a product. Neither do their dependencies.

Option 1 downsides:

• In order to make this truly modular and to allow the distributor to choose which data sources shall be provided, this would also have to be done for all the other data sources that are part of the core-pv module. So, several new modules would have to be added.
• The build logic of existing products would have to be modified to explicitly depend on the desired modules because unlike the core-pv module, this modules would not be pulled in as transitive dependencies. However, I think that this problem could be alleviated by adding another module that depends on all these new modules and adding this new module as a dependency of all modules that currently only depend on core-pv. Distributors wanting to exclude certain PV sources could exclude this dependency in their build configuration and add explicit dependencies to the desired modules instead.

Option 2 upsides:

• Very simple, localized change. Only touches a few lines of code in PVPool.
• Blocked data-source classes will not have to be loaded at all, so dependencies of these classes do not have to be included in the product.

Option 2 downsides:

• Every data source that shall not be used has to be listed explicitly. This configuration has to be adjusted when the class name of the blocked data source changes for some reason.

Option 3 upsides:

• Data sources can be overriden by type instead of the class name. This means that renaming the class providing the default implementation for a certain type will not effect existing product configurations and one does not have to no the class name providing the data source for a type that one would like to override.

Option 3 downsides:

• Data-source classes have to be loaded in order to determine their type. This means that dependencies for these classes must be present as well, even if the class is not used in the end. This could be resolved by combining the mechanism with the one from option 2.

I am happy to provide a PR that makes the necessary changes. However, I want to discuss which of the suggested solutions is preferred first (or maybe there even is a solution that I have not considered yet).

I have a preference for option 1, because from a software design perspective it is the cleanest one. However, I am aware that it also is the most invasive one. If such an invasive change shall be avoided, option 2 is probably the best one (possibly with option 3 added as an additional feature).

[smarsching]

I forgot to mention: If option 1 is chosen, I would be happy to supply a PR adding a core-pv-jackie module which provides the alternative data source for the ca type. If another option is chosen, this does not make a lot of sense because users would have to explicitly block the default implementation in oder to use this module, so we would probably keep it in the source repository specific to the custom product.

[kasemir]

org.phoebus.pv.ca.JCA_PVFactory will always be registered for PVs with the type ca.

want to use EPICS Jackie instead of JCA

This is similar to "pva:" using the 'core-pva' implementation, while "opva:" uses the older pvAccessJava library.
I see no problem with adding a new PV type that uses EPICS Jacky. The hardest part would be picking a prefix. Call it "ejca:"? You can make that the default, to any pv "X" will then turn into "ejca:X". I would also be fine with simply adding that to the core-pv package.

I don't see EPICS Jackie in widespread use to the point where we change "ca" to use Jackie and something like "oca" for the original CA client implementation. With pva we changed over when hardly anybody was using CS-Studio with PVAccess, but for Channel Access this would be a change with unpredictable side effects. Any additions to allow re-mapping "ca" to different implementations seems confusing and error prone. Just keeping "ca:" as is and adding a new "..ca..:" type for Jackie is straight forward and avoids any confusion.

Yes, just adding it to core-pv means that everybody gets "ca", "ejca", "pva", "opva", "mqtt", "sim", "loc", "sys", "eq" type PV support.

A separate step would be some way to select what goes into your product.
Yes, we could split core-pv into core-pv-ca, core-pv-ejca, core-pv-pva, core-pv-opva, core-pv-mqtt, core-pv-sim, core-pv-loc, core-pv-sys, core-pv-eq. At this point, I'm not sure that's really necessary. I have not had a single complaint about MQTT support being included. On the contrary, I would be concerned that some site-specific product suddenly lacks sim: or loc: PVs if we make the assembly of a product more convoluted.

[georgweiss]

My vote would go for option 1 due to the modularization aspect. Configuring a product to include and exclude as needed would in my view not be a heavy burden, and it would not be performed that often.

[smarsching]

Thanks for the quick reply.

I fully agree that the default implementation for the ca type should not be changed, as there might be subtle differences in the implementations that could cause issues for some users.

I also agree that including things like sim, loc, and even all the other implementations by default makes sense, so we rather need a mechanism that allows distributors to explicitly remove such implementations from their products than having to add them explicitly.

I am aware that we could simply add a new provider using a different type (e.g. jackie), but for our product, we want to be sure that user’s won’t accidentally use the “wrong” implementation by specifying the ca prefix somewhere. This would cause two implementations for Channel Access to be active in the same instance, which should work in theory, but still does not sound like a good idea. This is why I want some mechanism to completely disable the JCA_PVFactory in our product.

I had a closer look into how the “blocking” mechanism that I suggested could be implemented and discovered that implementing it is not as easy as I assumed: The ServiceLoader will always try to load the class before we have a chance of detecting the class name. When dependencies for the class are missing, the iterators next() method will throw a ServiceConfigurationError.

In the PVPool, we could make the call to the iterator’s next() method explicit and catch the ServiceConfigurationError, but I dislike this idea for two reasons:

First, the API description of ServiceLoader does not guarantee that after raising a ServiceConfigurationError next() can successfully be called again. The actual implementation is designed in a way that this will work, but it might not be wise to rely on that.

Second, we would either have to log a warning or error message for each ServiceConfigurationError, result in log entries for “problems” which are by design of the product configuration, or we would have to inspect the message of the ServiceConfigurationError in order to determine whether it was raised for one of the block classes. Again, this would work for the current implementation of ServiceLoader, but it would rely on even more intricate implementation details, so I would really like to avoid this.

In summary, I don’t think that the blocking mechanism that I originally suggested as option 2 can be implemented in a reasonable way.

This would leave option 3, but using this option would mean that a product would have to include dependencies that are not really needed.

So, I would like to circle back to option 1 and propse a refined version of it:

We create six new modules with the following contents:

• core-pv-ca: everything in org.phoebus.pv.ca
• core-pv-core: everything in org.phoebus.pv except the sub-packages covered by the other modules
• core-pv-mqtt: everything in org.phoebus.pv.mqtt
• core-pv-opva: everything in org.phoebus.pv.opva
• core-pv-pva: everything in org.phoebus.pv.pva
• core-pv-tango: everything in org.phoebus.pv.tga and org.phoebus.pv.tgc

The existing core-pv module would change into a meta module that depends on all these modules, but does not include any code of it own. The reason for moving the common code from core-pv into core-pv-core is that the other core-pv-* modules will obviously need a dependency on core-pv-core so that they can use the PV and PVFactory interfaces. If they depended on core-pv instead, but core-pv also depended on them, there would be a circular dependency.

This solution would leave support for the eq, loc, sim, and sys PVs in the core module. These PV sources do not have dependencies outside Phoebus anyway, so I an see no reason why anyone would want to exclude them in a custom build. The only exception is the eq source. which depends on org.epics.vtype, but dependencies on that library are so common across various Phoebus modules that building a Phoebus distribution without org.epics.vtype seems unfeasible anyway.

Any module depending on core-pv (= all existing code) would still pull in all the data sources that are currently part of core-pv, so for existing code and build configurations nothing would change. They would get exactly the same features that they have now, the only difference being that the code would be spread over six JARs instead of a single one.

However, distributors building their own product could exclude unwanted dependencies in their pom.xml. This is not quite as clean as making the dependencies explicit (the guide I linked sees exclusions as a “last resort”), but considering that this existing build configurations would not be broken, I think it is a reasonable compromise.

@kasemir What do you think, does this sound like a reasonable solution?

[kasemir]

OK, how about we do this:

core-pv gets split into a much smaller core-pv which only contains what's now in the org.phoebus.pv package: PVPool, PVFactory, PV and related interfaces and helpers like AccessRightsEventHandler, LongString, RefCountMap, PV, TimeHelper, ValueEventHandler. You mentioned "core-pv-core" as a name for that, but maybe we can keep "core-pv".

All the PV implementations move into their own modules: core-pv-ca, -formula, -loc, -mqtt, -opva, -pva, -sim, -sys, -tga, -tgc.

A new core-pv-jackie is added the same way as its own PV implementation module. It will register its own prefix, it will not use "ca:".

The default phoebus-product includes all core-pv-* implementations, but only core-pv is a hardwired dependency. All the core-pv-* implementations are added one by one in the product assembly.

Each site is thus free to pick and choose which core-pv-* implementation they want to include.
This allows you to for example include core-pv-jackie and exclude core-pv-ca. You can select "jackie:" as the default prefix, and if some display happens to specifically use "ca:" that PV will simply not work.
To most site-specific products this means they need to specifically add core-pv-ca, -formula, -loc, -pva, -sim, -sys to retain the same functionality, but that's an acknowledged one-time update to for example the site's product pom.

[smarsching]

This sounds good to me. I am going to prepare a PRs for these changes. It probably makes sense to create two separate PRs: One splitting the core-pv module and making the necessary changes and a second one adding the core-pv-jackie module using the jackie: prefix (I thought about using caj:, but this would probably lead to confusions with JCA / CAJ, and something like ca-jackie: seems too long).

[georgweiss]

Not sure about the prefix discussion... Would inclusion of core-pv-jackie imply that all PV references must be prefixed with jackie:, or would it suffice to specify it as org.phoebus.pv/default=jackie in the preferences?

[kasemir]

Including core-pv-jackie in a product means "jackie:..." PVs are now supported.
Adding org.phoebus.pv/default=jackie in the preferences means any PV "X" will default to "jackie:X".

What @smarsching wants, in my understanding, is add core-pv-jackie in their product, make "jackie:.." the default, AND exclude core-pv-ca so "ca:" would no longer be supported for them.

For the default product, I suggest supporting all PVs and keeping the default as "ca:" for the time being.
ESS might include all PV types, but make "pva:" the default.

[georgweiss]

Thanks @kasemir, that's what I understood from the discussion.

[smarsching]

@kasemir Your undstanding of what I intend to do is correct.