The stages are read by Core from the database, and associated libraries copied to disk in the temporary directory (./pipeline/{stage}). When a library jar is replaced (i.e. a library with the same id added) with one that has a different file name, the new jar will be copied into the same directory as the old one. The old will not be removed, causing the directory to contain two jars.
When starting the stage, all jars in the directory will be added to the classpath. This can cause some undefined behaviour dependent on the order that library jars are loaded and how classes get resolved.
Maybe the best approach in the short term is to remove all files from the temp directory before writing the new library jar.
See:
Stage classpath construction: https://github.com/Findwise/Hydra/blob/master/core/src/main/java/com/findwise/hydra/StageRunner.java#L242
StageRunner creation:
https://github.com/Findwise/Hydra/blob/master/core/src/main/java/com/findwise/hydra/NodeMaster.java#L132
The stages are read by Core from the database, and associated libraries copied to disk in the temporary directory (
./pipeline/{stage}). When a library jar is replaced (i.e. a library with the same id added) with one that has a different file name, the new jar will be copied into the same directory as the old one. The old will not be removed, causing the directory to contain two jars.When starting the stage, all jars in the directory will be added to the classpath. This can cause some undefined behaviour dependent on the order that library jars are loaded and how classes get resolved.
Maybe the best approach in the short term is to remove all files from the temp directory before writing the new library jar.
See:
Stage classpath construction: https://github.com/Findwise/Hydra/blob/master/core/src/main/java/com/findwise/hydra/StageRunner.java#L242
StageRunner creation:
https://github.com/Findwise/Hydra/blob/master/core/src/main/java/com/findwise/hydra/NodeMaster.java#L132