Update and reorganize the XOAI dependencies under local_lib

[poikilotherm]

Currently, Dataverse codebase uses a custom patched XOAI 4.1.0, provided in /local_lib.

It would be a good idea to

1. Find out what patches have been applied (needs anatomization of patched JAR sources with original sources)
2. Fork https://github.com/DSpace/xoai (which is the moved repo from https://github.com/lyncode) to @gdcc
3. Find out if our patches are still needed
4. As development in https://github.com/DSpace/xoai has stalled (see e.g. Several potential NullPointerException bugs. DSpace/xoai#72 (comment)), make it our package
5. Update LOTS of dependencies and especially get rid of log4j-1.2!
6. Setup Github Workflow, Code Coverage etc.
7. Make releases to Maven Central
8. Use & test in Dataverse upstream code

It might be worth a try to 1) not rename the packages but still publish under our @gdcc Maven group id and 2) create fork and a pull request for upstream plus setting up wei/pull to auto-create pull requests when DSpace updates their branch.

[landreev]

I am removing the log4j dependency that's being brought in by XOAI as part of https://github.com/IQSS/dataverse-security/issues/48; (I only found this a couple of days ago myself; about to make a PR into the main Dataverse project.)

But yes, in the process of tracing this dependency, I was reminded of this XOAI debacle, that we still have these locally-built libraries in the project, etc. (I can comment more on this;)
Once the immediate log4j issue is addressed, we can use this issue to address cleaning up the XOAI. I was going to open one myself, but thanks for beating me to it.

[landreev]

(Once we merge the pom.xml PR that removes the log4j jar from the war file, I'm going to update the title of this issue to reflect the fact that it is mainly about refactoring these XOAI dependencies, and not specifically about log4j. I will add some information on these XOAI libraries in the meantime).

[landreev]

As development in https://github.com/DSpace/xoai has stalled (see e.g. DSpace/xoai#72 (comment)), make it our package

Yes, the situation is back to where it was 6 years ago, when lyncode still nominally owned the package; but it wasn't being actively maintained and it was not possible to submit and merge a PR. We really liked the package, but it was impossible to use it without a few minor fixes. So that's what forced us to build those jars locally in the first place. We appear to have missed the window when DSpace was somewhat actively maintaining the package. And then these libraries got further stuck in our local_lib limbo. So we definitely need to assume a more active role there in order to clean up this dependency. But we will need to decide what exactly "mak[ing] it our package" should entail.

Bad news:

1. I'm seeing no trace of the local fork of the old lyncode repo from which those jars were built, anywhere.

Better news:

1. We still have all the sources (and they are included under local_lib, as source jars, I believe).
2. I will recreate that fork somewhere on github; either under IQSS/, or as my own, as the first step.
3. The changes were really minimal.
4. It's a fairly simple package to start with.

It might be worth a try to ... not rename the packages but still publish under our https://github.com/gdcc Maven group id and

That's an interesting idea. I never even considered that - publishing somebody else's packages? But that appears to be exactly what they did with reload4j - forked from log4j-1.2.17, released with the package pathnames unchanged, but under their own group id ... (assuming that's what you meant/had in mind above - ?). (I guess it's a slightly different situation with reload4j - since log4j-1 was officially retired/EOL-ed; and the new incarnation was made by the original author of log4j).

OK, that part we can figure out as we go. But we can definitely proceed with the other parts. Create a fork of the latest DSpace xoai-4 under gdcc, patch/fix what's necessary to get it to work with Dataverse, make pull requests, see how that goes etc.

[poikilotherm]

@landreev please take a look at https://github.com/gdcc/sword2-server for an example of what I did to release our own version to Maven Central.

This was a fork of what DANS did (which also was a fork of the original source) to push necessary changes from us, plus some more stuff to modernize and adapt for dependency updates. And it still has the old Java package layout, but being released under our Maven group ID. 🤠

[poikilotherm]

@landreev I started moving stuff around in https://github.com/gdcc/xoai

[landreev]

Added "feature: Harvesting" label; to make sure the issue is added to the pile of Harvesting issues we are prioritizing now as part of the GREI effort.

[mreekie]

sprint

• This came out of the spike in the previous sprint.

• This is a cleanup/maintenance issue. Came to attention during log4j security issue.

• This is a code dependecy for DV. We need to put this somewhere we can maintain it and control it.

• a lot of work has been done on this.

• medium. It's close to being able to build DV with the updated library version. However harvesting is a complicated subsystem and so everything will need to be verified and this will be labor consuming.