[Java] Inclusion of a git.properties resource in Arrow JARs causes confusion in Spring Boot applications

[asfimport]

By default Spring Boot reads classpath:git.properties to get information about the Git repository of an application. However, Arrow JARs also include a resource called git.properties, and this can cause Spring Boot to read the information from one of the Arrow libraries instead of from the application. This was reported to Spring Boot as an issue, but the Spring Boot developers say that they cannot automatically distinguish the application Git properties from git.properties resources from dependencies.

Would you consider omitting git.properties from Arrow JARs in future releases or will Spring Boot users that (directly or indirectly) use Arrow need to work around this by letting git-commit-id-plugin generate the application Git properties in an alternative location and configuring Spring Boot to read the information from that alternative location (spring.info.git.location)? Of course other libraries could also cause this issue, but Arrow is the first and only library that I've encountered so far that publishes JARs with a git.properties resource in them.

It seems that the fact that Arrow JARs include git.properties resources also caused ARROW-6361.

Reporter: Nils Breunese
Assignee: David Dali Susanibar Arce / @davisusanibar

Related issues:

• [Java] sbt docker publish fails due to Arrow dependecies (is related to)

PRs and other links:

• GitHub Pull Request apache/arrow#13578

_{Note: This issue was originally created as ARROW-16825. Please see the migration documentation for further details.}

[asfimport]

Kouhei Sutou / @kou:
I'm not familiar with .jar. A .jar doesn't include .git.properties in general, right? If so, our .jar should not include .git.properties.

[asfimport]

Nils Breunese:
In my experience library .jar files indeed generally do not contain a resources called git.properties, but any .jar can contain any resource with any name in general. It's just that an application framework like Spring Boot may look for a resource called git.properties to find things like the commit ID and branch name the artifact was built from, assuming it will find the information for the application, but if libraries also include a resource with this name, then the framework might find the information for that library instead of the application.

For my own use case I've worked around this by generating the Git properties for my application under a different resource name and pointing Spring Boot at this alternative resource name, but omitting git.properties from Arrow library JARs could avoid the necessity for this for other users.

I don't know if there is a specific reason why the Arrow library JARs currently do include this resource. I guess I would in principle also think that it's nice to able to find Git metadata for an artifact in that resource.

[asfimport]

Kouhei Sutou / @kou:
@StevenMPhillips Do you remember why our .jar provide git.properties? It seems that your initial commit fa5f029#diff-d8225ebcfc11b480a6e4f54e183b67c3ead51635a167c106d928c2abf1f9ef66R233-R274 / ARROW-1 includes it.

[asfimport]

David Dali Susanibar Arce / @davisusanibar:
Hi @kou / @lidavidm /  [~breun] base on current functionality current git.properties is created at building time and added to the jar artifacts containing commit metadata information.

A proposal could be rename file that contains metadata about commit: From: git.properties To: gitarrowinfo.properties

Main reason: git.properties is a generic name that could be use for third party libraries for a different purpose than inspect arrow metadata information (i.e: SpringBoot git.properties configuration).

Current Arrow Java artifacts are using git-commit-id-maven-plugin to generate a file ./git.properties with commit metadata such as:

#Generated by Git-Commit-Id-Plugin
#Wed Jun 08 11:43:58 UTC 2022
git.build.user.email=
git.build.host=Mac-1654686060479.local
git.dirty=true
git.remote.origin.url=https\://github.com/apache/arrow
git.closest.tag.name=apache-arrow-8.0.0
git.commit.id.describe-short=apache-arrow-3.0.0-3327-dirty
git.commit.user.email=kou@clear-code.com
git.commit.time=08.06.2022 @ 07\:51\:03 UTC
git.commit.message.full=ARROW-16785\: [Packaging][Linux] Add FindThrift.cmake (\#13337)\n\nThis is a follow-up of ARROW-16721/\#13292.\n\nAuthored-by\: Sutou Kouhei <kou@clear-code.com>\nSigned-off-by\: KrisztiSz<szucs.krisztian@gmail.com>
git.build.version=9.0.0.dev191
git.commit.message.short=ARROW-16785\: [Packaging][Linux] Add FindThrift.cmake (\#13337)
git.commit.id.abbrev=5fe3a60
git.branch=5fe3a606f16b183cf706ae8d9a4adaff593c18a1
git.build.user.name=
git.closest.tag.commit.count=11694
git.commit.id.describe=apache-arrow-3.0.0-3327-g5fe3a60-dirty
git.commit.id=5fe3a606f16b183cf706ae8d9a4adaff593c18a1
git.tags=
git.build.time=08.06.2022 @ 11\:43\:58 UTC
git.commit.user.name=Sutou Kouhei

 

Artifacts affected: 

find . -name '*.jar' -exec sh -c 'for f; do jar -tf "$f" | grep "gitarrowinfo.properties" | sed "s|^|$f,|"; done' sh {} +

./plasma/target/arrow-plasma-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./tools/target/arrow-tools-9.0.0-SNAPSHOT-jar-with-dependencies.jar,gitarrowinfo.properties
./tools/target/arrow-tools-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./memory/memory-unsafe/target/arrow-memory-unsafe-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./memory/memory-netty/target/arrow-memory-netty-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./memory/memory-core/target/arrow-memory-core-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./dataset/target/arrow-dataset-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./flight/flight-sql/target/flight-sql-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./flight/flight-integration-tests/target/flight-integration-tests-9.0.0-SNAPSHOT-jar-with-dependencies.jar,gitarrowinfo.properties
./flight/flight-integration-tests/target/flight-integration-tests-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./flight/flight-grpc/target/flight-grpc-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./flight/flight-core/target/flight-core-9.0.0-SNAPSHOT-jar-with-dependencies.jar,gitarrowinfo.properties
./flight/flight-core/target/flight-core-9.0.0-SNAPSHOT-shaded.jar,gitarrowinfo.properties
./flight/flight-core/target/flight-core-9.0.0-SNAPSHOT-shaded-ext.jar,gitarrowinfo.properties
./flight/flight-core/target/flight-core-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./adapter/jdbc/target/arrow-jdbc-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./adapter/avro/target/arrow-avro-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./adapter/orc/target/arrow-orc-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./algorithm/target/arrow-algorithm-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./compression/target/arrow-compression-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./format/target/arrow-format-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./gandiva/target/arrow-gandiva-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./performance/target/benchmarks.jar,gitarrowinfo.properties
./performance/target/arrow-performance-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./vector/target/arrow-vector-9.0.0-SNAPSHOT-shade-format-flatbuffers.jar,gitarrowinfo.properties
./vector/target/arrow-vector-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties
./c/target/arrow-c-data-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties 

 

Please let me know if this approach could be possible or not.

 

[asfimport]

Nils Breunese:
Either no longer generating or renaming the Git properties resource would solve the issue that I reported in this ticket.

If you decide on keeping it, I would suggest adding hyphens or underscores to the file name to make it easier to read, e.g. arrow-git.properties.

[asfimport]

David Li / @lidavidm:
Issue resolved by pull request 13578
#13578