Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 9 additions & 10 deletions Documentation/CACHE-HOWTO.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,18 +17,17 @@

### Overview

Cache configuration provides you additional control over incremental maven behavior. Follow it step by step to
Cache configuration provides you additional control over incremental Maven behavior. Follow it step by step to
understand how it works and figure out your optimal config

### Minimal config

Absolutely minimal config which enables incremental maven with local cache
Absolutely minimal config which enables incremental Maven with local cache

```xml
<?xml version="1.0" encoding="UTF-8" ?>
<cache xmlns="org:apache:maven:cache:config:v1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="org:apache:maven:cache:config:v1 cache-config.xsd">
<cache xmlns="org:apache:maven:cache:config:v1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="org:apache:maven:cache:config:v1 http://maven.apache.org/xsd/cache-config-1.0.0.xsd">

<configuration>
<enabled>true</enabled>
Expand Down Expand Up @@ -73,7 +72,7 @@ Add all the project specific source code files in `<glob>`. Scala in this case:

### Adding source directory for bespoke project layouts

In most of the cases incremental maven will recognize directories automatically by build introspection. If not, you can
In most of the cases incremental Maven will recognize directories automatically by build introspection. If not, you can
add additional directories with `<include>`. Also you can filter out undesirable dirs and files by using exclude tag

```xml
Expand Down Expand Up @@ -134,7 +133,7 @@ Add `executionControl/runAlways` section
```xml
<?xml version="1.0" encoding="UTF-8" ?>
<cache xmlns="org:apache:maven:cache:config:v1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="org:apache:maven:cache:config:v1 cache-config.xsd">
xsi:schemaLocation="org:apache:maven:cache:config:v1 http://maven.apache.org/xsd/cache-config-1.0.0.xsd">
<configuration>
...
</configuration>
Expand All @@ -157,7 +156,7 @@ Add `executionControl/runAlways` section

### I occasionally cached build with `-DskipTests=true` and tests do not run now

If you add command line flags to your build, they do not participate in effective pom - maven defers final value
If you add command line flags to your build, they do not participate in effective pom - Maven defers final value
resolution to plugin runtime. To invalidate build if filed value is different in runtime, add reconciliation section
to `executionControl`:

Expand All @@ -180,7 +179,7 @@ to `executionControl`:

Please notice `skipValue` attribute. It denotes value which forces skipped execution.
Read `propertyName="skipTests" skipValue="true"` as if property skipTests has value true, plugin will skip execution If
you declare such value incremental maven will reuse appropriate full-build though technically they are different, but
you declare such value incremental Maven will reuse appropriate full-build though technically they are different, but
because full-build is better it is safe to reuse

### How to renormalize line endings in working copy after committing .gitattributes (git 2.16+)
Expand All @@ -201,4 +200,4 @@ git reset --hard
### I want to cache interim build and override it later with final version

Solution: set `-Dremote.cache.save.final=true` to nodes which produce final builds. Such builds will not be overridden
and eventually will replace all interim builds
and eventually will replace all interim builds
4 changes: 2 additions & 2 deletions Documentation/CACHE-PARAMETERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ This documents contains various configuration parameters supported by cache engi
| `-Dremote.cache.configPath=true/false` | Location of cache configuration file | Cache config is not in default location |
| `-Dremote.cache.enabled=true/false` | Remote cache and associated features disabled/enabled | To remove noise from logs then remote cache is not available |
| `-Dremote.cache.save.enabled=true/false` | Remote cache save allowed or not | To designate nodes which allowed to push in remote shared cache |
| `-Dremote.cache.save.final=true/false` | Is it allowed or not to override produced cache | To ensure that reference build is not overriden by interim build |
| `-Dremote.cache.failFast=true/false` | Fail on the first module whcih cannot be restored from cache | Remote cache setup/tuning/troubleshooting |
| `-Dremote.cache.save.final=true/false` | Is it allowed or not to override produced cache | To ensure that reference build is not overridden by interim build |
| `-Dremote.cache.failFast=true/false` | Fail on the first module which cannot be restored from cache | Remote cache setup/tuning/troubleshooting |
| `-Dremote.cache.baselineUrl=<http url>` | Location of baseline build for comparison | Remote cache setup/tuning/troubleshooting |

## Project level properties
Expand Down
8 changes: 4 additions & 4 deletions Documentation/CACHE-REMOTE.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

# Overview

This document describes generic approach to cache setup. The process require expertise in maven equivalent to expertise
This document describes generic approach to cache setup. The process require expertise in Maven equivalent to expertise
required to author and Maven your project build, it implies good knowledge of both Maven and the project. Due to Maven
model limitation the process is manual, but allows you to achieve sufficient control and transparency over caching
logic.
Expand Down Expand Up @@ -113,8 +113,8 @@ Once discrepancy between remote and local builds detected cache will fail with d
in `target/incremental-maven` directory:

```
* buildinfo-baseline-3c64673e23259e6f.xml - build specficiation from baseline build
* buildinfo-db43936e0666ce7.xml - build specification of locall build
* buildinfo-baseline-3c64673e23259e6f.xml - build specification from baseline build
* buildinfo-db43936e0666ce7.xml - build specification of local build
* buildsdiff.xml - comparison report with list of discrepancies
```

Expand Down Expand Up @@ -242,7 +242,7 @@ relax consistency rules in favor of compatibility, remove property from tracked

Current implementation doesn't support version changes between cache entries. It will result in cache invalidation for
each new version.
To mitigate the issue please consider migrating off traditional maven release approach - try to use single version id in
To mitigate the issue please consider migrating off traditional Maven release approach - try to use single version id in
project (eg `<version>MY-PROJECT-LOCAL</version>`). Such approach simplifies git branching workflow significantly.

Deployment of artifacts with specific version from builds with cache is not supported yet.
Expand Down
44 changes: 22 additions & 22 deletions Documentation/CACHE.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,19 @@

## Overview

Idea of Incremental Maven is to specify module inputs and outputs and make them known to standard maven core. This
Idea of Incremental Maven is to specify module inputs and outputs and make them known to standard Maven core. This
allows accurate analysis and determination of out-of-date build artifacts in the build dependencies graph. Making the
dependency graph analysis deterministic leads to improvements in build times by avoiding re-building unnecessary
modules.
Cache does not make any low-level interventions to build process and delegates actual build work to maven core. This
guarantees that build results are identical to results produced by standard maven and are fully reproducible.
To achieve accurate input and outputs calculation incremental maven combines automatic introspection
Cache does not make any low-level interventions to build process and delegates actual build work to Maven core. This
guarantees that build results are identical to results produced by standard Maven and are fully reproducible.
To achieve accurate input and outputs calculation incremental Maven combines automatic introspection
of [project object model](https://maven.apache.org/pom.html#What_is_the_POM) in conjunction with configuration-driven
rules for fine-grained content and execution control. For content analysis it digests based approach which is more
reliable over widely used file timestamps in tools like Make or Apache Ant. Deterministic build state allows reliably
cache even intermediate outputs of build and share them between teams using remote cache. Deterministic inputs
calculation allows distributed and parallel builds running in heterogeneous environments (like cloud of build agents)
could efficiently reuse cached build artifacts. Therefore incremental maven is particularly well-suited for large maven
could efficiently reuse cached build artifacts. Therefore incremental Maven is particularly well-suited for large Maven
projects that have significant number of small modules. Remote cache in conjunction with precise input identification
effectively enables "change once - build once" approach.

Expand All @@ -51,7 +51,7 @@ of today build with some tolerance (implementation, configuration and environmen

### Implementation insights

At very simple form, the incremental maven is essentially a hash function which takes maven project and produces hash
At very simple form, the incremental Maven is essentially a hash function which takes Maven project and produces hash
code (checksum). Then hash value is used to fetch and restore build result.
As with any hash, there could be collisions and instabilities. Collision could happen if the same hash produced from the
different build states and will result in unintended reuse. Instability means that same input yields different hash sums
Expand Down Expand Up @@ -79,7 +79,7 @@ verification it's still consumer's responsibility to verify final product qualit
Given all the information above, the Incremental Maven is recommended to use in scenarios when productivity and
performance are in priority. Typical cases are:

* Speedup CI. In conjunction with remote cache incremental maven could drastically reduce build times, validate pull
* Speedup CI. In conjunction with remote cache incremental Maven could drastically reduce build times, validate pull
requests faster and reduce load on CI nodes
* Speedup developer builds. By reusing cached builds developers could verify changes much faster and be more productive.
No more `-DskipTests` and similar.
Expand All @@ -92,35 +92,35 @@ This also allows you to validate cache correctness and reconcile cache outcomes

## Getting Started

To on-board incremental maven you need to complete several steps:
To on-board incremental Maven you need to complete several steps:

* Get incremental maven distribution
* Add cache config in .mvn
* Get incremental Maven distribution
* Add cache config in `.mvn`
* Validate build results and iteratively adjust config to project specifics
* Migrate CI to incremental maven with remote cache (to get full benefit) - optional
* Migrate CI to incremental Maven with remote cache (to get full benefit) - optional

### Get incremental maven distribution
### Get incremental Maven distribution

The recommended way is to add [Takari Maven Wrapper](https://github.com/takari/maven-wrapper) to your project. In that
case `maven-wrapper.properties` should reference the latest incremental maven distribution:
case `maven-wrapper.properties` should reference the latest incremental Maven distribution:

```properties
distributionUrl=https://your-server/maven-incremental.zip
wrapperUrl=https://your-server/maven-wrapper-0.5.5.jar
```

Benefits of using maven wrapper are following:
Benefits of using Maven wrapper are following:

* simple distribution across workstations and CI envs
* maven stays compatible to your branch
* Maven stays compatible to your branch
* further upgrades are simplified significantly
If you refuse wrapper - then download, unzip and install it just as usual maven. Further it will be assumed you use
maven wrapper (`mvnw`)
If you refuse wrapper - then download, unzip and install it just as usual Maven. Further it will be assumed you use
Maven wrapper (`mvnw`)

### Adding cache config

Copy [default config](maven-cache-config.xml) and [xml schema](../maven-core/src/main/resources/cache-config.xsd)
to [.mvn](https://maven.apache.org/configure.html) dir of yor project.
Copy [default config `maven-cache-config.xml`](maven-cache-config.xml)
to [`.mvn/`](https://maven.apache.org/configure.html) dir of your project.
To get overall understanding of cache machinery it is recommended to review the config and read comments. In typical
scenario you need to adjust:

Expand All @@ -135,15 +135,15 @@ See also:
* [Remote cache setup](CACHE-REMOTE.md) - instruction how to setup shared cache
* [Cache How-To](CACHE-HOWTO.md) - cookbook for frequently encountered questions
* [Cache Parameters](CACHE-PARAMETERS.md) - description of supported parameters
* Attached [sample config file](maven-cache-config.xml) and elements annotations in xsd schema. (Ctrl+Q in idea should
* Attached [sample `maven-cache-config.xml` config file](maven-cache-config.xml) and elements annotations in xsd schema. (Ctrl+Q in idea should
show annotations in popup)

### Adjusting cache config

Having incremental maven and the config in place you're all set. To run first cacheable build just
Having incremental Maven and the config in place you're all set. To run first cacheable build just
execute: `mvnw clean install`

* Ensure that the config is picked up and incremental maven is picked up. Just check log output - you will notice cache
* Ensure that the config is picked up and incremental Maven is picked up. Just check log output - you will notice cache
related output or initialization error message.
* Navigate to your local repo directory - there should be sibling next to your local repo named `cache`
* Find `buildinfo.xml` for typical module and review it. Ensure that
Expand Down
5 changes: 2 additions & 3 deletions Documentation/maven-cache-config.xml
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,8 @@
See the License for the specific language governing permissions and
limitations under the License.
-->
<cache xmlns="org:apache:maven:cache:config:v1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="org:apache:maven:cache:config:v1 cache-config.xsd">
<cache xmlns="org:apache:maven:cache:config:v1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="org:apache:maven:cache:config:v1 http://maven.apache.org/xsd/cache-config-1.0.0.xsd">

<configuration>
<enabled>true</enabled>
Expand Down
2 changes: 0 additions & 2 deletions apache-maven/src/main/appended-resources/META-INF/LICENSE.vm
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,6 @@ subject to the terms and conditions of the following licenses:
#* *##set ( $spdx = 'EPL-1.0' )
#* *##elseif ( $license.url.contains( "www.apache.org/licenses/LICENSE-2.0" ) )
#* *##set ( $spdx = 'ASL-2.0' )
#* *##elseif ( $license.url.contains( "www.eclipse.org/org/documents/edl-v10.php" ) )
#* *##set ( $spdx = 'EDL-1.0' )
#* *##elseif ( $license.url.contains( "www.mozilla.org/en-US/MPL/1.1" ) )
#* *##set ( $spdx = 'MPL-1.1' )
#* *##elseif ( $license.url.contains( "www.gnu.org/licenses/gpl" ) )
Expand Down
23 changes: 0 additions & 23 deletions apache-maven/src/main/appended-resources/licenses/EDL-1.0.txt

This file was deleted.

69 changes: 37 additions & 32 deletions maven-core/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -156,17 +156,6 @@ under the License.
<artifactId>zero-allocation-hashing</artifactId>
<version>0.9</version>
</dependency>
<dependency>
<groupId>jakarta.xml.bind</groupId>
<artifactId>jakarta.xml.bind-api</artifactId>
<version>2.3.3</version>
</dependency>
<dependency>
<groupId>com.sun.xml.bind</groupId>
<artifactId>jaxb-impl</artifactId>
<version>2.3.3</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>com.github.albfernandez</groupId>
<artifactId>juniversalchardet</artifactId>
Expand Down Expand Up @@ -217,7 +206,43 @@ under the License.
</configuration>
<executions>
<execution>
<id>modello-site-doc</id>
<id>modello-site-docs-cache</id>
<phase>pre-site</phase>
<goals>
<goal>xdoc</goal>
<goal>xsd</goal>
</goals>
<configuration>
<version>1.0.0</version>
<models>
<model>src/main/mdo/cache-buildinfo.mdo</model>
<model>src/main/mdo/cache-buildsdiff.mdo</model>
<model>src/main/mdo/cache-config.mdo</model>
<model>src/main/mdo/cache-report.mdo</model>
</models>
</configuration>
</execution>
<execution>
<id>modello-cache</id>
<goals>
<goal>java</goal>
<goal>xpp3-reader</goal>
<goal>xpp3-writer</goal>
<goal>xsd</goal>
</goals>
<configuration>
<version>1.0.0</version>
<models>
<model>src/main/mdo/cache-buildinfo.mdo</model>
<model>src/main/mdo/cache-buildsdiff.mdo</model>
<model>src/main/mdo/cache-config.mdo</model>
<model>src/main/mdo/cache-report.mdo</model>
</models>
<outputDirectory>${basedir}/target/generated-sources/modello</outputDirectory>
</configuration>
</execution>
<execution>
<id>modello-site-docs-extension</id>
<phase>pre-site</phase>
<goals>
<goal>xdoc</goal>
Expand Down Expand Up @@ -248,26 +273,6 @@ under the License.
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>jaxb2-maven-plugin</artifactId>
<version>2.5.0</version>
<executions>
<execution>
<id>xjc</id>
<goals>
<goal>xjc</goal>
</goals>
</execution>
</executions>
<configuration>
<packageName>org.apache.maven.caching.jaxb</packageName>
<sources>
<source>src/main/resources/cache-config.xsd</source>
<source>src/main/resources/cache-domain.xsd</source>
</sources>
</configuration>
</plugin>
</plugins>
</build>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
*/

import org.apache.maven.artifact.Artifact;
import org.apache.maven.caching.jaxb.CacheReportType;
import org.apache.maven.caching.xml.report.CacheReportType;
import org.apache.maven.caching.xml.BuildInfo;
import org.apache.maven.execution.MavenSession;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
* under the License.
*/

import org.apache.maven.caching.jaxb.ProjectsInputInfoType;
import org.apache.maven.caching.xml.buildinfo.ProjectsInputInfoType;
import org.apache.maven.execution.MavenSession;
import org.apache.maven.project.MavenProject;

Expand Down
Loading