diff --git a/maven-resolver-named-locks/src/site/markdown/index.md.vm b/maven-resolver-named-locks/src/site/markdown/index.md.vm index 0d8d40166..24ed4760b 100644 --- a/maven-resolver-named-locks/src/site/markdown/index.md.vm +++ b/maven-resolver-named-locks/src/site/markdown/index.md.vm @@ -23,28 +23,41 @@ Named locks are essentially locks that are assigned to some given (opaque) ID. I resources that each can have unique ID assigned (i.e., file with an absolute path, some entities with unique ID), then you can use named locks to make sure they are being protected from concurrent read and write actions. -Named locks provide support classes for implementations, and provide out of the box seven named lock implementations and three name mappers. +Named locks provide support classes for implementations, and provide out of the box several lock and name mapper implementations. -Out of the box, "local" (local to JVM) named lock implementations are the following: +Following implementations are "local" (local to JVM) named lock implementations: - `rwlock-local` implemented in `org.eclipse.aether.named.providers.LocalReadWriteLockNamedLockFactory` that uses JVM `java.util.concurrent.locks.ReentrantReadWriteLock`. - `semaphore-local` implemented in `org.eclipse.aether.named.providers.LocalSemaphoreNamedLockFactory` that uses JVM `java.util.concurrent.Semaphore`. +- `noop` implemented in `org.eclipse.aether.named.providers.NoopNamedLockFactory` that uses no locking. + +Note about "local" locks: they are in-JVM, in a way, they properly coordinate in case of multithreaded access from +same JVM, but do not cover accesses across multiple processes and/or multiple hosts access. +In other words, local named locks are only suited within one JVM with a multithreaded access. + +Following named lock implementations use underlying file system advisory file locking: + - `file-lock` implemented in `org.eclipse.aether.named.providers.FileLockNamedLockFactory` that uses JVM `java.nio.channels.FileLock`. -- `noop` implemented in `org.eclipse.aether.named.providers.NoopNamedLockFactory` that uses no locking. -Out of the box, "distributed" named lock implementations are the following (separate modules which require additional dependencies): +The `file-lock` implementation uses file system advisory file locking, hence, concurrently running Maven processes +set up to use `file-lock` implementation can safely share one local repository. This is almost certain on +local file systems across all operating systems. In case of NFS mounts, file advisory locking MAY work if NFSv4+ +used with complete setup (with all the necessary services like `RPC` and `portmapper` needed to implement NFS advisory +file locking, check your NFS and/or OS manuals for details). In short: if your (local or remote) FS correctly support +and implements advisory locking, it should work. Local FS usually does, while with NFS your mileage may vary. + +Finally, "distributed" named lock implementations are the following (separate modules which require additional dependencies and remote services): - `rwlock-redisson` implemented in `org.eclipse.aether.named.redisson.RedissonReadWriteLockNamedLockFactory`. - `semaphore-redisson` implemented in `org.eclipse.aether.named.redisson.RedissonSemaphoreNamedLockFactory`. - `semaphore-hazelcast-client` implemented in `org.eclipse.aether.named.hazelcast.HazelcastClientCPSemaphoreNamedLockFactory`. - `semaphore-hazelcast` implemented in `org.eclipse.aether.named.hazelcast.HazelcastCPSemaphoreNamedLockFactory`. -Local named locks are only suited within one JVM with a multithreaded build. -Sharing a local repository between multiple Maven processes (i.e., on a busy CI server) requires a distributed named lock! - +Sharing a local repository between multiple hosts (i.e., on a busy CI server) may be best done with one of distributed named lock, +if NFS locking is not working for you. The aforementioned (opaque) IDs need to be mapped from artifacts and metadata. @@ -54,3 +67,5 @@ Out of the box, name mapper implementations are the following: - `gav` implemented in `org.eclipse.aether.internal.impl.synccontext.named.GAVNameMapper`. - `discriminating` implemented in `org.eclipse.aether.internal.impl.synccontext.named.DiscriminatingNameMapper`. - `file-gav` implemented in `org.eclipse.aether.internal.impl.synccontext.named.FileGAVNameMapper`. + +Note: the `file-gav` name mapper MUST be used with `file-lock` named locking, no other mapper will work with it. diff --git a/src/site/markdown/local-repository.md b/src/site/markdown/local-repository.md index 9ee50e2d0..3554c3962 100644 --- a/src/site/markdown/local-repository.md +++ b/src/site/markdown/local-repository.md @@ -25,30 +25,16 @@ remote, but also to store the artifacts locally installed (locally built and installed, to be more precise). Both of these artifacts were stored in bulk in the local repository. +## Implementations + Local repository implementations implement the `LocalRepositoryManager` (LRM) interface, and Resolver out of the box provides two implementations for it: -"simple" and "enhanced". - -## Simple LRM +"enhanced" used at runtime and "simple" meant to be used in tests and alike +scenarios (is not meant for production use). -Simple is a fully functional LRM implementation, but is used -mainly in tests, it is not recommended in production environments. - -To manually instantiate a simple LRM, one needs to invoke following code: +### Enhanced LRM -```java -LocalRepositoryManager simple = new SimpleLocalRepositoryManagerFactory() - .newInstance( session, new LocalRepository( baseDir ) ); -``` - -Note: This code snippet above instantiates a component, that is not -recommended way to use it, as it should be rather injected whenever possible. -This example above is merely a showcase how to obtain LRM implementation -in unit tests. - -## Enhanced LRM - -Enhanced LRM on the other hand is enhanced with several extra +Enhanced LRM is enhanced with several extra features, one most notable is scoping cached content by its origin and context: if you downloaded an artifact A1 from repository R1 and later initiate build that requires same artifact A1, but repository R1 @@ -58,7 +44,7 @@ Those two, originating from two different repositories may not be the same thing This is meant to protect users from "bad practice" (artifact coordinates are unique in ideal world). -### Split Local Repository +#### Split Local Repository Latest addition to the enhanced LRM is *split* feature. By default, split feature is **not enabled**, enhanced LRM behaves as it behaved in all @@ -75,7 +61,7 @@ The split feature is implemented by the `LocalPathPrefixComposer` interface, that adds different "prefixes" for the locally stored artifacts, based on their context. -#### Note About Release And Snapshot Differentiation +##### Note About Release And Snapshot Differentiation The prefix composer is able to differentiate between release and snapshot versioned artifacts, and this is clear-cut: Maven Artifacts are either @@ -99,7 +85,7 @@ The GAV level metadata gets differentiated based on version it carries, so they may end up in releases or snapshots, depending on their value of `metadata/version` field. -#### Use Cases +##### Use Cases Most direct use case is simpler local repository eviction. One can delete all locally built artifacts without deleting the cached ones, hence, no @@ -139,7 +125,7 @@ $ mvn ... -Daether.enhancedLocalRepository.split \ For complete reference of enhanced LRM configuration possibilities, refer to [configuration page](configuration.html). -#### Split Repository Considerations +##### Split Repository Considerations **Word of warning**: on every change of "split" parameters, user must be aware of the consequences. For example, if one change all aspects of split @@ -149,7 +135,7 @@ is unchanged! Simply put, as all prefixes will be "new", the composed paths will point to potentially non-existing locations, hence, resolver will consider it as a "new" local repository in every aspect. -#### Implementing Custom Split Strategy +##### Implementing Custom Split Strategy To implement custom split strategy, one needs to create a component of type `LocalPathPrefixComposerFactory` and override the default component @@ -160,3 +146,37 @@ class that provides all the defaults. The factory should create a stateless instance of a composer configured from passed in session, that will be used with the enhanced LRM throughout the session. + +### Simple LRM + +Simple is a fully functional LRM implementation, but is used +mainly in tests, it is not recommended in production environments. + +To manually instantiate a simple LRM, one needs to invoke following code: + +```java +LocalRepositoryManager simple = new SimpleLocalRepositoryManagerFactory() + .newInstance( session, new LocalRepository( baseDir ) ); +``` + +Note: This code snippet above instantiates a component, that is not +recommended way to use it, as it should be rather injected whenever possible. +This example above is merely a showcase how to obtain LRM implementation +in unit tests. + +## Shared Access to Local Repository + +In case of shared (multi-threaded, multi-process or even multi host) access +to local repository, coordination is a must, as local repository is hosted +on file system, and each thread may read and write concurrently into it, +causing other threads or processes to get incomplete or partially written data. + +Hence, since Resolver 1.7.x version, there is a pluggable API called "Named Locks" +available, providing out of the box lock implementations for cases like: + +* multi-threaded, in JVM locking (the default) +* multi-process locking using file system advisory locking +* multi-host locking using Hazelcast or Redisson (needs Hazelcast or Redisson cluster) + +For details see [Named Locks module](maven-resolver-named-locks/). +