7662 make solrconfig less static#8320
Closed
poikilotherm wants to merge 60 commits intoIQSS:developfrom
Closed
Conversation
poikilotherm
added
Type: Feature
Contributor
|
I take it someone saw my morning's failed attempt at a drop-in replacement of Solr-8.11.1? |
Contributor
Author
Actually @donsizemore : no I didn't . This one is completely selfish, wanting to create an up-to-date solrconfig.xml for container images with arbitrary Solr versions. |
Contributor
Author
|
@donsizemore WDYT: should the Or should we even ship a complete Solr What would you prefer for gdcc/dataverse-ansible? |
This was referenced Dec 23, 2021
poikilotherm
force-pushed
the
7662-solrconfig
branch
from
27c7579 to
368358f
Compare
poikilotherm
force-pushed
the
7662-solrconfig
branch
from
6c1a830 to
c3b6754
Compare
poikilotherm
force-pushed
the
7662-solrconfig
branch
from
c3b6754 to
9422c99
Compare
poikilotherm
force-pushed
the
7662-solrconfig
branch
2 times, most recently
from
a8c783b to
490ea3c
Compare
…tatic solrconfig.xml IQSS#7662
…Dataverse specific changes IQSS#7662
Simple Makefile to download Solr, extract the default configset and create a Dataverse flavored one. - Uses Maven to find the Solr distribution version to download. - Uses xsltproc to apply our XSLT transformations to sorlconfig.xml - Replaces the managed-schema with the static one we provide - Zips the configset to make it distributable as artifact
As the SolrJ dependency is going to be used within more than the WAR module, it is getting moved to the parent POM. In the same go, the dependency is versioned with a property "solr.version", that makes it easy to switch to a different Solr version. The property has been present before, but now is updated to 8.11.1 to reflect the latest upstream changes. The Sphinx guide conf.py is also updated to retrieve the property from the parent POM instead of the WAR POM.
The TSV parser needs to verify if a certain line is a header line and matching the spec. To avoid duplicated validation code, this validator can be used with an arbitrary list of strings (so it can be reused for blocks, fields and vocabularies). As we will need to validate URLs in certain fields, this validator also offers a helper function to create predicates checking for valid URLs.
The Block POJO now contains the header specification (uses the Validator class to perform the validation) and allows to parse a line into a List. A later relaxation of the spec allowing for reordering of fields, etc is possible, while the calling code of the parser can reuse the found header definition. A builder pattern is used to parse and validate the actual definition. As the block may only be used once the definition, all fields and vocabularies have been parsed (if the is an error within the TSV the parsing has to fail!), the builder pattern is a natural match to that.
poikilotherm
force-pushed
the
7662-solrconfig
branch
from
df49a40 to
ff0e91c
Compare
This simple class will allow to make the parser somewhat configurable, so future changes and command line options can be integrated more easily.
Instead of defining a static trigger, we want to be able to configure the trigger sign. Due to this, we use the keyword only and move the trigger handling into the ParsingState (which is analysing the line for state transition anyway).
- Implement first details of the Block POJO - Change parsing with BlockBuilder to use an internal state with a not-exposed Block object - The BlockBuilder may manipulate the Block, but after calling build() the calling code will have no option to edit the POJO (proper capsulation and sealing)
Add field types and make them usable as predicates for fields. Add test.
Predicates are not null safe - need to make validate() check for null
Includes all the predicates according to spec and test for them.
Member
|
@poikilotherm does this PR close the following issue? (And if so, can we please add the "closes" syntax to the description?) |
Closed
4 tasks
poikilotherm
force-pushed
the
7662-solrconfig
branch
from
0365f67 to
a13743a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Solr updates become easier when changes are trackable. Moving the config files around results in cumbersome searches for change history. Also, it's hard to track why and when certain additions happened. Declaring necessary changes to a vanilla
solrconfig.xmlmakes this much easier.This pull request will also add necessary bits to installer, docs etc to make use this. Tools like container builds and dataverse-ansible @donsizemore might reuse the XSLTs directly.
TODOs:
Add solr-maven-plugin to start a Solr server with a single Maven command (great for ephemeral dev usage)dataverse-ansibleWhich issue(s) this PR closes:
Special notes for your reviewer:
This is a draft as long as outstanding TODOs are present.
Note the change to Sphinx guides retrieving the Solr version from the parent POM to make updates easier and avoid missing changes.
Suggestions on how to test this:
mvn -f modules/solr-configset/pom.xml packageto compile the config, create the ZIP and execute integration test.Usual integration testing should make sure the index configuration is working properly.
Does this PR introduce a user interface change? If mockups are available, please link/include them here:
Nope.
Is there a release notes update needed for this change?:
Yes that will be necessary.
Additional documentation:
None so far. Will include updated docs later.