EM-1185: Webpage Index Issues #483
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose:
This does two things:
Webpagestables_widgetfield. ElasticSearch was breaking when attempting to index records with data in this column, as ES dynamically inferred that it was a nested datatype, and was attempting to analyze and index its nested structure. Because each Webpage has potentially vastly differenttables_widgetdata structures, ES's initially inferred nested structure (which it would infer from the first record it came across withtables_widgetdata) was invalid for subsequent records. There are native ways in ES to set up dynamic mappings of these nested structures, but we do not needtables_widgetto be indexed. It's worth looking into their dynamic mapping system when we implement ES for Cortex Beta, however.Webpagesfields, and disables tokenization/analysis of more fieldsJIRA:
https://cb-content-enablement.atlassian.net/browse/EM-1185
Steps to Take On Prod
This has already been tested on Production (as it was the only place it was happening), is in place and works.
Changes:
Changes to setup
Architectural changes
Migrations
Library changes
Side effects
Screenshots
Before
N/A
After
N/A
QA Links:
Since this is already on Prod and works, I will not be pushing this to a feature server. I have, however, provided verification steps below if you're interested in testing locally.
How to Verify These Changes
Specific pages to visit
Steps to take
Webpageswith differenttables_widgetdataResponsive considerations
Relevant PRs/Dependencies:
N/A
Additional Information
N/A