Skip to content

GCS lookup support#11026

Merged
pjain1 merged 5 commits intoapache:masterfrom
pjain1:gs_lookup_support
Mar 29, 2021
Merged

GCS lookup support#11026
pjain1 merged 5 commits intoapache:masterfrom
pjain1:gs_lookup_support

Conversation

@pjain1
Copy link
Copy Markdown
Member

@pjain1 pjain1 commented Mar 24, 2021

Description

Support loading lookups from GCS similar to S3.


Key changed/added classes in this PR
  • GoogleTimestampVersionedDataFinder.java

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

Copy link
Copy Markdown
Member

@clintropolis clintropolis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment thread docs/development/extensions-core/lookups-cached-global.md Outdated
Comment thread docs/development/extensions-core/lookups-cached-global.md
Copy link
Copy Markdown
Member

@clintropolis clintropolis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm 👍

Copy link
Copy Markdown
Member

@nishantmonu51 nishantmonu51 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 , lgtm.

@pjain1 pjain1 merged commit 2fdc313 into apache:master Mar 29, 2021
@pjain1 pjain1 deleted the gs_lookup_support branch March 29, 2021 20:10
@clintropolis clintropolis added this to the 0.22.0 milestone Aug 12, 2021
jon-wei added a commit to jon-wei/druid that referenced this pull request Nov 22, 2021
* IMPLY-6556 remove offending settings.xml for intellij inspections

* GCS lookup support (apache#11026)

* GCS lookup support

* checkstyle fix

* review comments

* review comments

* remove unused import

* remove experimental from Kinesis with caveats (apache#10998)

* remove experimental from Kinesis with caveats

* add suggested known issue

* spelling fixes

* Bump aliyun SDK to 3.11.3 (apache#11044)

* Update reset-cluster.md (apache#10990)

fixed Error: Could not find or load main class org.apache.druid.cli.Main

* Make imply-view-manager non-experimental (apache#316)

* Make druid.indexer.task.ignoreTimestampSpecForDruidInputSource default to true, for backwards compat (apache#315)

* Add explicit EOF and use assert instead of exception (apache#11041)

* Add Calcite Avatica protobuf handler (apache#10543)

* bump to latest of same version node and npm versions, bump frontend-maven-plugin (apache#11057)

* request logs through kafka emitter (apache#11036)

* request logs through kafka emitter

* travis fixes

* review comments

* kafka emitter unit test

* new line

* travis checks

* checkstyle fix

* count request lost when request topic is null

* IMPLY-6556 map local repository instead .m2

* remove outdated info from faq (apache#11053)

* remove outdated info from faq

* Add an option for ingestion task to drop (mark unused) all existing segments that are contained by interval in the ingestionSpec (apache#11025)

* Auto-Compaction can run indefinitely when segmentGranularity is changed from coarser to finer.

* Add option to drop segments after ingestion

* fix checkstyle

* add tests

* add tests

* add tests

* fix test

* add tests

* fix checkstyle

* fix checkstyle

* add docs

* fix docs

* address comments

* address comments

* fix spelling

* Allow list for JDBC connection properties to address CVE-2021-26919 (apache#11047)

* Allow list for JDBC connection properties to address CVE-2021-26919

* fix tests for java 11

* Fix compile issue from dropExisting in ingest-service (apache#320)

Co-authored-by: Slava Mogilevsky <triggerwoods91@gmail.com>
Co-authored-by: Parag Jain <pjain1@apache.org>
Co-authored-by: Charles Smith <38529548+techdocsmith@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: frank chen <frank.chen021@outlook.com>
Co-authored-by: Tushar Raj <43772524+tushar-1728@users.noreply.github.com>
Co-authored-by: Jonathan Wei <jon-wei@users.noreply.github.com>
Co-authored-by: Jihoon Son <jihoonson@apache.org>
Co-authored-by: Lasse Krogh Mammen <lkm@bookboon.com>
Co-authored-by: Clint Wylie <cwylie@apache.org>
Co-authored-by: Maytas Monsereenusorn <maytasm@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants