[BEAM-11665] Publish Beam GCP BOM #13737

kileys · 2021-01-12T19:30:17Z

Create a Beam GCP BOM for projects that use Beam and Google Cloud Platform.
Create a Maven archetype using the new BOM and run examples on it
Add BOM validation to release jenkins test and validation script

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Choose reviewer(s) and mention them in a comment (R: @username).
Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Post-Commit Tests Status (on master branch)

Lang	Dataflow	Samza	Twister2
Go	---	---	---
Java
Python		---	---
XLang		---	---

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website	Whitespace	Typescript
Non-portable
Portable	---		---	---	---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

kileys · 2021-01-12T19:30:44Z

Run Gradle Publish

codecov · 2021-01-12T19:50:41Z

Codecov Report

Merging #13737 (c5b7295) into master (6675586) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #13737      +/-   ##
==========================================
- Coverage   82.75%   82.74%   -0.01%     
==========================================
  Files         466      466              
  Lines       57525    57525              
==========================================
- Hits        47604    47601       -3     
- Misses       9921     9924       +3

Impacted Files	Coverage Δ
.../python/apache_beam/testing/test_stream_service.py	`88.63% <0.00%> (-4.55%)`	⬇️
sdks/python/apache_beam/utils/interactive_utils.py	`88.09% <0.00%> (-2.39%)`	⬇️
...che_beam/runners/interactive/interactive_runner.py	`90.90% <0.00%> (-1.82%)`	⬇️
sdks/python/apache_beam/internal/metrics/metric.py	`86.45% <0.00%> (-1.05%)`	⬇️
sdks/python/apache_beam/transforms/util.py	`95.66% <0.00%> (-0.18%)`	⬇️
...ks/python/apache_beam/runners/worker/sdk_worker.py	`89.69% <0.00%> (-0.16%)`	⬇️
...hon/apache_beam/runners/worker/bundle_processor.py	`93.70% <0.00%> (-0.13%)`	⬇️
sdks/python/apache_beam/dataframe/frames.py	`91.88% <0.00%> (+0.41%)`	⬆️
sdks/python/apache_beam/runners/direct/executor.py	`96.82% <0.00%> (+0.52%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6675586...c5b7295. Read the comment docs.

kileys · 2021-01-12T21:14:52Z

Run Gradle Publish

suztomo · 2021-01-13T17:35:58Z

@kileys Can I see the generated BOM somewhere?

sdks/java/bom/common.gradle

kileys · 2021-01-20T02:15:37Z

@kileys Can I see the generated BOM somewhere?
I updated the artifact name, but you can see the pom here:

@suztomo
https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-sdks-java-bom-gcp/2.28.0-SNAPSHOT/beam-sdks-java-bom-gcp-2.28.0-20210112.211655-2.pom

kileys · 2021-01-20T02:16:21Z

Run Gradle Publish

elharo

<url>http://beam.apache.org</url> should use https
2 <inceptionYear>2016</inceptionYear> should be 2021
<organizationUrl>http://www.apache.org</organizationUrl> should use https
<archive>http://www.mail-archive.com/dev%beam.apache.org</archive> should use https

kileys · 2021-01-20T17:57:37Z

Run Dataflow PostRelease

kileys · 2021-01-20T21:04:17Z

Run Dataflow PostRelease

kileys · 2021-01-20T22:33:33Z

Run Dataflow PostRelease

kileys · 2021-01-20T22:42:01Z

<url>http://beam.apache.org</url> should use https
2 <inceptionYear>2016</inceptionYear> should be 2021

<organizationUrl>http://www.apache.org</organizationUrl> should use https

<archive>http://www.mail-archive.com/dev%beam.apache.org</archive> should use https

@elharo
That's an outdated pom before I made the changes. Also, for #2, it's inception year not current year.

Here's the newest pom:
https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-sdks-java-google-cloud-platform-bom/2.28.0-SNAPSHOT/beam-sdks-java-google-cloud-platform-bom-2.28.0-20210120.122741-1.pom

kileys · 2021-01-21T01:26:26Z

Run Dataflow PostRelease

kileys · 2021-01-21T01:26:39Z

R: @kennknowles

suztomo · 2021-01-22T22:14:23Z

I see it's working as expected. (Effective pom after I replaced the version iwth the latest release 2.27: https://gist.github.com/suztomo/ab8bf9dc2653ce2a886a5a57a0204eb2#file-gistfile1-txt-L190 )

I want Guava version to be more higher: #13740

udim · 2021-09-21T20:06:45Z

release/src/main/scripts/run_rc_validation.sh

    -PgcsBucket=${USER_GCS_BUCKET:5}  # skip 'gs://' prefix
+
+    echo "**************************************************************************"
+    echo "* Java mobile game on DataflowRunner using Beam GCP BOM: UserScore, HourlyTeamScore, Leaderboard"


Should this be included as part of the release validation spreadsheet?
In other words, is this required to pass to validate the RC?

kileys added 2 commits January 12, 2021 19:19

Publish beam gcp bom

90561ac

move beam bom to dep

8a7269c

refactor

b9d4d34

ignore bom in javadoc project

e49030d

elharo reviewed Jan 13, 2021

View reviewed changes

sdks/java/bom/common.gradle Outdated Show resolved Hide resolved

sdks/java/bom/common.gradle Outdated Show resolved Hide resolved

sdks/java/bom/common.gradle Outdated Show resolved Hide resolved

fix urls, rename artifact

8f8e296

suztomo mentioned this pull request Jan 14, 2021

[BEAM-11626] Guava version 25.1-jre for Hadoop/Cassandra and Guava version 30.1 (latest) for the rest #13740

Closed

4 tasks

kileys added 6 commits January 14, 2021 23:59

Merge branch 'master' of github.com:apache/beam into beam-bom

1768113

add new archetype project

f829c05

scripts to run examples with bom

a6c0e15

add task to run bom examples

c02ffe7

add maven archetype for beam gcp bom

969dfe7

add to release script

c5b7295

elharo reviewed Jan 20, 2021

View reviewed changes

kileys changed the title ~~[WIP] Publish Beam GCP BOM~~ [BEAM-11665] Publish Beam GCP BOM Jan 20, 2021

suztomo approved these changes Jan 22, 2021

View reviewed changes

suztomo mentioned this pull request Jan 25, 2021

[BEAM-11626] Upgrading Guava to 30.1-jre while keeping 25.1-jre for Hadoop/Cassandra modules #13804

Merged

4 tasks

kennknowles approved these changes Jan 25, 2021

View reviewed changes

kennknowles merged commit cec7f10 into apache:master Jan 25, 2021

suztomo mentioned this pull request Jan 28, 2021

[BEAM-11648] In preparation for new BigQuery sink, update GCP BOM #13765

Merged

udim reviewed Sep 21, 2021

View reviewed changes

[BEAM-11665] Publish Beam GCP BOM #13737

[BEAM-11665] Publish Beam GCP BOM #13737

Uh oh!

Conversation

kileys commented Jan 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Post-Commit Tests Status (on master branch)

Pre-Commit Tests Status (on master branch)

GitHub Actions Tests Status (on master branch)

Uh oh!

kileys commented Jan 12, 2021

Uh oh!

codecov bot commented Jan 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

kileys commented Jan 12, 2021

Uh oh!

suztomo commented Jan 13, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kileys commented Jan 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kileys commented Jan 20, 2021

Uh oh!

elharo left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kileys commented Jan 20, 2021

Uh oh!

kileys commented Jan 20, 2021

Uh oh!

kileys commented Jan 20, 2021

Uh oh!

kileys commented Jan 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kileys commented Jan 21, 2021

Uh oh!

kileys commented Jan 21, 2021

Uh oh!

suztomo commented Jan 22, 2021

Uh oh!

udim Sep 21, 2021

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kileys commented Jan 12, 2021 •

edited

Loading

codecov bot commented Jan 12, 2021 •

edited

Loading

kileys commented Jan 20, 2021 •

edited

Loading

elharo left a comment •

edited

Loading

kileys commented Jan 20, 2021 •

edited

Loading