Transform service test suite #27578

chamikaramj · 2023-07-20T16:46:30Z

Adds a test suite for the Docker Composed based Transform Service.

Passing run: https://ci-beam.apache.org/job/beam_PostCommit_TransformService_Direct_PR/44/

Currently only includes Python BigTableIO integration tests, but we can add more tests to the suite in the future.

This addresses #27941.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

github-actions · 2023-07-20T18:34:58Z

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @jrmccluskey for label python.
R: @Abacn for label build.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

chamikaramj · 2023-07-20T18:46:50Z

(not ready for review yet)

chamikaramj · 2023-07-21T23:07:38Z

Run Seed Job

codecov · 2023-07-21T23:26:03Z

Codecov Report

Merging #27578 (9e2f809) into master (3d02b3b) will increase coverage by 0.33%.
Report is 79 commits behind head on master.
The diff coverage is n/a.

❗ Current head 9e2f809 differs from pull request most recent head a2f8bf9. Consider uploading reports for the commit a2f8bf9 to get more accurate results

@@            Coverage Diff             @@
##           master   #27578      +/-   ##
==========================================
+ Coverage   70.59%   70.92%   +0.33%     
==========================================
  Files         857      861       +4     
  Lines      103944   105083    +1139     
==========================================
+ Hits        73375    74529    +1154     
+ Misses      29004    28989      -15     
  Partials     1565     1565

Flag	Coverage Δ
python	`79.91% <ø> (+0.35%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 34 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

chamikaramj · 2023-07-21T23:26:31Z

Run TransformService_Direct PostCommit

github-actions · 2023-07-29T12:13:38Z

Reminder, please take a look at this pr: @jrmccluskey @Abacn

Abacn · 2023-07-30T20:52:31Z

waiting on author

chamikaramj · 2023-07-31T05:21:30Z

Run Seed Job

chamikaramj · 2023-07-31T05:31:41Z

Run TransformService_Direct PostCommit

chamikaramj · 2023-07-31T05:38:35Z

Run TransformService_Direct PostCommit

chamikaramj · 2023-07-31T17:47:46Z

Run TransformService_Direct PostCommit

chamikaramj · 2023-07-31T18:18:00Z

Run TransformService_Direct PostCommit

chamikaramj · 2023-07-31T18:19:03Z

Run Seed Job

chamikaramj · 2023-07-31T18:31:48Z

Run TransformService_Direct PostCommit

chamikaramj · 2023-07-31T19:52:54Z

Run TransformService_Direct PostCommit

chamikaramj · 2023-07-31T21:20:20Z

Run Seed Job

chamikaramj · 2023-07-31T21:30:14Z

Run TransformService_Direct PostCommit

chamikaramj · 2023-08-09T01:40:01Z

Run Seed Job

chamikaramj · 2023-08-09T01:51:08Z

Run TransformService_Direct PostCommit

chamikaramj · 2023-08-09T20:26:23Z

Run Seed Job

chamikaramj · 2023-08-09T20:36:46Z

Run TransformService_Direct PostCommit

chamikaramj · 2023-08-09T20:52:30Z

R: @ahmedabu98

github-actions · 2023-08-09T20:53:36Z

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control

ahmedabu98 · 2023-08-14T16:57:30Z

.test-infra/jenkins/job_PostCommit_TransformService_Direct.groovy

Did you mean to embed the value into this string?

ahmedabu98 · 2023-08-14T17:22:40Z

.test-infra/jenkins/job_PostCommit_TransformService_Direct.groovy

FYI tests can run faster if we use Python wheels
@Abacn helped me with that here

Great. Added the switch for that.

ahmedabu98 · 2023-08-14T17:36:15Z

sdks/python/test-suites/direct/xlang/build.gradle

Can we add a comment here that these tests are run with Py38?

Actually, these test can run with all Python versions. Updated.

(JavaUsingPython will be restricted to Python 3.8 I'll mention that later when we add such tests).

ahmedabu98 · 2023-08-14T17:40:31Z

sdks/python/apache_beam/io/gcp/bigtableio_it_test.py

I may be missing something but how does this test suite make sure to build the necessary jars before running? For example in this test we need :sdks:java:io:google-cloud-platform:expansion-service

These jars are included in the expansion service container and are built as a part of container building in the ":sdks:java:expansion-service:container:docker" dependency.

chamikaramj · 2023-08-14T18:26:02Z

Run Seed Job

chamikaramj · 2023-08-14T18:41:29Z

Run Seed Job

chamikaramj · 2023-08-14T19:29:36Z

Run Seed Job

chamikaramj · 2023-08-14T19:52:10Z

Run TransformService_Direct PostCommit

chamikaramj

Thanks. PTAL.

chamikaramj · 2023-08-14T18:00:44Z

.test-infra/jenkins/job_PostCommit_TransformService_Direct.groovy

chamikaramj · 2023-08-14T18:02:54Z

.test-infra/jenkins/job_PostCommit_TransformService_Direct.groovy

Great. Added the switch for that.

chamikaramj · 2023-08-14T18:03:54Z

sdks/python/apache_beam/io/gcp/bigtableio_it_test.py

These jars are included in the expansion service container and are built as a part of container building in the ":sdks:java:expansion-service:container:docker" dependency.

chamikaramj · 2023-08-14T20:20:13Z

sdks/python/test-suites/direct/xlang/build.gradle

Actually, these test can run with all Python versions. Updated.

(JavaUsingPython will be restricted to Python 3.8 I'll mention that later when we add such tests).

ahmedabu98

Just one comment. After that, this LGTM!

Most recent test pass (before spotless fix commit): https://ci-beam.apache.org/job/beam_PostCommit_TransformService_Direct_PR/44/

ahmedabu98 · 2023-08-15T20:41:24Z

buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy

I think we need to also depend on task initializeForDataflowJob to set up wheels when we eventually add dataflow tests:

beam/sdks/python/test-suites/dataflow/common.gradle

Line 41 in 369265d

task initializeForDataflowJob{

if (usesDataflowRunner) { dependsOn ":sdks:python:test-suites:dataflow:py${project.ext.pythonVersion.replace('.', '')}:initializeForDataflowJob" }

chamikaramj · 2023-08-15T23:23:00Z

Thanks!

damccorm · 2023-08-17T16:00:57Z

Looks like this broke https://github.com/apache/beam/actions/workflows/local_env_tests.yml?query=branch%3Amaster

damccorm · 2023-08-17T16:05:55Z

https://ci-beam.apache.org/job/beam_PostCommit_TransformService_Direct/ is also repeatedly failing

chamikaramj · 2023-08-17T16:10:29Z

#28039 should fix the "PostCommit_TransformService_Direct/ failure

github-actions bot added python build infra labels Jul 20, 2023

github-actions bot added the Next Action: Reviewers label Jul 20, 2023

chamikaramj marked this pull request as draft July 20, 2023 18:46

chamikaramj force-pushed the transform_service_test_suite branch from 15a6d6a to 8402eb7 Compare July 21, 2023 23:00

github-actions bot added the slow-review label Jul 29, 2023

github-actions bot added Next Action: Author and removed Next Action: Reviewers slow-review labels Jul 30, 2023

chamikaramj force-pushed the transform_service_test_suite branch from 8402eb7 to a0e6c48 Compare July 31, 2023 00:28

github-actions bot added Next Action: Reviewers and removed Next Action: Author labels Jul 31, 2023

github-actions bot removed io gcp labels Aug 9, 2023

github-actions bot added io gcp and removed io gcp labels Aug 9, 2023

chamikaramj marked this pull request as ready for review August 9, 2023 20:52

github-actions bot added io gcp labels Aug 10, 2023

ahmedabu98 reviewed Aug 14, 2023

View reviewed changes

chamikaramj commented Aug 14, 2023

View reviewed changes

ahmedabu98 reviewed Aug 15, 2023

View reviewed changes

Transform service test suite

a2f8bf9

chamikaramj force-pushed the transform_service_test_suite branch from 9e2f809 to a2f8bf9 Compare August 15, 2023 23:22

chamikaramj merged commit 618a14d into apache:master Aug 16, 2023

damccorm mentioned this pull request Aug 17, 2023

Revert transform service changes #28048

Closed

3 tasks

Transform service test suite #27578

Transform service test suite #27578

Uh oh!

Conversation

chamikaramj commented Jul 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

GitHub Actions Tests Status (on master branch)

Uh oh!

github-actions bot commented Jul 20, 2023

Uh oh!

chamikaramj commented Jul 20, 2023

Uh oh!

chamikaramj commented Jul 21, 2023

Uh oh!

codecov bot commented Jul 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

chamikaramj commented Jul 21, 2023

Uh oh!

github-actions bot commented Jul 29, 2023

Uh oh!

Abacn commented Jul 30, 2023

Uh oh!

chamikaramj commented Jul 31, 2023

Uh oh!

chamikaramj commented Jul 31, 2023

Uh oh!

chamikaramj commented Jul 31, 2023

Uh oh!

chamikaramj commented Jul 31, 2023

Uh oh!

chamikaramj commented Jul 31, 2023

Uh oh!

chamikaramj commented Jul 31, 2023

Uh oh!

chamikaramj commented Jul 31, 2023

Uh oh!

chamikaramj commented Jul 31, 2023

Uh oh!

chamikaramj commented Jul 31, 2023

Uh oh!

chamikaramj commented Jul 31, 2023

Uh oh!

chamikaramj commented Aug 9, 2023

Uh oh!

chamikaramj commented Aug 9, 2023

Uh oh!

chamikaramj commented Aug 9, 2023

Uh oh!

chamikaramj commented Aug 9, 2023

Uh oh!

chamikaramj commented Aug 9, 2023

Uh oh!

github-actions bot commented Aug 9, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chamikaramj Aug 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chamikaramj commented Aug 14, 2023

Uh oh!

chamikaramj commented Aug 14, 2023

Uh oh!

chamikaramj commented Aug 14, 2023

Uh oh!

chamikaramj commented Jul 20, 2023 •

edited

Loading

codecov bot commented Jul 21, 2023 •

edited

Loading

chamikaramj Aug 14, 2023 •

edited

Loading

chamikaramj Aug 14, 2023 •

edited

Loading