[Testing] Add slim java-test-image which can run most integration tests #9625
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Doesn't include CPP client, Python client, Pulsar SQL or Pulsar Connectors - Produces a relatively slim 750MB image (+461MB to base image) which doesn't depend on building pulsar and pulsar-all images like the current pulsar-test-latest-version image - Enables a lot quicker feedback loop when debugging and fixing integration tests locally since building the image takes less than 1.5 minutes on a modern laptop - using this image is also a preparation for speeding up CI where this slim image would be used to run the "core" integration tests which don't require CPP client, Python client, Pulsar SQL or Pulsar Connectors - Usage example to run single CLI integration test method against java-test-image: ./build/build_java_test_image.sh export PULSAR_TEST_IMAGE_NAME=apachepulsar/java-test-image:latest mvn -B -f tests/pom.xml test -DintegrationTests -DredirectTestOutputToFile=false -DtestRetryCount=0 -DfailIfNoTests=false -Dtest=CLITest#testCreateSubscriptionCommand
lhotari
force-pushed
the
lh-add-slim-java-test-image
branch
from
24c05b8 to
9c1e27a
Compare
zymap
requested review from
aahmed-se,
codelipenghui,
ivankelly,
jiazhai,
merlimat and
sijie
zymap
reviewed
Feb 19, 2021
Member
zymap
left a comment
zymap
left a comment
There was a problem hiding this comment.
That would be wonderful if you can paste your PR's description into the test README.md.
merlimat
approved these changes
Feb 19, 2021
zymap
approved these changes
Feb 20, 2021
jiazhai
approved these changes
Feb 21, 2021
sijie
approved these changes
Feb 22, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
The feedback loop in fixing and debugging integration tests is extremely slow at the moment because of the long time to build the pulsar-test-latest-version docker image and the images pulsar and pulsar-all that it depends on.
Introduce a slim docker image which can run most integration tests.
Modifications
Adds a maven module for building
java-test-image.Produces a relatively slim 750MB image (+461MB to base image) which doesn't
depend on building
pulsarandpulsar-allimages like the currentpulsar-test-latest-version image.Image doesn't include CPP client, Python client, Pulsar SQL or Pulsar Connectors since most integration tests don't require those.
Enables a lot quicker feedback loop when debugging and fixing integration tests
locally since building the image takes less than 1.5 minutes on a modern laptop.
Adding this image is also a preparation for speeding up CI where this slim image would be used to run the "core" integration tests which don't require CPP client, Python client, Pulsar SQL or Pulsar Connectors.
Usage example to run single CLI integration test method against java-test-image:
Adds changes to the integration test infrastructure so that the name of the docker image to use can be passed in
PULSAR_TEST_IMAGE_NAMEenvironment variable.In addition, adds
skipSourceReleaseAssemblymaven property which can be used to skip building theapache-pulsar-*-src.tar.gzwhen it's not needed. This is to speed up building of the docker image. It's used in the providedbuild/build_java_test_image.shscript.