Skip to content

Conversation

@davorbonaci
Copy link
Member

Revert "mvn verify" back to "mvn install" since it breaks the followup script.

@davorbonaci
Copy link
Member Author

Unfortunately, this is not as easy. See Travis results here. Error:

The log length has exceeded the limit of 4 Megabytes (this usually means that test suite is raising the same exception over and over).

The build has been terminated.

@mxm
Copy link
Contributor

mxm commented Mar 7, 2016

The Flink Runner needs the verify phase to run the integration tests. The output on Travis needs to be reduced just to ERROR logging. Otherwise it will exceed the maximum log file length. For the Flink runner I've added a log4j-test.property which is loaded when the tests are executed.

@davorbonaci
Copy link
Member Author

R: @dhalperi

I've suppressed all [INFO] logs for the time being. The right fix is likely to decrease the log level of some of Flink logs from [INFO] to [DEBUG], but let's try to avoid that for now.

Let's see will this pass.

@dhalperi
Copy link
Contributor

dhalperi commented Mar 7, 2016

The grep command did not work -- Flink's INFO logs do not match the template you grep -v'ed. Plus some of these logs are many-lines-long where only the first line has INFO.

https://travis-ci.org/davorbonaci/beam/jobs/114395010

Test testJobWithObjectReuse(org.apache.beam.runners.flink.TfIdfITCase) successfully run.
================================================================================
16/03/07 23:48:58 INFO flink.TfIdfITCase: 
================================================================================
Test testJobCollectionExecution(org.apache.beam.runners.flink.TfIdfITCase) is running.
--------------------------------------------------------------------------------
16/03/07 23:48:58 INFO flink.FlinkPipelineExecutionEnvironment: Creating the required Batch Execution Environment.
16/03/07 23:48:58 INFO flink.FlinkPipelineRunner: Executing pipeline using FlinkPipelineRunner.
16/03/07 23:48:58 INFO flink.FlinkPipelineRunner: Translating pipeline to Flink program.
enterCompositeTransform- 32296114null
|   visitTransform- 3aa481c6Create.Values
|   enterCompositeTransform- 6440c3eTFIDF.ComputeTfIdf
|   |   enterCompositeTransform- 5d7fbda8Keys
|   |   |   visitTransform- 6d7c908cKeys [AnonymousParDo]
|   |   leaveCompositeTransform- 5d7fbda8Keys
<snip>

Revert "mvn verify" back to "mvn install" since it breaks the following script. When we are at it, update the script to be less sensitive whether artifacts are installed.

This doesn't fully fix the Travis coverage, since there's still too much logging generated in the 'verify' phase.
@davorbonaci
Copy link
Member Author

@dhalperi, ready for another look.

It fixes the interaction with the following script, but doesn't address the logging output in the 'verify' phase. Will file a JIRA issue for that.

@davorbonaci
Copy link
Member Author

JIRA for followup: https://issues.apache.org/jira/browse/BEAM-98

@dhalperi
Copy link
Contributor

dhalperi commented Mar 8, 2016

LGTM

@asfgit asfgit merged commit f8da287 into apache:master Mar 8, 2016
asfgit pushed a commit that referenced this pull request Mar 8, 2016
@davorbonaci davorbonaci deleted the travis branch March 8, 2016 00:52
dhalperi added a commit to GoogleCloudPlatform/DataflowJavaSDK that referenced this pull request Mar 8, 2016
@mxm
Copy link
Contributor

mxm commented Mar 8, 2016

We need mvn verify to properly execute all the tests. Changing back to "install" in my opinion is not an option. We should instead adjust the script which fails.

So another followup :) https://issues.apache.org/jira/browse/BEAM-99

edit: This is actually not true. install will also walk through the verify phase.

lukecwik referenced this pull request in lukecwik/incubator-beam Mar 22, 2018
mareksimunek pushed a commit to mareksimunek/beam that referenced this pull request May 9, 2018
dmvk pushed a commit to dmvk/beam that referenced this pull request May 15, 2018
tvalentyn pushed a commit to tvalentyn/beam that referenced this pull request May 15, 2018
dmvk referenced this pull request in seznam/beam Aug 17, 2018
kennknowles pushed a commit that referenced this pull request Oct 16, 2018
mxm pushed a commit to mxm/beam that referenced this pull request Jan 16, 2020
hengfengli referenced this pull request in hengfengli/beam Mar 21, 2022
Old values and new values of a mod can be any primitive type, not only
null or string. This fix allows for that.
sjvanrossum pushed a commit to sjvanrossum/beam that referenced this pull request May 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants