Skip to content

Conversation

@Polber
Copy link
Contributor

@Polber Polber commented Jul 18, 2024

Apache Beam depends on Jackson 2.15.4 as of apache/beam#31473 which is breaking at least the JMS To PubSub template when deserializing pipeline options due to

java.lang.NoSuchFieldError: READ_UNKNOWN_ENUM_VALUES_USING_DEFAULT_VALUE
  at com.fasterxml.jackson.databind.deser.std.EnumDeserializer.createContextual(EnumDeserializer.java:211)
  ...
  at org.apache.beam.sdk.options.PipelineOptionsFactory.computeDeserializerForMethod(PipelineOptionsFactory.java:1739)
  ...

which was a field added in 2.15.4

@Polber Polber requested review from Abacn and fozzie15 July 18, 2024 23:29
@Polber Polber self-assigned this Jul 18, 2024
Abacn
Abacn previously approved these changes Jul 18, 2024
@Polber Polber marked this pull request as draft July 18, 2024 23:32
@Polber
Copy link
Contributor Author

Polber commented Jul 18, 2024

Not sure why checks aren't running. Looking into it

@Polber
Copy link
Contributor Author

Polber commented Jul 18, 2024

Might be this https://www.githubstatus.com/

@Abacn
Copy link
Contributor

Abacn commented Jul 18, 2024

wait, at least need to add this line in several places per #1681

RowJsonUtils.increaseDefaultStreamReadConstraints(100 * 1024 * 1024);

@Polber
Copy link
Contributor Author

Polber commented Jul 18, 2024

Job is queued: https://github.com/GoogleCloudPlatform/DataflowTemplates/actions/runs/10000088771
Definitely a github issue. Don't love that it doesn't block submission

@Polber Polber force-pushed the jkinard/jackson branch from 03aff41 to 8bf042d Compare July 19, 2024 19:11
Signed-off-by: Jeffrey Kinard <jeff@thekinards.com>
@Polber Polber force-pushed the jkinard/jackson branch from 8bf042d to 84d72c6 Compare July 19, 2024 19:11
@Polber Polber marked this pull request as ready for review July 19, 2024 19:12
@Abacn
Copy link
Contributor

Abacn commented Jul 19, 2024

there will be integration test failures which I observed during 2.57.0rc1 validation, need to fix some code path, see #1739 (comment)

@codecov
Copy link

codecov bot commented Jul 19, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 42.31%. Comparing base (d7db191) to head (84d72c6).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1739      +/-   ##
============================================
- Coverage     42.32%   42.31%   -0.01%     
+ Complexity     3182     3180       -2     
============================================
  Files           794      794              
  Lines         46244    46244              
  Branches       4951     4951              
============================================
- Hits          19572    19568       -4     
- Misses        25074    25077       +3     
- Partials       1598     1599       +1     
Components Coverage Δ
spanner-templates 63.69% <ø> (-0.01%) ⬇️
spanner-import-export 64.42% <ø> (-0.03%) ⬇️
spanner-live-forward-migration 74.97% <ø> (ø)
spanner-live-reverse-replication 51.84% <ø> (ø)
spanner-bulk-migration 82.79% <ø> (ø)

see 2 files with indirect coverage changes

@Polber Polber requested a review from Abacn July 22, 2024 16:58
@Polber
Copy link
Contributor Author

Polber commented Jul 22, 2024

there will be integration test failures which I observed during 2.57.0rc1 validation, need to fix some code path, see #1739 (comment)

@Abacn I already included these changes when I upgraded beam in #1695.

IT's passed other than a couple Kafka IT's which have already been failing - I'm looking into those, but error is unrelated to the jackson upgrade.

@Polber Polber merged commit 5be5817 into GoogleCloudPlatform:main Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants