Skip to content

KAFKA-7157: Fix handling of nulls in TimestampConverter#7070

Merged
rhauch merged 4 commits intoapache:trunkfrom
rayokota:Nimfadora-KAFKA-7157
Jul 12, 2019
Merged

KAFKA-7157: Fix handling of nulls in TimestampConverter#7070
rhauch merged 4 commits intoapache:trunkfrom
rayokota:Nimfadora-KAFKA-7157

Conversation

@rayokota
Copy link
Copy Markdown
Contributor

@rayokota rayokota commented Jul 10, 2019 •
edited
Loading

This is based on the PR from @Nimfadora, with some minor cleanup: #6446

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@rhauch
Copy link
Copy Markdown
Contributor

rhauch commented Jul 12, 2019

retest this please

rhauch
rhauch approved these changes Jul 12, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@rhauch rhauch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the cleanup, @rayokota. And thanks for the original PR, @Nimfadora.

LGTM, pending a green build. (I restarted the build since there were no previous build results; not sure why that was.)

@rhauch rhauch merged commit fa042bc into apache:trunk Jul 12, 2019
rhauch pushed a commit that referenced this pull request Jul 12, 2019
@rayokota @rhauch
KAFKA-7157: Fix handling of nulls in TimestampConverter (#7070)
813bf6c 
Fix handling of nulls in TimestampConverter.

Authors: Valeria Vasylieva <valeria.vasylieva@gmail.com>, Robert Yokota <rayokota@gmail.com>
Reviewers: Arjun Satish <arjun@confluent.io>, Randall Hauch <rhauch@gmail.com>
rhauch pushed a commit that referenced this pull request Jul 12, 2019
@rayokota @rhauch
KAFKA-7157: Fix handling of nulls in TimestampConverter (#7070)
10a72da 
Fix handling of nulls in TimestampConverter.

Authors: Valeria Vasylieva <valeria.vasylieva@gmail.com>, Robert Yokota <rayokota@gmail.com>
Reviewers: Arjun Satish <arjun@confluent.io>, Randall Hauch <rhauch@gmail.com>
rhauch pushed a commit that referenced this pull request Jul 12, 2019
@rayokota @rhauch
KAFKA-7157: Fix handling of nulls in TimestampConverter (#7070)
ccc751f 
Fix handling of nulls in TimestampConverter.

Authors: Valeria Vasylieva <valeria.vasylieva@gmail.com>, Robert Yokota <rayokota@gmail.com>
Reviewers: Arjun Satish <arjun@confluent.io>, Randall Hauch <rhauch@gmail.com>
rhauch pushed a commit that referenced this pull request Jul 12, 2019
@rayokota @rhauch
KAFKA-7157: Fix handling of nulls in TimestampConverter (#7070)
2e9d140 
Fix handling of nulls in TimestampConverter.

Authors: Valeria Vasylieva <valeria.vasylieva@gmail.com>, Robert Yokota <rayokota@gmail.com>
Reviewers: Arjun Satish <arjun@confluent.io>, Randall Hauch <rhauch@gmail.com>
rhauch pushed a commit that referenced this pull request Jul 12, 2019
@rayokota @rhauch
KAFKA-7157: Fix handling of nulls in TimestampConverter (#7070)
c05ed1e 
Fix handling of nulls in TimestampConverter.

Authors: Valeria Vasylieva <valeria.vasylieva@gmail.com>, Robert Yokota <rayokota@gmail.com>
Reviewers: Arjun Satish <arjun@confluent.io>, Randall Hauch <rhauch@gmail.com>
rhauch pushed a commit that referenced this pull request Jul 12, 2019
@rayokota @rhauch
KAFKA-7157: Fix handling of nulls in TimestampConverter (#7070)
de8663a 
Fix handling of nulls in TimestampConverter.

Authors: Valeria Vasylieva <valeria.vasylieva@gmail.com>, Robert Yokota <rayokota@gmail.com>
Reviewers: Arjun Satish <arjun@confluent.io>, Randall Hauch <rhauch@gmail.com>
rayokota added a commit to confluentinc/kafka that referenced this pull request Jul 16, 2019
@rayokota
KAFKA-7157: Fix handling of nulls in TimestampConverter (apache#7070)
5080ace 
Fix handling of nulls in TimestampConverter.

Authors: Valeria Vasylieva <valeria.vasylieva@gmail.com>, Robert Yokota <rayokota@gmail.com>
Reviewers: Arjun Satish <arjun@confluent.io>, Randall Hauch <rhauch@gmail.com>
ijuma added a commit to confluentinc/kafka that referenced this pull request Jul 20, 2019
@ijuma
Merge remote-tracking branch 'apache-github/2.3' into ccs-2.3
90e4b24 
* apache-github/2.3:
  MINOR: Update documentation for enabling optimizations (apache#7099)
  MINOR: Remove stale streams producer retry default docs. (apache#6844)
  KAFKA-8635; Skip client poll in Sender loop when no request is sent (apache#7085)
  KAFKA-8615: Change to track partition time breaks TimestampExtractor (apache#7054)
  KAFKA-8670; Fix exception for kafka-topics.sh --describe without --topic mentioned (apache#7094)
  KAFKA-8602: Separate PR for 2.3 branch (apache#7092)
  KAFKA-8530; Check for topic authorization errors in OffsetFetch response (apache#6928)
  KAFKA-8662; Fix producer metadata error handling and consumer manual assignment (apache#7086)
  KAFKA-8637: WriteBatch objects leak off-heap memory (apache#7050)
  KAFKA-8620: fix NPE due to race condition during shutdown while rebalancing (apache#7021)
  HOT FIX: close RocksDB objects in correct order (apache#7076)
  KAFKA-7157: Fix handling of nulls in TimestampConverter (apache#7070)
  KAFKA-6605: Fix NPE in Flatten when optional Struct is null (apache#5705)
  Fixes apache#8198 KStreams testing docs use non-existent method pipe (apache#6678)
  KAFKA-5998: fix checkpointableOffsets handling (apache#7030)
  KAFKA-8653; Default rebalance timeout to session timeout for JoinGroup v0 (apache#7072)
  KAFKA-8591; WorkerConfigTransformer NPE on connector configuration reloading (apache#6991)
  MINOR: add upgrade text (apache#7013)
  Bump version to 2.3.1-SNAPSHOT
xiowu0 pushed a commit to linkedin/kafka that referenced this pull request Aug 22, 2019
@rayokota @xiowu0
[LI-CHERRY-PICK] [813bf6c] KAFKA-7157: Fix handling of nulls in Times…
c3f7bc0 
…tampConverter (apache#7070)

TICKET = KAFKA-7157
LI_DESCRIPTION =
EXIT_CRITERIA = HASH [813bf6c]
ORIGINAL_DESCRIPTION =

Fix handling of nulls in TimestampConverter.

Authors: Valeria Vasylieva <valeria.vasylieva@gmail.com>, Robert Yokota <rayokota@gmail.com>
Reviewers: Arjun Satish <arjun@confluent.io>, Randall Hauch <rhauch@gmail.com>

(cherry picked from commit 813bf6c)
@kkonstantine kkonstantine mentioned this pull request Mar 21, 2020
Closed
3 tasks
@hamzanasirr
Copy link
Copy Markdown

hamzanasirr commented Apr 3, 2020

Hi, from where can I see how to implement this in a connector? Or is it done in the usual way?

@kkonstantine
Copy link
Copy Markdown
Contributor

kkonstantine commented Apr 6, 2020

@hamzanasirr not sure what are you looking to implement. This was a fix to a transformation (aka SMT).

@hamzanasirr
Copy link
Copy Markdown

hamzanasirr commented Apr 6, 2020 •
edited
Loading

@hamzanasirr not sure what are you looking to implement. This was a fix to a transformation (aka SMT).

Hi @kkonstantine,
I am using this SMT in one of my projects. I am using this on a date column to convert it to Date datatype from milliseconds coming in the form of string or int. But when it's null, it throws NPE. Even though it should not as it is fixed.

That's why I asked if, to implement this fix, do I need to add any additional config in my sink properties file?

Note: I am using the sink connector of MongoDB.

@ake1234
Copy link
Copy Markdown

ake1234 commented May 20, 2020

I'm also using this SMT but its throws null pointer exception for null timestamp values. Can someone help us how to implement this changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rhauch rhauch rhauch approved these changes

Assignees

No one assigned

Labels

connect

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@rayokota @rhauch @hamzanasirr @kkonstantine @ake1234