Skip to content

[Feature] Avoid logging full rows with sensitive information on conversion failure in Flink CDC #4290

@atallahade

Description

@atallahade

Search before asking

  • I searched in the issues and found nothing similar.

Motivation

Using a Flink CDC connector, when a row fails to convert according to the given schema, Paimon logs the entire row, which may contain sensitive information.
You can see this in CdcRecordUtils.java. For example:

2024-10-08 14:14:48,673 [] INFO  org.apache.paimon.flink.sink.cdc.CdcRecordUtils              [] - Failed to convert value <REDACTED_ROW> to type <REDACTED_SCHEMA>. Waiting for schema update.

Solution

Logging only the primary key could be a solution.

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions