Skip to content

[Task]: Only handle nulls for nullable columns in pandas to Beam row conversion #22948

@TheNeuralBit

Description

@TheNeuralBit

What needs to happen?

Currently when we convert a pandas DataFrame to Beam Rows we check for nulls in every column. This is wasteful, we should instead only do null-checking for the columns that we know to be nullable.

Issue Priority

Priority: 2

Issue Component

Component: dsl-dataframe

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions