Skip to content

Boolean operators in expressions are ignored #667

@timsaucer

Description

@timsaucer

Describe the bug

When attempting to create and expression using operators like and and or, no errors are reported but the resultant operations do not operate as expected. It appears the first expression is evaluated and others are ignored.

To Reproduce
This minimal code will reproduce the behavior:

ctx = SessionContext()

batch = pa.RecordBatch.from_arrays(
    [pa.array([1, 2, 3])],
    names=["a"],
)

df = ctx.create_dataframe([[batch]])

df.with_column("b", col("a") == lit(1) or col("a") == lit(3)).show()
df.with_column("b", col("a") == lit(3) or col("a") == lit(1)).show()

This generates the following results:

DataFrame()
+---+-------+
| a | b     |
+---+-------+
| 1 | true  |
| 2 | false |
| 3 | false |
+---+-------+
DataFrame()
+---+-------+
| a | b     |
+---+-------+
| 1 | false |
| 2 | false |
| 3 | true  |
+---+-------+

Expected behavior
If these types of operations are not supported, an error should be generated. Even better would be to fully support these operations since it will mean a great deal for adoption across the python community.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions