Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [6.0.2] - 2026-02-19

### Fixed
- Fixed `dataframe_from_result_table` crashing with pandas 3.0 when a datetime column contains all null values.
- Removed legacy pandas 1.x code path in `parse_datetime` (minimum supported pandas is now 2.3.1).

## [6.0.1] - 2025-12-25

### Fixed
Expand Down
13 changes: 2 additions & 11 deletions azure-kusto-data/azure/kusto/data/helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -119,19 +119,10 @@
return frame[col]


def parse_datetime(frame, col, force_version: Optional[str] = None) -> "pd.Series":
# Pandas before version 2 doesn't support the "format" arg
def parse_datetime(frame, col) -> "pd.Series":
import pandas as pd

args = {}
if (force_version or pd.__version__).startswith("2."):
args = {"format": "ISO8601", "utc": True}
else:
# if frame contains ".", replace "Z" with ".000Z"
# Using bitwise NOT (~) on the boolean Series is the idiomatic pandas way to negate the mask
contains_dot = frame[col].str.contains("\\.")
frame.loc[~contains_dot, col] = frame.loc[~contains_dot, col].str.replace("Z", ".000Z")
frame[col] = pd.to_datetime(frame[col], errors="coerce", **args)
frame[col] = pd.to_datetime(frame[col], format="ISO8601", utc=True, errors="coerce")

Check warning on line 125 in azure-kusto-data/azure/kusto/data/helpers.py

View workflow job for this annotation

GitHub Actions / build (3.13)

Argument type is unknown   Argument corresponds to parameter "arg" in function "to_datetime" (reportUnknownArgumentType)

Check warning on line 125 in azure-kusto-data/azure/kusto/data/helpers.py

View workflow job for this annotation

GitHub Actions / build (3.10)

Argument type is unknown   Argument corresponds to parameter "arg" in function "to_datetime" (reportUnknownArgumentType)

Check warning on line 125 in azure-kusto-data/azure/kusto/data/helpers.py

View workflow job for this annotation

GitHub Actions / build (3.11)

Argument type is unknown   Argument corresponds to parameter "arg" in function "to_datetime" (reportUnknownArgumentType)

Check warning on line 125 in azure-kusto-data/azure/kusto/data/helpers.py

View workflow job for this annotation

GitHub Actions / build (3.12)

Argument type is unknown   Argument corresponds to parameter "arg" in function "to_datetime" (reportUnknownArgumentType)

Check warning on line 125 in azure-kusto-data/azure/kusto/data/helpers.py

View workflow job for this annotation

GitHub Actions / build (3.9)

Argument type is unknown   Argument corresponds to parameter "arg" in function "to_datetime" (reportUnknownArgumentType)
return frame[col]


Expand Down
2 changes: 1 addition & 1 deletion azure-kusto-data/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "azure-kusto-data"
version = "6.0.1"
version = "6.0.2"
description = "Kusto Data Client"
authors = [
{ name = "Microsoft Corporation", email = "kustalk@microsoft.com" },
Expand Down
44 changes: 27 additions & 17 deletions azure-kusto-data/tests/test_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
import pytest

from azure.kusto.data._models import KustoResultTable
from azure.kusto.data.helpers import dataframe_from_result_table, parse_datetime

Check warning on line 10 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.13)

Type of "parse_datetime" is partially unknown   Type of "parse_datetime" is "(frame: Unknown, col: Unknown) -> Series" (reportUnknownVariableType)

Check warning on line 10 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.10)

Type of "parse_datetime" is partially unknown   Type of "parse_datetime" is "(frame: Unknown, col: Unknown) -> Series" (reportUnknownVariableType)

Check warning on line 10 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.11)

Type of "parse_datetime" is partially unknown   Type of "parse_datetime" is "(frame: Unknown, col: Unknown) -> Series" (reportUnknownVariableType)

Check warning on line 10 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.12)

Type of "parse_datetime" is partially unknown   Type of "parse_datetime" is "(frame: Unknown, col: Unknown) -> Series" (reportUnknownVariableType)

Check warning on line 10 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.9)

Type of "parse_datetime" is partially unknown   Type of "parse_datetime" is "(frame: Unknown, col: Unknown) -> Series" (reportUnknownVariableType)
from azure.kusto.data.response import KustoResponseDataSetV2
import pandas
import numpy
Expand Down Expand Up @@ -131,28 +131,38 @@


def test_datetime_parsing():
"""Test parse_datetime function with different pandas versions and datetime formats"""

# Test with pandas v2 behavior (force version 2)
df_v2 = pandas.DataFrame(
"""Test parse_datetime correctly parses mixed datetime formats"""
df = pandas.DataFrame(
{
"mixed": ["2023-12-12T01:59:59.352Z", "2023-12-12T01:54:44Z"],
}
)

# Force pandas v2 behavior
result_v2 = parse_datetime(df_v2, "mixed", force_version="2.0.0")
assert str(result_v2[0]) == "2023-12-12 01:59:59.352000+00:00"
assert str(result_v2[1]) == "2023-12-12 01:54:44+00:00"
# Test with pandas v1 behavior (force version 1)
result = parse_datetime(df, "mixed")
assert str(result[0]) == "2023-12-12 01:59:59.352000+00:00"

Check warning on line 142 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.13)

Argument type is partially unknown   Argument corresponds to parameter "object" in function "__new__"   Argument type is "Series | Any | ndarray[Unknown, dtype[Any]] | Unknown" (reportUnknownArgumentType)

Check warning on line 142 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.10)

Argument type is partially unknown   Argument corresponds to parameter "object" in function "__new__"   Argument type is "Series | Any | ndarray[Unknown, Unknown] | Unknown" (reportUnknownArgumentType)

Check warning on line 142 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.11)

Argument type is partially unknown   Argument corresponds to parameter "object" in function "__new__"   Argument type is "Series | Any | ndarray[Unknown, dtype[Any]] | Unknown" (reportUnknownArgumentType)

Check warning on line 142 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.12)

Argument type is partially unknown   Argument corresponds to parameter "object" in function "__new__"   Argument type is "Series | Any | ndarray[Unknown, dtype[Any]] | Unknown" (reportUnknownArgumentType)

Check warning on line 142 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.9)

Argument type is partially unknown   Argument corresponds to parameter "object" in function "__new__"   Argument type is "Series | Any | ndarray[Any, Unknown] | Unknown | ndarray[Any, dtype[Unknown]]" (reportUnknownArgumentType)
assert str(result[1]) == "2023-12-12 01:54:44+00:00"

Check warning on line 143 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.13)

Argument type is partially unknown   Argument corresponds to parameter "object" in function "__new__"   Argument type is "Series | Any | ndarray[Unknown, dtype[Any]] | Unknown" (reportUnknownArgumentType)

Check warning on line 143 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.10)

Argument type is partially unknown   Argument corresponds to parameter "object" in function "__new__"   Argument type is "Series | Any | ndarray[Unknown, Unknown] | Unknown" (reportUnknownArgumentType)

Check warning on line 143 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.11)

Argument type is partially unknown   Argument corresponds to parameter "object" in function "__new__"   Argument type is "Series | Any | ndarray[Unknown, dtype[Any]] | Unknown" (reportUnknownArgumentType)

Check warning on line 143 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.12)

Argument type is partially unknown   Argument corresponds to parameter "object" in function "__new__"   Argument type is "Series | Any | ndarray[Unknown, dtype[Any]] | Unknown" (reportUnknownArgumentType)

Check warning on line 143 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.9)

Argument type is partially unknown   Argument corresponds to parameter "object" in function "__new__"   Argument type is "Series | Any | ndarray[Any, Unknown] | Unknown | ndarray[Any, dtype[Unknown]]" (reportUnknownArgumentType)

df_v1 = pandas.DataFrame(
{
"mixed": ["2023-12-12T01:59:59.352Z", "2023-12-12T01:54:44Z"],
}

def test_all_null_datetime_column():
"""Test dataframe_from_result_table with a column where all datetime values are null (pandas 3.0 regression)"""
df = dataframe_from_result_table(
KustoResultTable(
{
"TableName": "Table_0",
"Columns": [
{"ColumnName": "timestamp", "ColumnType": "datetime"},
{"ColumnName": "value", "ColumnType": "real"},
],
"Rows": [
[None, 10],
[None, 11],
],
}
)
)

# Force pandas v1 behavior - it should add .000 to dates without milliseconds
result_v1 = parse_datetime(df_v1, "mixed", force_version="1.5.3")
assert str(result_v1[0]) == "2023-12-12 01:59:59.352000+00:00"
assert str(result_v1[1]) == "2023-12-12 01:54:44+00:00"
assert len(df) == 2
assert pandas.isnull(df["timestamp"][0])

Check failure on line 165 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.13)

Invalid conditional operand of type "DataFrame | Series | Unknown"   Method __bool__ for type "DataFrame" returns type "NoReturn" rather than "bool"   Method __bool__ for type "Series" returns type "NoReturn" rather than "bool" (reportGeneralTypeIssues)

Check failure on line 165 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.10)

Invalid conditional operand of type "DataFrame | Series | Unknown"   Method __bool__ for type "DataFrame" returns type "NoReturn" rather than "bool"   Method __bool__ for type "Series" returns type "NoReturn" rather than "bool" (reportGeneralTypeIssues)

Check failure on line 165 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.11)

Invalid conditional operand of type "DataFrame | Series | Unknown"   Method __bool__ for type "DataFrame" returns type "NoReturn" rather than "bool"   Method __bool__ for type "Series" returns type "NoReturn" rather than "bool" (reportGeneralTypeIssues)

Check failure on line 165 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.12)

Invalid conditional operand of type "DataFrame | Series | Unknown"   Method __bool__ for type "DataFrame" returns type "NoReturn" rather than "bool"   Method __bool__ for type "Series" returns type "NoReturn" rather than "bool" (reportGeneralTypeIssues)

Check failure on line 165 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.9)

Invalid conditional operand of type "DataFrame | Series | NDArray[bool_]"   Method __bool__ for type "DataFrame" returns type "NoReturn" rather than "bool"   Method __bool__ for type "Series" returns type "NoReturn" rather than "bool" (reportGeneralTypeIssues)
assert pandas.isnull(df["timestamp"][1])

Check failure on line 166 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.13)

Invalid conditional operand of type "DataFrame | Series | Unknown"   Method __bool__ for type "DataFrame" returns type "NoReturn" rather than "bool"   Method __bool__ for type "Series" returns type "NoReturn" rather than "bool" (reportGeneralTypeIssues)

Check failure on line 166 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.10)

Invalid conditional operand of type "DataFrame | Series | Unknown"   Method __bool__ for type "DataFrame" returns type "NoReturn" rather than "bool"   Method __bool__ for type "Series" returns type "NoReturn" rather than "bool" (reportGeneralTypeIssues)

Check failure on line 166 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.11)

Invalid conditional operand of type "DataFrame | Series | Unknown"   Method __bool__ for type "DataFrame" returns type "NoReturn" rather than "bool"   Method __bool__ for type "Series" returns type "NoReturn" rather than "bool" (reportGeneralTypeIssues)

Check failure on line 166 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.12)

Invalid conditional operand of type "DataFrame | Series | Unknown"   Method __bool__ for type "DataFrame" returns type "NoReturn" rather than "bool"   Method __bool__ for type "Series" returns type "NoReturn" rather than "bool" (reportGeneralTypeIssues)

Check failure on line 166 in azure-kusto-data/tests/test_helpers.py

View workflow job for this annotation

GitHub Actions / build (3.9)

Invalid conditional operand of type "DataFrame | Series | NDArray[bool_]"   Method __bool__ for type "DataFrame" returns type "NoReturn" rather than "bool"   Method __bool__ for type "Series" returns type "NoReturn" rather than "bool" (reportGeneralTypeIssues)
assert df["value"][0] == 10
assert df["value"][1] == 11
2 changes: 1 addition & 1 deletion azure-kusto-ingest/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "azure-kusto-ingest"
version = "6.0.1"
version = "6.0.2"
description = "Kusto Ingest Client"
authors = [
{ name = "Microsoft Corporation", email = "kustalk@microsoft.com" },
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "azure-kusto-python"
version = "6.0.1"
version = "6.0.2"
description = "Microsoft Azure Kusto Python SDK"
dependencies = [
"azure-kusto-data",
Expand Down
2 changes: 1 addition & 1 deletion quick-start/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "quick-start"
version = "6.0.1"
version = "6.0.2"
description = "Microsoft Azure Kusto Python SDK"
dependencies = [
"azure-kusto-data",
Expand Down
Loading