From d887bc823de7971176f6adf9df56cdb272213f65 Mon Sep 17 00:00:00 2001 From: Wei Qi Lu Date: Thu, 16 Apr 2026 10:54:21 -0700 Subject: [PATCH 01/12] remove random pr link --- python/CHANGELOG.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/python/CHANGELOG.md b/python/CHANGELOG.md index 0920923b4..32f0bfc07 100644 --- a/python/CHANGELOG.md +++ b/python/CHANGELOG.md @@ -3,9 +3,6 @@ All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/). - -https://github.com/sift-stack/sift/commit/e1305363e95cafb4c980fdd493f69e72660a52fb - ## [v0.13.0] - March 24, 2026 ### What's New #### SiftClient Export API From 83677b5d91eb85a7a6d92dbc24c0ea518d60f818 Mon Sep 17 00:00:00 2001 From: Wei Qi Lu Date: Thu, 16 Apr 2026 12:13:53 -0700 Subject: [PATCH 02/12] initial changelog draft --- python/CHANGELOG.md | 75 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) diff --git a/python/CHANGELOG.md b/python/CHANGELOG.md index 32f0bfc07..70343609d 100644 --- a/python/CHANGELOG.md +++ b/python/CHANGELOG.md @@ -3,6 +3,81 @@ All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/). +## [v1.0.0] - April 16, 2026 + +This is the first major release of `sift-stack-py`. `sift_client` has reached feature parity with `sift_py` and is now the recommended interface for all new development. `sift_py` (deprecated since [v0.10.0](#v0100---january-30-2026)) continues to work and ship in this release; all features previously available only in `sift_py`, including data imports and exports, are now available in `sift_client`. + +### Migrating from `sift_py` to `sift_client` + +`sift_client` introduces a unified, Pythonic interface and is not a drop-in replacement for `sift_py`. The major differences are summarized below. For complete examples, see the [sift_client documentation](https://sift-stack.github.io/sift/python/latest/#sift-client-api-new). + +#### Unified client vs. per-service classes + +`sift_py` exposed a separate service class per domain (`IngestionService`, `RuleService`, `AssetService`, `ReportTemplateService`, etc.). `sift_client` exposes a single `SiftClient` with resource accessors: + +```python +# sift_py +from sift_py.grpc.transport import SiftChannelConfig, use_sift_channel +from sift_py.asset.service import AssetService + +with use_sift_channel(SiftChannelConfig(uri=uri, apikey=apikey)) as channel: + asset_service = AssetService(channel) + asset = asset_service.get_asset(asset_id="asset123") + +# sift_client +from sift_client import SiftClient + +client = SiftClient(api_key=apikey, grpc_url=grpc_url, rest_url=rest_url) +asset = client.assets.get(asset_id="asset123") +``` + +#### Sync and async interfaces + +`sift_py` is primarily synchronous, with async support limited to data queries and the gRPC channel. `sift_client` provides both sync and async interfaces for every resource: + +```python +# Sync +asset = client.assets.get(asset_id="asset123") + +# Async +asset = await client.async_.assets.get(asset_id="asset123") +``` + +#### Typed Pydantic models with methods + +`sift_py` returned a mix of raw protobuf types and thin wrappers, with inconsistent update semantics. `sift_client` returns immutable Pydantic models with convenience methods and typed update models: + +```python +asset = client.assets.get(asset_id="asset123") +asset.archive(archive_runs=True) +asset.update({"tags": ["production", "v2"]}) +``` + +### What's New + +#### Data Import API in SiftClient +The `sift_client` module now exposes a data import API supporting CSV, Parquet, TDMS, and HDF5. + +#### Test Result Logging +Adds optional logging of test result create and update events, with logging now running in a subprocess while a test runs. + +#### Progress Indicators +Adds progress indicators for job polling and file downloads for better visibility during long-running operations. + +### Bugfixes +- Fall back to `application/octet-stream` for unknown MIME types on file attachment uploads. +- Add `py.typed` to the generated proto directory so type checkers pick up protobuf types correctly. +- Update `sift-stream-bindings` to pick up upstream fixes. + +### Full Changelog +- [Add data import API to sift_client](https://github.com/sift-stack/sift/pull/515) +- [Add optional logging of test result create and update events](https://github.com/sift-stack/sift/pull/529) +- [Add progress indicators for job polling and file downloads](https://github.com/sift-stack/sift/pull/517) +- [Refactor files using run_in_executor](https://github.com/sift-stack/sift/pull/518) +- [Update exports.proto to support parquet](https://github.com/sift-stack/sift/pull/510) +- [Add py.typed file to proto dir](https://github.com/sift-stack/sift/pull/524) +- [Update sift-stream-bindings version](https://github.com/sift-stack/sift/pull/513) + ## [v0.13.0] - March 24, 2026 ### What's New #### SiftClient Export API From ccff2e0e4d55becfe9b999af90cbef506bf4d3a0 Mon Sep 17 00:00:00 2001 From: Wei Qi Lu Date: Thu, 16 Apr 2026 22:02:15 -0700 Subject: [PATCH 03/12] update changelog to minor --- python/CHANGELOG.md | 55 +++------------------------------------------ 1 file changed, 3 insertions(+), 52 deletions(-) diff --git a/python/CHANGELOG.md b/python/CHANGELOG.md index 70343609d..700e4f620 100644 --- a/python/CHANGELOG.md +++ b/python/CHANGELOG.md @@ -3,69 +3,20 @@ All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/). -## [v1.0.0] - April 16, 2026 - -This is the first major release of `sift-stack-py`. `sift_client` has reached feature parity with `sift_py` and is now the recommended interface for all new development. `sift_py` (deprecated since [v0.10.0](#v0100---january-30-2026)) continues to work and ship in this release; all features previously available only in `sift_py`, including data imports and exports, are now available in `sift_client`. - -### Migrating from `sift_py` to `sift_client` - -`sift_client` introduces a unified, Pythonic interface and is not a drop-in replacement for `sift_py`. The major differences are summarized below. For complete examples, see the [sift_client documentation](https://sift-stack.github.io/sift/python/latest/#sift-client-api-new). - -#### Unified client vs. per-service classes - -`sift_py` exposed a separate service class per domain (`IngestionService`, `RuleService`, `AssetService`, `ReportTemplateService`, etc.). `sift_client` exposes a single `SiftClient` with resource accessors: - -```python -# sift_py -from sift_py.grpc.transport import SiftChannelConfig, use_sift_channel -from sift_py.asset.service import AssetService - -with use_sift_channel(SiftChannelConfig(uri=uri, apikey=apikey)) as channel: - asset_service = AssetService(channel) - asset = asset_service.get_asset(asset_id="asset123") - -# sift_client -from sift_client import SiftClient - -client = SiftClient(api_key=apikey, grpc_url=grpc_url, rest_url=rest_url) -asset = client.assets.get(asset_id="asset123") -``` - -#### Sync and async interfaces - -`sift_py` is primarily synchronous, with async support limited to data queries and the gRPC channel. `sift_client` provides both sync and async interfaces for every resource: - -```python -# Sync -asset = client.assets.get(asset_id="asset123") - -# Async -asset = await client.async_.assets.get(asset_id="asset123") -``` - -#### Typed Pydantic models with methods - -`sift_py` returned a mix of raw protobuf types and thin wrappers, with inconsistent update semantics. `sift_client` returns immutable Pydantic models with convenience methods and typed update models: - -```python -asset = client.assets.get(asset_id="asset123") -asset.archive(archive_runs=True) -asset.update({"tags": ["production", "v2"]}) -``` +## [v0.14.0] - April 17, 2026 ### What's New #### Data Import API in SiftClient -The `sift_client` module now exposes a data import API supporting CSV, Parquet, TDMS, and HDF5. +The `sift_client` module now exposes a data import API supporting CSV, Parquet, TDMS, and HDF5. With this addition, all features previously available only in `sift_py` are now available in `sift_client`, which remains the recommended interface for new development. `sift_py` (deprecated since [v0.10.0](#v0100---january-30-2026)) continues to work and ship in this release. #### Test Result Logging -Adds optional logging of test result create and update events, with logging now running in a subprocess while a test runs. +Test result create and update events can now be optionally written to a `.jsonl` log file during a test run, then replayed against the Sift API later via the new `import_test_result_log` script. #### Progress Indicators Adds progress indicators for job polling and file downloads for better visibility during long-running operations. ### Bugfixes -- Fall back to `application/octet-stream` for unknown MIME types on file attachment uploads. - Add `py.typed` to the generated proto directory so type checkers pick up protobuf types correctly. - Update `sift-stream-bindings` to pick up upstream fixes. From ae77a74db46c5187b29a5700a9343d7703203946 Mon Sep 17 00:00:00 2001 From: Wei Qi Lu Date: Thu, 16 Apr 2026 22:25:14 -0700 Subject: [PATCH 04/12] import migration explanation --- python/CHANGELOG.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/python/CHANGELOG.md b/python/CHANGELOG.md index 700e4f620..7ebc86b89 100644 --- a/python/CHANGELOG.md +++ b/python/CHANGELOG.md @@ -10,6 +10,17 @@ This project adheres to [Semantic Versioning](http://semver.org/). #### Data Import API in SiftClient The `sift_client` module now exposes a data import API supporting CSV, Parquet, TDMS, and HDF5. With this addition, all features previously available only in `sift_py` are now available in `sift_client`, which remains the recommended interface for new development. `sift_py` (deprecated since [v0.10.0](#v0100---january-30-2026)) continues to work and ship in this release. +Migrating from `sift_py`: the per-format upload services are replaced by a single `client.data_imports.import_from_path` method. Where `sift_py` required hand-building a config object for every import, `sift_client` auto-detects it from the file extension. Use the new public `client.data_imports.detect_config(...)` first if you need to inspect or patch the result. The call returns a Job you can optionally wait on. + +```python +# sift_py +csv_service = CsvUploadService(rest_conf) +csv_service.upload("data.csv", csv_config) + +# sift_client +client.data_imports.import_from_path("data.csv", asset=my_asset) +``` + #### Test Result Logging Test result create and update events can now be optionally written to a `.jsonl` log file during a test run, then replayed against the Sift API later via the new `import_test_result_log` script. From fe03283140ee2b8afee8f92f08e71546002aad61 Mon Sep 17 00:00:00 2001 From: Wei Qi Lu Date: Fri, 17 Apr 2026 15:31:22 -0700 Subject: [PATCH 05/12] update dates and version --- python/CHANGELOG.md | 2 +- python/lib/sift_client/resources/sync_stubs/__init__.pyi | 8 +++----- python/pyproject.toml | 2 +- 3 files changed, 5 insertions(+), 7 deletions(-) diff --git a/python/CHANGELOG.md b/python/CHANGELOG.md index 7ebc86b89..180a7cc7c 100644 --- a/python/CHANGELOG.md +++ b/python/CHANGELOG.md @@ -3,7 +3,7 @@ All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/). -## [v0.14.0] - April 17, 2026 +## [v0.14.0] - April 20, 2026 ### What's New diff --git a/python/lib/sift_client/resources/sync_stubs/__init__.pyi b/python/lib/sift_client/resources/sync_stubs/__init__.pyi index 95dd37cdf..a10e0ac52 100644 --- a/python/lib/sift_client/resources/sync_stubs/__init__.pyi +++ b/python/lib/sift_client/resources/sync_stubs/__init__.pyi @@ -653,9 +653,8 @@ class DataImportAPI: is inferred from the file extension when ``data_type`` is not provided. - CSV, Parquet, and HDF5 files are supported for auto-detection. - For other formats (TDMS), create the config manually - using ``TdmsImportConfig``. + CSV, Parquet, HDF5, and TDMS files are supported for + auto-detection. For CSV files, the server scans the first two rows for an optional JSON metadata row. Row 1 is checked first; row 2 is checked only @@ -733,8 +732,7 @@ class DataImportAPI: completion before proceeding. When ``config`` is omitted the file format is auto-detected via - ``detect_config`` (CSV, Parquet, and HDF5). For other formats - (TDMS), ``config`` must be provided. + ``detect_config`` (CSV, Parquet, HDF5, and TDMS). When ``asset`` is provided it overrides the config value; otherwise the config's ``asset_name`` is used. If neither ``run`` nor ``run_name`` is provided (and none is diff --git a/python/pyproject.toml b/python/pyproject.toml index 1ef8fdb2c..f4255f672 100644 --- a/python/pyproject.toml +++ b/python/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "sift_stack_py" -version = "0.13.0" +version = "0.14.0" description = "Python client library for the Sift API" requires-python = ">=3.8" readme = { file = "README.md", content-type = "text/markdown" } From e9b34d0f79c8524899444bdd08ca13757ed9c949 Mon Sep 17 00:00:00 2001 From: Wei Qi Lu Date: Wed, 22 Apr 2026 11:28:12 -0700 Subject: [PATCH 06/12] include hdf5 and tdms PR links, update import example --- python/CHANGELOG.md | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/python/CHANGELOG.md b/python/CHANGELOG.md index 180a7cc7c..1c6f5e5de 100644 --- a/python/CHANGELOG.md +++ b/python/CHANGELOG.md @@ -3,42 +3,50 @@ All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/). -## [v0.14.0] - April 20, 2026 +## [v0.14.0] - April 22, 2026 ### What's New #### Data Import API in SiftClient The `sift_client` module now exposes a data import API supporting CSV, Parquet, TDMS, and HDF5. With this addition, all features previously available only in `sift_py` are now available in `sift_client`, which remains the recommended interface for new development. `sift_py` (deprecated since [v0.10.0](#v0100---january-30-2026)) continues to work and ship in this release. -Migrating from `sift_py`: the per-format upload services are replaced by a single `client.data_imports.import_from_path` method. Where `sift_py` required hand-building a config object for every import, `sift_client` auto-detects it from the file extension. Use the new public `client.data_imports.detect_config(...)` first if you need to inspect or patch the result. The call returns a Job you can optionally wait on. - +Migrating from `sift_py`: the per-format upload services (`CsvUploadService`, `ParquetUploadService`, `Hdf5UploadService`, `TdmsUploadService`) are replaced by a single `client.data_import.import_from_path` method. Where `sift_py` required hand-building a config object for every import, `sift_client` auto-detects it from the file extension. Use the new public `client.data_import.detect_config(...)` first if you need to inspect or patch the result before importing. `import_from_path` returns a `Job` you can optionally wait on. ```python -# sift_py +# sift_py (deprecated) +from sift_py.data_import.csv import CsvUploadService +from sift_py.rest import SiftRestConfig + +rest_conf: SiftRestConfig = {"uri": sift_uri, "apikey": apikey} csv_service = CsvUploadService(rest_conf) -csv_service.upload("data.csv", csv_config) +import_service = csv_service.simple_upload(asset_name="my_asset", path="data.csv") +import_service.wait_until_complete() # sift_client -client.data_imports.import_from_path("data.csv", asset=my_asset) +from sift_client import SiftClient + +client = SiftClient(api_key=apikey, grpc_url=grpc_url, rest_url=rest_url) +job = client.data_import.import_from_path("data.csv", asset="my_asset") +job.wait_until_complete() ``` #### Test Result Logging -Test result create and update events can now be optionally written to a `.jsonl` log file during a test run, then replayed against the Sift API later via the new `import_test_result_log` script. +Test result create and update events can now be optionally written to a `.jsonl` log file during a test run, then replayed against the Sift API later via the new `import-test-result-log` CLI command (installed with the package). #### Progress Indicators Adds progress indicators for job polling and file downloads for better visibility during long-running operations. ### Bugfixes - Add `py.typed` to the generated proto directory so type checkers pick up protobuf types correctly. -- Update `sift-stream-bindings` to pick up upstream fixes. ### Full Changelog - [Add data import API to sift_client](https://github.com/sift-stack/sift/pull/515) +- [Add HDF5 client-side detect_config](https://github.com/sift-stack/sift/pull/536) +- [Add TDMS client-side detect_config](https://github.com/sift-stack/sift/pull/538) - [Add optional logging of test result create and update events](https://github.com/sift-stack/sift/pull/529) - [Add progress indicators for job polling and file downloads](https://github.com/sift-stack/sift/pull/517) - [Refactor files using run_in_executor](https://github.com/sift-stack/sift/pull/518) - [Update exports.proto to support parquet](https://github.com/sift-stack/sift/pull/510) - [Add py.typed file to proto dir](https://github.com/sift-stack/sift/pull/524) -- [Update sift-stream-bindings version](https://github.com/sift-stack/sift/pull/513) ## [v0.13.0] - March 24, 2026 ### What's New From 78054542d6a1b8457e5fa141583ae7c444885023 Mon Sep 17 00:00:00 2001 From: Wei Qi Lu Date: Wed, 22 Apr 2026 14:00:42 -0700 Subject: [PATCH 07/12] include parquet export and more detail on file formats --- python/CHANGELOG.md | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/python/CHANGELOG.md b/python/CHANGELOG.md index 1c6f5e5de..0391e9a18 100644 --- a/python/CHANGELOG.md +++ b/python/CHANGELOG.md @@ -29,6 +29,31 @@ job = client.data_import.import_from_path("data.csv", asset="my_asset") job.wait_until_complete() ``` +Format-by-format support: +- **CSV**: auto-detected from `.csv`. Supports an optional JSON metadata row (row 1 or row 2) for specifying channel names, units, data types, and the time column format. +- **Parquet**: requires an explicit `data_type` (`PARQUET_FLATDATASET` or `PARQUET_SINGLE_CHANNEL_PER_ROW`) since `.parquet` alone doesn't disambiguate the layout. Detection only reads the file footer, so it stays fast on large files. +- **HDF5**: new in this release. Auto-detected from `.h5` / `.hdf5`. Detection runs fully client-side (no server round-trip) and walks each dataset. +- **TDMS**: new in this release. Auto-detected from `.tdms`. Detection is also fully client-side and maps TDMS groups and channels onto Sift channels. + +#### Parquet as an Export Output Format +`client.data_export.export(...)` now accepts `ExportOutputFormat.PARQUET` alongside the existing CSV and Sun/WinPlot options. Unlike the `sift_py` `DataService` + `DataFrame.to_parquet()` pattern (which buffers everything in memory), this runs as a server-side job and scales to large exports. + +```python +from sift_client import SiftClient +from sift_client.sift_types.export import ExportOutputFormat + +client = SiftClient(api_key=apikey, grpc_url=grpc_url, rest_url=rest_url) +job = client.data_export.export( + runs=["run-id"], + start_time=start, + stop_time=stop, + output_format=ExportOutputFormat.PARQUET, +) +files = job.wait_and_download(output_dir="./exports") +``` + +`wait_and_download` polls the job to completion, downloads the resulting archive, and returns the list of downloaded file paths. Both arguments shown are optional: if `output_dir` is omitted, a temporary directory is created and used. By default the downloaded zip is extracted (and the archive deleted) so you get the Parquet files directly; pass `extract=False` to keep the zip instead, which is useful if you want to hand the whole bundle off to another system without unpacking it client-side. + #### Test Result Logging Test result create and update events can now be optionally written to a `.jsonl` log file during a test run, then replayed against the Sift API later via the new `import-test-result-log` CLI command (installed with the package). @@ -45,7 +70,7 @@ Adds progress indicators for job polling and file downloads for better visibilit - [Add optional logging of test result create and update events](https://github.com/sift-stack/sift/pull/529) - [Add progress indicators for job polling and file downloads](https://github.com/sift-stack/sift/pull/517) - [Refactor files using run_in_executor](https://github.com/sift-stack/sift/pull/518) -- [Update exports.proto to support parquet](https://github.com/sift-stack/sift/pull/510) +- [Add Parquet as an export output format](https://github.com/sift-stack/sift/pull/510) - [Add py.typed file to proto dir](https://github.com/sift-stack/sift/pull/524) ## [v0.13.0] - March 24, 2026 From 9e7da959d3c5c3e5587a65e91f171ddcc46dc819 Mon Sep 17 00:00:00 2001 From: Wei Qi Lu Date: Wed, 22 Apr 2026 14:13:31 -0700 Subject: [PATCH 08/12] added parquet export example for deprecated sift_py --- python/CHANGELOG.md | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/python/CHANGELOG.md b/python/CHANGELOG.md index 0391e9a18..d53875197 100644 --- a/python/CHANGELOG.md +++ b/python/CHANGELOG.md @@ -36,15 +36,32 @@ Format-by-format support: - **TDMS**: new in this release. Auto-detected from `.tdms`. Detection is also fully client-side and maps TDMS groups and channels onto Sift channels. #### Parquet as an Export Output Format -`client.data_export.export(...)` now accepts `ExportOutputFormat.PARQUET` alongside the existing CSV and Sun/WinPlot options. Unlike the `sift_py` `DataService` + `DataFrame.to_parquet()` pattern (which buffers everything in memory), this runs as a server-side job and scales to large exports. +`client.data_export.export(...)` now accepts `ExportOutputFormat.PARQUET` alongside the existing CSV and Sun/WinPlot options. Unlike the `sift_py` `DataService` + `DataFrame.to_parquet()` pattern (async-only, buffers everything in memory, name-strings only), the new export API runs as a server-side job, works sync or async, accepts `Asset`/`Channel` objects or IDs, and scales to large exports. ```python +# sift_py (deprecated): no dedicated export API, so query in-memory and write yourself +import pandas as pd +from sift_py.data.query import ChannelQuery, DataQuery +from sift_py.data.service import DataService +from sift_py.grpc.transport import use_sift_async_channel + +async with use_sift_async_channel({"uri": sift_uri, "apikey": apikey}) as channel: + result = await DataService(channel).execute(DataQuery( + asset_name="my_asset", + start_time=start, + end_time=stop, + channels=[ChannelQuery(channel_name="my_channel")], + )) + pd.DataFrame(result.all_channels()[0].columns()).to_parquet("out.parquet") + +# sift_client from sift_client import SiftClient from sift_client.sift_types.export import ExportOutputFormat client = SiftClient(api_key=apikey, grpc_url=grpc_url, rest_url=rest_url) job = client.data_export.export( - runs=["run-id"], + assets=["my_asset"], # accepts Asset objects or IDs + channels=["my_channel"], # accepts Channel objects or IDs start_time=start, stop_time=stop, output_format=ExportOutputFormat.PARQUET, From 12649a5b20eebd8e8567162a7726010b14bc820a Mon Sep 17 00:00:00 2001 From: Wei Qi Lu Date: Wed, 22 Apr 2026 15:24:37 -0700 Subject: [PATCH 09/12] add detail on hdf5/tdms imports, update detect_config context --- python/CHANGELOG.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/python/CHANGELOG.md b/python/CHANGELOG.md index d53875197..e76fc3b01 100644 --- a/python/CHANGELOG.md +++ b/python/CHANGELOG.md @@ -10,7 +10,7 @@ This project adheres to [Semantic Versioning](http://semver.org/). #### Data Import API in SiftClient The `sift_client` module now exposes a data import API supporting CSV, Parquet, TDMS, and HDF5. With this addition, all features previously available only in `sift_py` are now available in `sift_client`, which remains the recommended interface for new development. `sift_py` (deprecated since [v0.10.0](#v0100---january-30-2026)) continues to work and ship in this release. -Migrating from `sift_py`: the per-format upload services (`CsvUploadService`, `ParquetUploadService`, `Hdf5UploadService`, `TdmsUploadService`) are replaced by a single `client.data_import.import_from_path` method. Where `sift_py` required hand-building a config object for every import, `sift_client` auto-detects it from the file extension. Use the new public `client.data_import.detect_config(...)` first if you need to inspect or patch the result before importing. `import_from_path` returns a `Job` you can optionally wait on. +Migrating from `sift_py`: the per-format upload services (`CsvUploadService`, `ParquetUploadService`, `Hdf5UploadService`, `TdmsUploadService`) collapse into a single `client.data_import.import_from_path` method. `sift_py` only auto-detected for CSV via `simple_upload`; other formats required more setup. `sift_client` unifies all four with auto-detection built into `import_from_path` itself: the `config` argument is optional, so the common call takes just a file path and target asset. Call `client.data_import.detect_config(...)` first if you want to inspect or patch the config before importing. `import_from_path` returns a `Job` you can optionally wait on. ```python # sift_py (deprecated) from sift_py.data_import.csv import CsvUploadService @@ -32,8 +32,8 @@ job.wait_until_complete() Format-by-format support: - **CSV**: auto-detected from `.csv`. Supports an optional JSON metadata row (row 1 or row 2) for specifying channel names, units, data types, and the time column format. - **Parquet**: requires an explicit `data_type` (`PARQUET_FLATDATASET` or `PARQUET_SINGLE_CHANNEL_PER_ROW`) since `.parquet` alone doesn't disambiguate the layout. Detection only reads the file footer, so it stays fast on large files. -- **HDF5**: new in this release. Auto-detected from `.h5` / `.hdf5`. Detection runs fully client-side (no server round-trip) and walks each dataset. -- **TDMS**: new in this release. Auto-detected from `.tdms`. Detection is also fully client-side and maps TDMS groups and channels onto Sift channels. +- **HDF5**: new in this release. Auto-detected from `.h5` / `.hdf5`. Detection works out-of-the-box for files with a compound-dataset layout (first field = time) or a shared root-level `time` dataset; other layouts may need a hand-built `config`. +- **TDMS**: new in this release. Auto-detected from `.tdms` and recognizes the common timing conventions (group-level `xchannel`, first-channel `TimeStamp`, or per-channel waveform properties). `TdmsImportConfig` controls handling of untimed channels (`fallback_method`), complex values (`complex_component`), scaled vs. raw data, and waveform start-time overrides. #### Parquet as an Export Output Format `client.data_export.export(...)` now accepts `ExportOutputFormat.PARQUET` alongside the existing CSV and Sun/WinPlot options. Unlike the `sift_py` `DataService` + `DataFrame.to_parquet()` pattern (async-only, buffers everything in memory, name-strings only), the new export API runs as a server-side job, works sync or async, accepts `Asset`/`Channel` objects or IDs, and scales to large exports. From abbca721833ce565da7a3f08f786a93cf76f6fec Mon Sep 17 00:00:00 2001 From: Wei Qi Lu Date: Mon, 27 Apr 2026 12:43:44 -0700 Subject: [PATCH 10/12] update changelog to include the module refactor --- python/CHANGELOG.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/python/CHANGELOG.md b/python/CHANGELOG.md index e76fc3b01..534f5edbc 100644 --- a/python/CHANGELOG.md +++ b/python/CHANGELOG.md @@ -3,7 +3,7 @@ All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/). -## [v0.14.0] - April 22, 2026 +## [v0.14.0] - April 27, 2026 ### What's New @@ -77,6 +77,9 @@ Test result create and update events can now be optionally written to a `.jsonl` #### Progress Indicators Adds progress indicators for job polling and file downloads for better visibility during long-running operations. +#### Use Upstream Packages for `google.api` and `protoc_gen_openapiv2` +Previously the package shipped its own bundled copies of the generated Python bindings for `google.api` and `protoc_gen_openapiv2`, which could collide with the same modules installed elsewhere in a user's environment. Those bundled copies have been removed, and the package now declares `googleapis-common-protos` and `protoc-gen-openapiv2` as runtime dependencies, so pip installs the upstream-maintained versions instead. `buf/validate` remains bundled for now, since Buf publishes the protovalidate runtime to PyPI but withholds the generated Python bindings. + ### Bugfixes - Add `py.typed` to the generated proto directory so type checkers pick up protobuf types correctly. @@ -89,6 +92,7 @@ Adds progress indicators for job polling and file downloads for better visibilit - [Refactor files using run_in_executor](https://github.com/sift-stack/sift/pull/518) - [Add Parquet as an export output format](https://github.com/sift-stack/sift/pull/510) - [Add py.typed file to proto dir](https://github.com/sift-stack/sift/pull/524) +- [Remove vendored google.api and protoc_gen_openapiv2 modules from buf](https://github.com/sift-stack/sift/pull/543) ## [v0.13.0] - March 24, 2026 ### What's New From 7240f6aee917150095dc438ddcd456d661abef8c Mon Sep 17 00:00:00 2001 From: Wei Qi Lu Date: Tue, 28 Apr 2026 11:36:59 -0700 Subject: [PATCH 11/12] update changelog to include buf changes --- python/CHANGELOG.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/python/CHANGELOG.md b/python/CHANGELOG.md index 534f5edbc..3be54601d 100644 --- a/python/CHANGELOG.md +++ b/python/CHANGELOG.md @@ -3,7 +3,7 @@ All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/). -## [v0.14.0] - April 27, 2026 +## [v0.14.0] - April 28, 2026 ### What's New @@ -77,8 +77,10 @@ Test result create and update events can now be optionally written to a `.jsonl` #### Progress Indicators Adds progress indicators for job polling and file downloads for better visibility during long-running operations. -#### Use Upstream Packages for `google.api` and `protoc_gen_openapiv2` -Previously the package shipped its own bundled copies of the generated Python bindings for `google.api` and `protoc_gen_openapiv2`, which could collide with the same modules installed elsewhere in a user's environment. Those bundled copies have been removed, and the package now declares `googleapis-common-protos` and `protoc-gen-openapiv2` as runtime dependencies, so pip installs the upstream-maintained versions instead. `buf/validate` remains bundled for now, since Buf publishes the protovalidate runtime to PyPI but withholds the generated Python bindings. +#### Stop Bundling `google.api`, `protoc_gen_openapiv2`, and `buf.validate` +Previously the package shipped its own bundled copies of the generated Python bindings for `google.api`, `protoc_gen_openapiv2`, and `buf.validate`. With this release: +- `google.api` and `protoc_gen_openapiv2` are now pulled in via the `googleapis-common-protos` and `protoc-gen-openapiv2` runtime dependencies, so pip installs the upstream-maintained versions. +- `buf.validate` has been removed entirely. The protovalidate field annotations were stripped from the affected `.proto` files, so the generated `_pb2.py` files no longer reference `buf.validate`. ### Bugfixes - Add `py.typed` to the generated proto directory so type checkers pick up protobuf types correctly. @@ -93,6 +95,7 @@ Previously the package shipped its own bundled copies of the generated Python bi - [Add Parquet as an export output format](https://github.com/sift-stack/sift/pull/510) - [Add py.typed file to proto dir](https://github.com/sift-stack/sift/pull/524) - [Remove vendored google.api and protoc_gen_openapiv2 modules from buf](https://github.com/sift-stack/sift/pull/543) +- [Remove protovalidate from client-side protos](https://github.com/sift-stack/sift/pull/XXX) ## [v0.13.0] - March 24, 2026 ### What's New From 81c889f168b241f2eb76166d2afdd5096a9f5480 Mon Sep 17 00:00:00 2001 From: Wei Qi Lu Date: Tue, 28 Apr 2026 11:38:55 -0700 Subject: [PATCH 12/12] add pull request number --- python/CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/CHANGELOG.md b/python/CHANGELOG.md index 3be54601d..3f8e38963 100644 --- a/python/CHANGELOG.md +++ b/python/CHANGELOG.md @@ -95,7 +95,7 @@ Previously the package shipped its own bundled copies of the generated Python bi - [Add Parquet as an export output format](https://github.com/sift-stack/sift/pull/510) - [Add py.typed file to proto dir](https://github.com/sift-stack/sift/pull/524) - [Remove vendored google.api and protoc_gen_openapiv2 modules from buf](https://github.com/sift-stack/sift/pull/543) -- [Remove protovalidate from client-side protos](https://github.com/sift-stack/sift/pull/XXX) +- [Remove protovalidate from client-side protos](https://github.com/sift-stack/sift/pull/545) ## [v0.13.0] - March 24, 2026 ### What's New