New/to cdf by stephTchembeu · Pull Request #513 · PySport/kloppy

stephTchembeu · 2025-10-28T09:40:47Z

No description provided.

UnravelSports · 2025-10-28T10:21:28Z

+                dataset.metadata.date + frame.timestamp
+            )
+            # Period
+            frame_data["period"] = periods.get(frame.period.id, "unknownn")


Change "unknownn" to "MISSING_MANDATORY_PERIOD_ID"

UnravelSports · 2025-10-28T10:23:16Z

+            Orientation,
+            BallState,
+        )
+


We should add a check to warn the user if they are progressing with only_alive=True in their kloppy dataset, because CDF expects all frames.

if all([True for x in tracking_dataset if x.ball_state == BallState.ALIVE]): warnings.warn( "All frames in 'tracking_dataset' are 'ALIVE', the Commond Data Format expects 'DEAD' frames as well. Set `only_alive=False` in your kloppy `.load_tracking()` call to include 'DEAD' frames.", UserWarning, )

UnravelSports · 2025-10-28T10:24:15Z

+
+            # Teams and players
+            home_players = []
+            for player, coordinates in frame.players_coordinates.items():


This duplicated code (for home and away players) to loop over players needs to be a function.

UnravelSports · 2025-10-28T10:24:50Z

+                            }
+                        )
+                    except KeyError:
+                        continue


What KeyError are we handling here? Should be handled without exception

UnravelSports · 2025-10-28T10:26:05Z

+
+            # teams within the tracking data.
+
+            home_players_id = []


What is this loop for?

UnravelSports · 2025-10-28T10:36:46Z

+                    "formation": (
+                        home_team.formations.at_start()
+                        if home_team.formations.items
+                        else self.get_starting_formation(


This is a small mistake in the common-data-format-validator, only "id" and "players" are mandatory here. So, we can remove "jersey_color" and "formation" as keys on the frame by frame level. If "home_team.name" is not None, we can add it too, but it's also not required.

UnravelSports · 2025-10-28T10:36:55Z

+                        away_team.formations.at_start()
+                        if away_team.formations.items
+                        else self.get_starting_formation(
+                            [


UnravelSports · 2025-10-28T10:37:37Z

+            starters_ids.append(player.player_id)
+
+        for player in home_team.players:
+            try:


Why are we catching KeyErrors here?

UnravelSports · 2025-10-28T10:38:03Z

+
+        meta_away_players = []
+        for player in away_team.players:
+            try:


same, again, these two operations for home and away are the same, we can make a function out of this

UnravelSports · 2025-10-28T10:39:58Z

+            # Add to tracking list
+            outputs.tracking_data.append(frame_file)
+
+        ###################### build now the metadata.


Create two separate functions in the serializer.py, one for generating meta data and one for tracking data to clean it up.

UnravelSports · 2025-10-28T12:36:02Z

@koenvo how do we want to make this functionality ultimately available to the end user? It should be captured in some type of common_data_format.store(dataset, file_path) type operation, right?

koenvo · 2025-10-28T13:13:21Z

@koenvo how do we want to make this functionality ultimately available to the end user? It should be captured in some type of common_data_format.store(dataset, file_path) type operation, right?

Yes, or we can start with a helper like:

dataset.to_cdf(file_path)

...


def to_cdf(self, file_path):
    # load and save?
    from common_data_format import save
    save(self, file_path)

UnravelSports · 2025-10-28T15:01:08Z

@koenvo how do we want to make this functionality ultimately available to the end user? It should be captured in some type of common_data_format.store(dataset, file_path) type operation, right?

Yes, or we can start with a helper like:
dataset.to_cdf(file_path)

...


def to_cdf(self, file_path):
    # load and save?
    from common_data_format import save
    save(self, file_path)

Oh, yeah I thought you were against doing the to_cdf option, I definitely prefer this as well.

koenvo · 2025-10-28T15:21:37Z

@koenvo how do we want to make this functionality ultimately available to the end user? It should be captured in some type of common_data_format.store(dataset, file_path) type operation, right?

Yes, or we can start with a helper like:
dataset.to_cdf(file_path)

...


def to_cdf(self, file_path):
    # load and save?
    from common_data_format import save
    save(self, file_path)
Oh, yeah I thought you were against doing the to_cdf option, I definitely prefer this as well.

I believe the initial idea was to put the actual implementation into the to_cdf method, and that isn't the best idea.

This adds comprehensive write support to the open_as_file() function with efficient memory management and streaming capabilities. Key features: - BufferedStream: SpooledTemporaryFile wrapper with chunked I/O (5MB memory threshold) - Write modes: 'wb' (write), 'ab' (append) - binary only - Adapter pattern: write_from_stream() method (opt-in for adapters) - Compression support: .gz, .bz2, .xz files handled automatically - Local files and S3 URIs supported via FSSpecAdapter - Protocols for type safety: SupportsRead, SupportsWrite Implementation details: - read_from()/write_to() methods use shutil.copyfileobj for chunked copying - Context manager pattern buffers writes and flushes on exit - No breaking changes to existing read functionality

Cleaned up CDF Serializer

stephTchembeu added 2 commits October 28, 2025 09:42

squash feta/to_cdf

e252c91

squash

1e2d6e6

UnravelSports reviewed Oct 28, 2025

View reviewed changes

UnravelSports [JB] and others added 4 commits October 30, 2025 14:44

cleaned up CDF Serializer

8eaafbb

WIP: add write support

fbf20c6

Follow output pattern for SportsCode

3e1073a

UnravelSports added this to the 3.19.0 milestone Nov 18, 2025

UnravelSports [JB] and others added 15 commits December 11, 2025 10:36

merge pr-515

e249156

working writer

f419105

cdf improve write

7993ec8

io

9e00f8e

improved (now complete) skillcorner position mapping

6b904b2

remove setup.py

87883a2

add import error

c731a76

improved tests, 2 providers

3100eb5

skillcorner additional test files

b72f03e

improved meta data

c4055e7

remove error, add warning

5ffa0d0

fix test

766e2f0

Merge pull request #5 from UnravelSports/pr-513

769ab73

Cleaned up CDF Serializer

Merge branch 'master' into new/to_cdf

9ba3f97

failing tests

75da5d6

UnravelSports mentioned this pull request Dec 17, 2025

[CDF] to_cdf #519

Open

probberechts closed this Jan 8, 2026

Conversation

stephTchembeu commented Oct 28, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

UnravelSports commented Oct 28, 2025

Uh oh!

koenvo commented Oct 28, 2025

Uh oh!

UnravelSports commented Oct 28, 2025

Uh oh!

koenvo commented Oct 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants