Skip to content

Add data downloading functionality and gdown dependency for test setup#57

Merged
romeokienzler merged 19 commits into
mainfrom
improve_tests
May 20, 2026
Merged

Add data downloading functionality and gdown dependency for test setup#57
romeokienzler merged 19 commits into
mainfrom
improve_tests

Conversation

@romeokienzler
Copy link
Copy Markdown
Collaborator

This pull request updates the integration test workflow to use pre-generated data downloaded from Google Drive instead of generating data locally. It introduces the gdown library as a new dependency and modifies the test logic to automate the download and extraction of the dataset if not already present.

Dependency and test workflow updates:

  • Added gdown as a dependency in pyproject.toml to enable downloading files from Google Drive.
  • Updated integrationtests/test_base_set.py to import gdown and zipfile for handling dataset downloads and extraction.

Test data handling improvements:

  • Modified the test_train function in integrationtests/test_base_set.py to:
    • Download a pre-generated dataset from Google Drive using gdown if the data_out directory is missing or empty.
    • Extract the downloaded zip file and clean up after extraction.
    • Skip the previous in-place dataset generation steps (now commented out).

@romeokienzler romeokienzler marked this pull request as draft May 5, 2026 09:38
@romeokienzler romeokienzler changed the title Add data downloading functionality and gdown dependency for test setup [WIP] Add data downloading functionality and gdown dependency for test setup May 5, 2026
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
…d related code

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
…low scenarios

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
…ation

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
…tion

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
…nd use numpy for calculations

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
…an and other metrics

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
…n and updating related tests

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
…on instructions

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
@romeokienzler romeokienzler self-assigned this May 15, 2026
@romeokienzler romeokienzler marked this pull request as ready for review May 15, 2026 08:52
@romeokienzler romeokienzler changed the title [WIP] Add data downloading functionality and gdown dependency for test setup Add data downloading functionality and gdown dependency for test setup May 15, 2026
@romeokienzler romeokienzler mentioned this pull request May 15, 2026
Comment thread integrationtests/conftest.py Outdated
Comment thread integrationtests/test_base_set.py Outdated
Comment thread integrationtests/test_base_set.py Outdated
Comment thread integrationtests/test_base_set.py Outdated
…d related functions

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
…pdate description for accuracy

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
…n value

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Copy link
Copy Markdown
Collaborator

@albanpuech albanpuech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much @romeokienzler !!!!!!!

Copy link
Copy Markdown
Collaborator

@albanpuech albanpuech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVED!
could you just add in the README some instructions for running the integration test?

Thank you!

@romeokienzler romeokienzler merged commit 5e6d837 into main May 20, 2026
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants