Skip to content

Point .pq files to datasets bucket#1203

Merged
joaquinvanschoren merged 1 commit intodevelopfrom
feature/new_pq_buckets
Jan 4, 2024
Merged

Point .pq files to datasets bucket#1203
joaquinvanschoren merged 1 commit intodevelopfrom
feature/new_pq_buckets

Conversation

@josvandervelde
Copy link
Member

@josvandervelde josvandervelde commented Jan 4, 2024

Goal

See Roadmap (high level) for the goal & reasoning.

In this PR

The .pq url is changed from
https://openml1.win.tue.nl/dataset45714/dataset_45714.pq to
https://openml1.win.tue.nl/datasets/0004/45714/dataset_45714.pq

How to test

I haven't tested this thoroughly. I just ran

$MINIO_URL = 'http://openml1.win.tue.nl/';
$data_id = 45714;
$bracket = sprintf('%04d', floor($data_id / 10000));
$padded_id = sprintf('%04d', $data_id);
$url = $MINIO_URL . 'datasets/' . $bracket . '/' . $padded_id . '/dataset_' . $data_id . '.pq';
echo($url);

In a php-sandbox and checked the resulting url.

@josvandervelde
Copy link
Member Author

Hi @joaquinvanschoren could you review this PR, and if it's OK, could you deploy it / tell me how to deploy it?

@joaquinvanschoren joaquinvanschoren merged commit 03ca939 into develop Jan 4, 2024
@joaquinvanschoren
Copy link
Contributor

Done, also deployed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants