Support dataset-cover-image.png upload for datasets metadata --update

[jmasukawa]

Follow-up to #959

Requiring the dataset cover / thumbnail image to be in the same dir as dataset-metadata.json means that if you create a dataset with kaggle datasets create --dir-mode=(zip|tar), the metadata image will be included in your dataset files, which the author may not want. I had set it up that way because it adhered to Data Package spec, but it's kind of annoying.

This PR adds a special named file, dataset-cover-image.(png|jpg|jpeg|webp) in the same location as your dataset-metadata.json to provide it as the image upload, without including it in your dataset.

/some/path/dataset-metadata.json (not uploaded)
/some/path/my-files/* (uploaded)
/some/path/dataset-cover-image.png (not uploaded)

We still support relative path via "image" property to adhere to Data Package spec. This was the first form of the API for Adaption Labs (and we don't want to break them), but there's also an edge case scenario where you want to re-use an image in your dataset as the cover image.

Local testing:

• http://go/scrcast/NjAyNDkwOTM0NzA5NDUyOHxlNGU1NWE0ZS0xYw

Note: there are some other random changes that showed up from running ./docker-hatch run lint:fmt

http://b/500108129

[jmasukawa]

changes to this file are from ./docker-hatch run lint:fmt

[jmasukawa]

This is the main code change in this PR.

We check if the image file path is absolute instead of assuming it is always relative

[stevemessick]

Nice!

[goeffthomas]

Requiring the dataset cover / thumbnail image to be in the same dir as dataset-metadata.json means that if you create a dataset with kaggle datasets create --dir-mode=(zip|tar), the metadata image will be included in your dataset files, which the author may not want. I had set it up that way because it adhered to Data Package spec, but it's kind of annoying.

Interesting. Does that mean that the metadata JSON is also baked into the dataset in these cases?

[goeffthomas]

Requiring the dataset cover / thumbnail image to be in the same dir as dataset-metadata.json means that if you create a dataset with kaggle datasets create --dir-mode=(zip|tar), the metadata image will be included in your dataset files, which the author may not want. I had set it up that way because it adhered to Data Package spec, but it's kind of annoying.

Interesting. Does that mean that the metadata JSON is also baked into the dataset in these cases?

Actually, I found the answer. It looks like we exclude it from the upload:

kaggle-cli/src/kaggle/api/kaggle_api_extended.py

Line 4920 in f74b05b

self.DATASET_METADATA_FILE,

. So maybe that's even a different alternative. We could let users specify a dataset-cover-image.(png|jpg...) that's co-located with the metadata JSON, and then if present, we pull that to work our magic. WDYT? Then users don't even have to put it in their metadata JSON, right?

[rosbo]

FYI: When Steve and I discussed this FR before our OOO, we were leaning towards using a co-located file with a specific name for the header image as Goeff suggested above.

[jmasukawa]

So maybe that's even a different alternative. We could let users specify a dataset-cover-image.(png|jpg...) that's co-located with the metadata JSON, and then if present, we pull that to work our magic. WDYT? Then users don't even have to put it in their metadata JSON, right?

FYI: When Steve and I discussed this FR before our OOO, we were leaning towards using a co-located file with a specific name for the header image as Goeff suggested above.

Ah, that's a smart idea, i wish i thought about that before. 😬 🤦

Given we supported relative path already for Adaption Labs, I'll update the PR to use dataset-cover-image.png and ignore it from dataset creation as the preferred, but i'll still leave support for relative path (because we had shipped a version that supported it).

I'll update docs to document the dataset-cover-image.png approach

[jmasukawa]

So maybe that's even a different alternative. We could let users specify a dataset-cover-image.(png|jpg...) that's co-located with the metadata JSON, and then if present, we pull that to work our magic. WDYT? Then users don't even have to put it in their metadata JSON, right?

FYI: When Steve and I discussed this FR before our OOO, we were leaning towards using a co-located file with a specific name for the header image as Goeff suggested above.

Ah, that's a smart idea, i wish i thought about that before. 😬 🤦

Given we supported relative path already for Adaption Labs, I'll update the PR to use dataset-cover-image.png and ignore it from dataset creation as the preferred, but i'll still leave support for relative path (because we had shipped a version that supported it).

I'll update docs to document the dataset-cover-image.png approach

@rosbo @stevemessick I updated to use this approach, PTAL. Thanks again for the suggestion.