Skip to content

Storage: download_to_file() decompresses gzip'd object even when passed "Accept-Encoding: gzip" #6035

@houglum

Description

@houglum
  1. Specify the API at the beginning of the title (for example, "BigQuery: ...")
    General, Core, and Other are also allowed as types
    • Storage
  2. OS type and version
    • Only tested on Linux (Debian), but I assume this happens regardless of OS
  3. Python version and virtual environment information python --version
    • Python 2.7.13
  4. google-cloud-python version pip show google-cloud, pip show google-<service> or pip freeze
    • pip freeze | grep storage gives 1.12.0
  5. Stacktrace if available
    • N/A
  6. Steps to reproduce
  7. Code example

Once the issue mentioned above is resolved and a new version of google-resumable-media-python is released, this library should:

  • Require a version of it >= the new version
  • Figure out what to do about the download_to_file() and download_to_filename() methods in blob.py [1] [2]. They currently look like they're supposed to download gzipped objects (those with Content-Encoding set to "gzip") in their compressed state, given that they add the "Accept-Encoding: gzip" header to the request. However, due to the issue mentioned above, they do not; they actually result in the decompressed bytes being written to the desired stream/file. Once this gets fixed and we pull in a new version with that fix, we'll probably want to allow users to specify whether or not the downloaded gzip'd file should be decompressed (this is the behavior today, so it should probably be the default), or if the downloaded file should contain the compressed bytes.

[1]https://github.com/GoogleCloudPlatform/google-cloud-python/blob/e4a24d30283ce42ae58051dfd7f6267f2939d115/storage/google/cloud/storage/blob.py#L542
[2] https://github.com/GoogleCloudPlatform/google-cloud-python/blob/e4a24d30283ce42ae58051dfd7f6267f2939d115/storage/google/cloud/storage/blob.py#L489

Metadata

Metadata

Assignees

Labels

api: storageIssues related to the Cloud Storage API.type: questionRequest for information or clarification. Not an issue.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions