Skip to content

scheme is hard-coded as http when converting response in azure_blobs #1703

@KatiRG

Description

@KatiRG

Summary

Functions _response_to_container() and _response_to_object() in class AzureBlobsStorageDriver of azure_blobs hard-code the URL scheme to http in their extra objects, i.e.:

extra = { "url": "http://%s%s" % (response.connection.host, response.connection.action),

This will cause a request to download from Azure blob storage to fail if the storage URL is secure (i.e. uses https).

Detailed Information

Other functions in this class do check secure to dynamically assign the scheme, e.g. in get_object_cdn_url():

scheme="https" if self.secure else "http",

We can fix the bug in the response functions in a similar way by dynamically assigning the scheme in the extra objects:

scheme='https' if self.secure else 'http',
        extra = {
            'url': '%s://%s%s' % (scheme, response.connection.host,
                                    response.connection.action),

Environment

Libcloud version: apache-libcloud==2.8.2
Python version: Python 3.8.10
Operating System: Ubuntu 20.04 LTS

Steps which are needed to reproduce problem:
I am using a secure blob storage account (i.e. its URL starts with https) for a CKAN project, version 2.9.

I am using AzureBlobsStorageDriver with the CKAN extension ckanext-cloudstorage, an extension that requires libcloud, and the extension ckanext-xloader (v 0.9.0), which accesses the blob storage account. Because of the bug reported here, Xloader is being passed an incorrect URL (starting with http instead of https), and therefore cannot download any data from the storage account:

ckanext.xloader.job_exceptions.HTTPError: b'Xloader received a bad HTTP response when trying to download the data file 
Client Error: The account being accessed does not support http.

But with the fix to dynamically assign the scheme based on secure, the correct URL is passed and the data can be downloaded.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions