Storage: 'test_bpo_set_unset_preserves_acls' no longer sees expected 'BadRequest'.

[tseaver]

From this Kokoro job:

____________ TestIAMConfiguration.test_bpo_set_unset_preserves_acls ____________
self = <tests.system.TestIAMConfiguration testMethod=test_bpo_set_unset_preserves_acls>
    def test_bpo_set_unset_preserves_acls(self):
        new_bucket_name = "bpo-acls" + unique_resource_id("-")
        self.assertRaises(
            exceptions.NotFound, Config.CLIENT.get_bucket, new_bucket_name
        )
        bucket = retry_429(Config.CLIENT.create_bucket)(new_bucket_name)
        self.case_buckets_to_delete.append(new_bucket_name)
        blob_name = "my-blob.txt"
        blob = bucket.blob(blob_name)
        payload = b"DEADBEEF"
        blob.upload_from_string(payload)
        # Preserve ACLs before setting BPO
        bucket_acl_before = list(bucket.acl)
        blob_acl_before = list(bucket.acl)
        # Set BPO
        bucket.iam_configuration.bucket_policy_only_enabled = True
        bucket.patch()
        # While BPO is set, cannot get / set ACLs
        with self.assertRaises(exceptions.BadRequest):
>           bucket.acl.reload()
E           AssertionError: BadRequest not raised
tests/system.py:1709: AssertionError

[tseaver]

I'm not sure this is actually "flaky": the back-end appears to no longer return a BadRequest for bucketAccessControls.list when the bucket-policy-only flag is set.

@frankyn Can you confirm that the back-end behavior is changed?

[tseaver]

Very strange: the call to Bucket.acl.reload does raise the expected BadRequest when running under Python 2.7, but does not raise as expected under Python 3.6.

[tseaver]

Running the system test with Python 3.6, here is what I see from a breakpoint inside BucketACL.reload:

(Pdb) l
432  	        if self.user_project is not None:
433  	            query_params["userProject"] = self.user_project
434  	
435  	        self.entities.clear()
436  	
437  ->	        found = client._connection.api_request(
438  	            method="GET", path=path, query_params=query_params
439  	        )
440  	        self.loaded = True
441  	        for entry in found.get("items", ()):
442  	            self.add_entity(self.entity_from_dict(entry))
(Pdb) pp path
'/b/bpo-acls-1562617375855/acl'
(Pdb) pp query_params
{}
(Pdb) n
> /home/tseaver/projects/agendaless/Google/src/gcp/storage/google/cloud/storage/acl.py(438)reload()
-> method="GET", path=path, query_params=query_params
(Pdb) 
> /home/tseaver/projects/agendaless/Google/src/gcp/storage/google/cloud/storage/acl.py(440)reload()
-> self.loaded = True
(Pdb) pp found
{'items': [{'bucket': 'bpo-acls-1562617375855',
            'entity': 'project-owners-455332057385',
            'etag': 'CAI=',
            'id': 'bpo-acls-1562617375855/project-owners-455332057385',
            'kind': 'storage#bucketAccessControl',
            'projectTeam': {'projectNumber': '455332057385', 'team': 'owners'},
            'role': 'OWNER',
            'selfLink': 'https://www.googleapis.com/storage/v1/b/bpo-acls-1562617375855/acl/project-owners-455332057385'},
           {'bucket': 'bpo-acls-1562617375855',
            'entity': 'project-editors-455332057385',
            'etag': 'CAI=',
            'id': 'bpo-acls-1562617375855/project-editors-455332057385',
            'kind': 'storage#bucketAccessControl',
            'projectTeam': {'projectNumber': '455332057385', 'team': 'editors'},
            'role': 'OWNER',
            'selfLink': 'https://www.googleapis.com/storage/v1/b/bpo-acls-1562617375855/acl/project-editors-455332057385'},
           {'bucket': 'bpo-acls-1562617375855',
            'entity': 'project-viewers-455332057385',
            'etag': 'CAI=',
            'id': 'bpo-acls-1562617375855/project-viewers-455332057385',
            'kind': 'storage#bucketAccessControl',
            'projectTeam': {'projectNumber': '455332057385', 'team': 'viewers'},
            'role': 'READER',
            'selfLink': 'https://www.googleapis.com/storage/v1/b/bpo-acls-1562617375855/acl/project-viewers-455332057385'}],
 'kind': 'storage#bucketAccessControls'}
(Pdb) client._connection.api_request(method="GET", path=self.bucket.path)
{'kind': 'storage#bucket', 'id': 'bpo-acls-1562617375855', 'selfLink': 'https://www.googleapis.com/storage/v1/b/bpo-acls-1562617375855', 'projectNumber': '455332057385', 'name': 'bpo-acls-1562617375855', 'timeCreated': '2019-07-08T20:22:56.637Z', 'updated': '2019-07-08T20:22:58.175Z', 'metageneration': '2', 'iamConfiguration': {'bucketPolicyOnly': {'enabled': False}, 'uniformBucketLevelAccess': {'enabled': False}}, 'location': 'US', 'locationType': 'multi-region', 'storageClass': 'STANDARD', 'etag': 'CAI='}

Note that the resource's iamConfiguration contains 'bucketPolicyOnly': {'enabled': False} (?!)

[tseaver]

Under Python 2.7:

(Pdb) l
430  	        query_params = {}
431  	
432  	        if self.user_project is not None:
433  	            query_params["userProject"] = self.user_project
434  	
435  ->	        self.entities.clear()
436  	
437  	        found = client._connection.api_request(
438  	            method="GET", path=path, query_params=query_params
439  	        )
440  	        self.loaded = True
(Pdb) n
> /home/tseaver/projects/agendaless/Google/src/gcp/storage/google/cloud/storage/acl.py(437)reload()
-> found = client._connection.api_request(
(Pdb) pp path
'/b/bpo-acls-1562617750458/acl'
(Pdb) pp query_params
{}
(Pdb) n
> /home/tseaver/projects/agendaless/Google/src/gcp/storage/google/cloud/storage/acl.py(438)reload()
-> method="GET", path=path, query_params=query_params
(Pdb) 
BadRequest: BadReque...-only.',)
> /home/tseaver/projects/agendaless/Google/src/gcp/storage/google/cloud/storage/acl.py(438)reload()
-> method="GET", path=path, query_params=query_params
(Pdb) client._connection.api_request(method="GET", path=self.bucket.path)
{u'kind': u'storage#bucket', u'name': u'bpo-acls-1562617750458', u'timeCreated': u'2019-07-08T20:29:11.039Z', u'locationType': u'multi-region', u'updated': u'2019-07-08T20:29:12.575Z', u'projectNumber': u'455332057385', u'metageneration': u'2', u'location': u'US', u'iamConfiguration': {u'uniformBucketLevelAccess': {u'enabled': True, u'lockedTime': u'2019-10-06T20:29:12.539Z'}, u'bucketPolicyOnly': {u'enabled': True, u'lockedTime': u'2019-10-06T20:29:12.539Z'}}, u'etag': u'CAI=', u'id': u'bpo-acls-1562617750458', u'selfLink': u'https://www.googleapis.com/storage/v1/b/bpo-acls-1562617750458', u'storageClass': u'STANDARD'}

Here, the resource's iamConfiguration contains 'bucketPolicyOnly': {'enabled': True}, as expected.