Build Problems with Storage ACLs

[stephenplusplus]

I saw this locally yesterday, but the builds worked in Travis. Today, Travis is reporting the same errors:

https://travis-ci.org/GoogleCloudPlatform/gcloud-node/#L2635

1) storage acls buckets should add entity to default access controls:
     Error: Not Found
      at Object.handleResp (lib/common/util.js:9:4858)
      at Request._callback (lib/common/util.js:9:20254)
      at _stream_readable.js:908:16
  2) storage acls buckets should grant an account access:
     Error: Not Found
      at Object.handleResp (lib/common/util.js:9:4858)
      at Request._callback (lib/common/util.js:9:20254)
      at _stream_readable.js:908:16
  3) storage acls buckets should update an account:
     Error: Not Found
      at Object.handleResp (lib/common/util.js:9:4858)
      at Request._callback (lib/common/util.js:9:20254)
      at _stream_readable.js:908:16
  4) storage acls files should grant an account access:
     Error: Not Found
      at Object.handleResp (lib/common/util.js:9:4858)
      at Request._callback (lib/common/util.js:9:20254)
      at _stream_readable.js:908:16
  5) storage acls files should update an account:
     Error: Not Found
      at Object.handleResp (lib/common/util.js:9:4858)
      at Request._callback (lib/common/util.js:9:20254)
      at _stream_readable.js:908:16

[stephenplusplus]

Seems to be a problem upstream with defaultObjectAccessControls.delete. Using https://cloud.google.com/storage/docs/json_api/v1/defaultObjectAccessControls/insert to add a user works, then it cannot be deleted with https://cloud.google.com/storage/docs/json_api/v1/defaultObjectAccessControls/delete.

@jgeewax have you already heard of this issue?

[jgeewax]

I haven't -- do we need to escalate? If so can you give the minimum explanation of what exactly is failing? If something can't be deleted I'd expect a 403.. not a 404, right?

[stephenplusplus]

If something can't be deleted I'd expect a 403.. not a 404, right?

Yeah, it's weird, since doing a GET returns the object and a 200.

It probably couldn't hurt to have someone look at it.

To reproduce:

1. Create a new bucket in the dev console (ex. name: "unique-bucket-name-stephen") https://console.developers.google.com/project/nth-circlet-705/storage/browser
2. Use the API explorer to insert a default object acl https://cloud.google.com/storage/docs/json_api/v1/defaultObjectAccessControls/insert
   • bucket: bucket name created in step 1
   • fields: Leave blank
   • request body:
     • entity: user-[emailaddress]
     • role: READER
3. Confirm 200 response (expected)
4. Use the API explorer to delete that default object acl https://cloud.google.com/storage/docs/json_api/v1/defaultObjectAccessControls/delete
   • bucket: bucket name created in step 1
   • entity: entity used in step 2
5. Confirm 404 response (unexpected)

This was previously working, but we started getting the 404 yesterday.

[jgeewax]

/cc @Capstan

[stephenplusplus]

I found something interesting. After I insert "user-[emailaddress]", then list the defaultObjectAcls, it comes back like this:

{
  "kind": "storage#objectAccessControl",
  "entity": "user-00b4903a970617830a8fbdf72fc0df7976b1dd9624224cf3c6053d64383e6851",
  "role": "READER",
  "entityId": "00b4903a970617830a8fbdf72fc0df7976b1dd9624224cf3c6053d64383e6851",
  "etag": "CAY="
}

I can get the defaultObjectAcl by user-[emailaddress], but I can only delete it with its encoded name. Should we be getting back the encoded version at all?

[jgeewax]

/cc @hurstdog who can probably comment on what the intended behavior is and why.

[callmehiphop]

related to googleapis/google-cloud-go#147

[stephenplusplus]

@jgeewax Since this breaks our tests, we're blocked on getting a new release out as well as our various doc fixes for the last 2 weeks. Is this being looked at?

[jgeewax]

Sounds like GCS is going to send back some magical hash...

I'm asking @hurstdog and @Capstan about how we can now tell GCS to delete the ACL for user-<email> given that they're storing it as user-<magical hash>

[stephenplusplus]

Great, thanks!

[stephenplusplus]

My current plan is to give it until Wednesday, then skip these tests from the build so we can get a release. Let me know if that doesn't work.

The releases we have out now (< 0.16.0) will fail under the same test, since this was a change in the upstream API, not ours. If we wait much longer before a release, our bug reports (should we get any) will increase in difficulty to track down due to the scope of things we've changed.

[royalpeasantry]

I tracked down the issue, and it should be fixed with our release next week.

[stephenplusplus]

Awesome, thanks @royalpeasantry!