V3 Organization with nullified updatedAt

[eaglerainbow]

This is in reference to promregator/promregator#36, where the following situation has happened:

• CF platform running on API version 2.94.0
• Client application using cf-java-client 3.3.0.RELEASE -- source code as of state tested can be reviewed at https://github.com/promregator/promregator/tree/31b001aecf1e9223a2fc0794bd2f725ca87a8e00
• The interesting code part is at https://github.com/promregator/promregator/blob/31b001aecf1e9223a2fc0794bd2f725ca87a8e00/src/main/java/org/cloudfoundry/promregator/cfaccessor/CFAccessorImpl.java#L34
• Client application fires an organizationsV3().list() request, selecting a single known org there. Aim is to get the org's ID.
• The Org, however, was created somewhere at 2016-07-26T17:01:24Z. In the response, the updatedAt is apparently nullified.
• Error message (symptom), which is raised, reads:

Caused by: java.lang.IllegalStateException: Cannot build OrganizationResource, some of required attributes are not set [updatedAt]
	at org.cloudfoundry.client.v3.organizations.OrganizationResource$Builder.build(OrganizationResource.java:395) ~[cloudfoundry-client-3.3.0.RELEASE.jar!/:3.3.0.RELEASE]
	at org.cloudfoundry.client.v3.organizations.OrganizationResource.fromJson(OrganizationResource.java:200) ~[cloudfoundry-client-3.3.0.RELEASE.jar!/:3.3.0.RELEASE]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_162]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_162]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_162]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_162]
	at com.fasterxml.jackson.databind.introspect.AnnotatedMethod.call1(AnnotatedMethod.java:114) ~[jackson-databind-2.9.4.jar!/:2.9.4]
	at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator._createUsingDelegate(StdValueInstantiator.java:525) ~[jackson-databind-2.9.4.jar!/:2.9.4]
	... 70 common frames omitted

2018-03-29 22:02:08.018 ERROR 10 --- [nio-8080-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.cloudfoundry.reactor.util.JsonParsingException: Cannot construct instance of `org.cloudfoundry.client.v3.organizations.OrganizationResource`, problem: Cannot build OrganizationResource, some of required attributes are not set [updatedAt]
 at [Source: (byte[])"{
  "pagination": {
    "total_results": 1,
    "total_pages": 1,
    "first": {
      "href": "https://my.cf.api.com/v3/organizations?names=My-CF-Org&page=1&per_page=50"
    },
    "last": {
      "href": "https://my.cf.api.com/v3/organizations?names=My-CF-Org&page=1&per_page=50"
    },
    "next": null,
    "previous": null
  },
  "resources": [
    {
      "guid": "ac4d8f82-c606-44cf-bda5-6fa36e01bf20",
      "created_at": "2016-07-26T17:0"[truncated 248 bytes]; line: 25, column: 5] (through reference chain: org.cloudfoundry.client.v3.organizations.ListOrganizationsResponse$Json["resources"]->java.util.ArrayList[0])] with root cause 

• When requesting the same org via the organizationsV2 API, the nullified updatedAt attribute becomes visible (output of cf -v orgs):

{
  "next_url": null,
  "prev_url": null,
  "resources": [
    {
      "entity": {
        "app_events_url": "/v2/organizations/my-org-guid/app_events",
        "auditors_url": "/v2/organizations/my-org-guid/auditors",
        "billing_enabled": false,
        "billing_managers_url": "/v2/organizations/my-org-guid/billing_managers",
        "default_isolation_segment_guid": null,
        "domains_url": "/v2/organizations/my-org-guid/domains",
        "managers_url": "/v2/organizations/my-org-guid/managers",
        "name": "My-Org",
        "private_domains_url": "/v2/organizations/my-org-guid/private_domains",
        "quota_definition_guid": "my-quota-guid",
        "quota_definition_url": "/v2/quota_definitions/my-quota-guid",
        "space_quota_definitions_url": "/v2/organizations/my-org-guid/space_quota_definitions",
        "spaces_url": "/v2/organizations/my-org-guid/spaces",
        "status": "active",
        "users_url": "/v2/organizations/my-org-guid/users"
      },
      "metadata": {
        "created_at": "2016-07-26T17:01:24Z",
        "guid": "my-org-guid",
        "updated_at": null,
        "url": "/v2/organizations/my-org-guid"
      }
    }
  ],
  "total_pages": 1,
  "total_results": 1
} 

Further details can also be found at the linked issue above.

My expectation would have been that the V3 organization API is capable of handling this case. If the updatedAt field is really considered mandatory, then I would have expected that it would be defaulted with the value of createdAt, which apparently is available.

Case is not urgent, but something that might be worthwhile to look at - and perhaps covered in a beasty regression test...

Thanks for checking!

[twoseat]

Thanks for the report! Unfortunately the documentation we rely on doesn't indicate optionality for that field, and as it's shared among all resources I want to be careful about changing it (especially as we've never seen an example that had that field nulled before!) The good news is that once I confirm that it can be null the fix is trivial, so assuming a change is needed it will be in the next release.

[eaglerainbow]

That's totally fine for me.
It's also okay for me, if we find out that the issue is not with the client, but the protocol indicates that the server must fill the field (somehow).
The intention of this bug report simply is that you get aware of that "there is something out there" and are not taken by surprise (which I was 😁) ...

[twoseat]

On reflection we'll almost certainly make it optional - whether it 'should' be nullable or not, you're running a production version of the CF API where it is, so we'll need to allow for that ☹️

[eaglerainbow]

Well, I neither knew the link to the spec (where, BTW, it is also not said explicitly that the field is non-null as far as I can read it), nor do I have contact to the right folks on the server side (I simply don't know where to create a ticket for that). It also sounds reasonable in this case to me saying that the productive platform has a bug violating the protocol. Then it wouldn't be the task of the client to adjust, but the server needs to be fixed. Given the fact that this is "only the V3 interface" and not the much-more-often-used V2, I would allow to consider this case as teething troubles...

[eaglerainbow]

Perhaps three more observations which I just made:

• The same org (with the nullified updated_at) loads just fine with the organizationsV2() call using cf-java-client.
• Up to version 246, the documentation of the V2 API still lists an example of a nullified updated_at field in the metadata object. In younger versions, a valid timestamp is provided - using the same value as created_at.
• The nullified updated_at is there up to the oldest API version available on the server.

Sounds like that this is historical heritage that in earlier versions, it was possible that updated_at was not filled, but newer versions automatically default it with the created_at to prevent that it's null. I guess, when defining the V3 spec, simply nobody considered this case to happen anymore - and you wouldn't also notice this case in an integration test at all, unless you have an "ancient org" around (like in my case 🙄 ).

Just food for the brain...

[eaglerainbow]

Thanks for the fix!