State of "excludeFromIndexes" (datastore)

[sebelga]

Hello,

I have seen that since version 1.1.0 of the datastore api there is a new syntax for the "excludeFromIndexes" allowing embedded entities to have their properties excluded from indexes.

I am wondering if this new syntax is currently supported as it is not yet in the "master" documentation (it only appears under the "1.1.0" version).

I am sending the following payload to save an entity but all of the properties are marked as "indexed" in the google cloud console. Thus the "excludeFromIndexes" is not working as expected.

{
    "key": {
        "namespace": "my.namespace",
        "kind": "MyEntity",
        "path": [
            "MyEntity",
            null
        ]
    },
    "data": [
        {
            "name": "systems",
            "value": {
                "a": "123c",
                "b": "456c"
            }
        },
        {
            "name": "phoneNumber",
            "value": "0123456789"
        },
        {
            "name": "createdAt",
            "value": "2017-12-12T06:16:35.145Z"
        }
    ],
    "excludeFromIndexes": [
        "phoneNumber",
        "systems",
        "createdAt",
        "systems.a",
        "systems.b"
    ]
}

Thank for any help on this!

[stephenplusplus]

Hey @sebelga! I don't believe excludeFromIndexes is compatible with the array-notation format of data.

We originally only supported the old excludeFromIndexes markup if the user provided an array. That was so we could separate the excludeFromIndexes property from the properties within a user's entity. This style was a problem when nested properties needed to declare excludeFromIndexes. To handle that situation, we created the more robust, top-level excludeFromIndexes array, like you've defined. To opt-in, you just need to convert the data array to an object:

{
    "data": {
        "systems": {
            "a": "123c",
            "b": "456c"
        },
        "phoneNumber": "0123456789"
        "createdAt": "2017-12-12T06:16:35.145Z"
    }
}

Let me know if that doesn't fix the issue, or if that style for data creates any problems.

[sebelga]

Hi @stephenplusplus ! Ok I will give it a try and come back with my results.

I've found inconsistencies (see my comment here #2510 (comment)) and haven't been able to find the proper way to both declare "excludeFromIndexes" for a simple properties, for Arrays (exclude all their properties) and for a properties on an embedded entity.

I will give a try with what you comment and let you know. 👍

[beaulac]

If #2773 is merged, it would allow this syntax to also support (non-nested) arrays.

[stephenplusplus]

This issue was moved to googleapis/nodejs-datastore#14.