excludeFromIndexes: ['path[].to.some.array[]'] should exclude (the roots of) entity elements belonging to the array #100
Description
@google-cloud/datastoreversion: 1.4.0, or master@445304c7a9b97... (same same...)
Steps to reproduce
entity = {
key: key,
data: {
array: [
element1: 'string',
element2: { some: 'object' }
element3: 1337,
]
},
excludeFromIndexes: ['array[]']
};
datastore.save(entity);
- The property 'array' of saved entity above shows up as 'indexed' in Datastore Console. The reason seems to be as follows:
An array property apparently counts as excluded from indexes if and only if every element is excluded from indexes. (Other than entity and primitive properties, an array property can not be marked as excluded from indexes by itself, [ because semantically, it is supposed to just be some kind of "soft" container for one single "multi-valued" property, I guess ]. )
The excludeFromIndexes logic inentityToEntityProto(source filesrc/entity.js, latest update from PR #2773 (old repo) merged in Support excludeFromIndexes for primitives in arrays. #18) currently excludes only primitive elements of an array that is marked with"excludeFromIndexes: ['path.to.array[]']". Entity elements included in the array are simply skipped, and not excluded from indexing. Therefore, the array itself as a unit is not excluded from indexing, if it contains entity elements, despite marking it with"excludeFromIndexes: ['path.to.array[]']".
This seems illogical to me. It is perfectly possible to exclude entity elements inside array properties from indexing (which does not imply excluding all the nested properties of said entity elements from indexing, but only the root). It can be done with the low-level API for entity elements included in arrays. Those array properties, if all their individual elements are excluded, will then show up as 'not indexed' in the Datastore Console, as it should be, and will not waste index space.
I will submit a simple PR for this.
PS: Currently there are two different supported syntaxes to exclude entity properties from indexing.
a) "Array syntax" (apparently intended to be deprecated at some point):
key: key,
data: [{
name: 'rootProperty1',
value: 'some property at the root',
excludeFromIndexes: true/false
}, {
name: 'rootProperty2',
value: [ 'why', 'not', 'an', 'array'],
excludeFromIndexes: false
}, {
name: 'rootProperty3',
value: {
nestedProperty: 'nasty'
},
excludeFromIndexes: true
}
}]
};
This syntax only allows to specify "excludeFromIndexes" as true or false for each single property at the root. The current behaviour of this syntax for arrays is: If an array property is marked with excludedFromIndexes: true, then all its elements are marked as excluded from indexes, be they primitives or entities (the corresponding entity elements' sub-properties are not automatically excluded from indexes thereby). This is consistent with the behaviour of my pull-request for the other syntax.
I don't know if it is possible in some convoluted way with this syntax to mark sub- and sub-sub-properties and -arrays as excluded from indexes as well. I believe it is not. It seems this "old" syntax allows only for excluding first-level children/properties (including arrays) from indexing.
b) The "new" (and apparently favoured for the future) syntax referred to in this PR #101 here:
key: key,
data: {
just: { a: 'simple object', with: ['all', 'possible', 'kinds', 'of', 'nested', 'properties']}
}
excludedFromIndexes: ['paths[].to', 'every', 'single.property[].that', 'should.be[].excluded', 'from.indexing[]'];
};
This syntax would allow, on principle, the exclusion from indexes of properties at any level... were it just not for this bug: With the current buggy behaviour, it is not possible to exclude the roots of arrays containing entity elements from indexing. It is only possible to exclude primitive array elements from indexing, and to exclude all sub-properties of entity array elements from indexing, but not to exclude the entity array elements (and therefore the containing array property) themselves.