Cosmos: add support for pagination

[AndriySvyryd]

See https://docs.microsoft.com/en-us/dotnet/api/microsoft.azure.cosmos.queryrequestoptions.maxitemcount?view=azure-dotnet

[AndriySvyryd]

Also consider exposing the continuation token

The RU charge of a query with OFFSET LIMIT will increase as the number of terms being offset increases. For queries that have multiple pages of results, we typically recommend using continuation tokens. Continuation tokens are a "bookmark" for the place where the query can later resume. If you use OFFSET LIMIT, there is no "bookmark". If you wanted to return the query's next page, you would have to start from the beginning.

[ajcvickers]

/cc @roji

[roji]

It seems like we should have both a per-query setting and a global context option default (like non-tracked).

Some theoretical thoughts:

• This is related to keyset pagination in the sense that this is also about avoiding OFFSET pagination.
• The rough relational feature comparable to this is cursors, i.e. execute a query via a cursor (DECLARE CURSOR FOR ...) and then fetching the next page each time (FETCH NEXT ...). In both cases we're executing a roundtrip every time to get the next page. It's interesting that this logic is the default in Cosmos but not in relational.
• We could provide a relational feature where the query is paginated: the application would continue interacting with a simple QueryingEnumerable, and EF would send FETCH NEXT as needed under the hood.
• However, I'm not sure how that's better than the application implementing explicit keyset navigation itself, i.e. using OrderBy and Where to precisely define the page they want to fetch.
  • Explicit keyset pagination has the advantage of being able to move backwards (whereas a LINQ Enumerable can only move forward).
  • Cursors would have to be closed when the enumerable is disposed. This means the consuming application (e.g. GraphQL) must hold state during the pagination, which doesn't fit well with the disconnected model. Explicit keyset navigation doesn't require this - you just need to provide data on the row fetched in the last page, and go from there.
  • There may be some perf advantages in using cursors (same query being executed rather than multiple ones), but it isn't very clear at this point.
• Some differences between relational cursors and Cosmos continuation tokens:
  • A Cosmos continuation token doesn't hold up any server-side resources, whereas a relational cursor must be closed.
  • Cosmos is free to return less than MaxItemCount, so if the goal is to definitely return a page of 10, the application is responsible for refetching (EF in our scenario).