Proposal: Orbit Query Expressions

[dgeb]

In order to consolidate all finds and queries into a single query method (#202), it's necessary to first solve the problem posed by @gnarf: "How do we define queries?" (#192).

I'd like to propose "query expressions", a flexible and extensible way to define queries.

Note: an early version of this proposal used the name "OrbitQL", but I think that name is better left for a higher level representation of query expressions.

Query expressions are composable functions that can be expressed in JSON. Every expression has an op and args (short for "operation" and "arguments") and returns a value when evaluated. Arguments will be an array of values - whatever makes the most sense for a particular expression.

Several expressions will be shipped by default, although any number of custom expressions could be developed. Not every source has to understand every expression - they can simply throw an UnsupportedQueryExpressionException and allow another source to assist or rescue in the query's resolution.

A simple expression is get, which can be used to retrieve data at any path in a JSON document.

Here's a simple query of all planets:

query(
  {
    op: 'get',
    args: ['planet']
  }
)

Here's a query to find an individual planet:

query(
  {
    op: 'get',
    args: ['planet/p1']
  }
)

This could be written more concisely using an exp (short for "expression") helper, which simply outputs the JSON form of an expression:

query(exp('get', 'planet/p1');

A more advanced expression is filter, which takes the a path and where expression as ordered arguments.

The following query returns planets named jupiter:

query(
  exp('filter',
      'planet',
      exp('===', exp('get', 'attributes/name'), 'jupiter')
  })
});

Note the === expression tests any number of arguments for strict equality.

Complex conditionals could be expressed with ||, &&, !, and !! expressions.

The following query selects all planets named either jupiter or earth:

query(
  exp('filter',
      'planet',
      exp('||',
          exp('===', exp('get', 'attributes/name'), 'jupiter'),
          exp('===', exp('get', 'attributes/name'), 'earth'))
  })
});

And this query selects all moons of jupiter that have a mass over 9000:

query(
  exp('filter',
      'planet',
      exp('&&',
          exp('===', exp('get', 'relationships/planet'), 'planet:jupiter'),
          exp('>', exp('get', 'attributes/mass'), 9000))
  })
});

Note that all of these queries could be expanded to their simple form in JSON without use of the exp helper, and therefore would be serializable.

It's possible to define custom query expressions in other query languages like GraphQL:

query(
  exp('graphql', 'planet(id: p1) { attributes { name } }')
);

A graphql expression might only be understood by some future GraphQLSource.

Similarly, a fetch expression might accept named arguments for pagination, filter, and include parameters particular to JSONAPI:

query(
  exp('fetch', 
      'planet',
      {include: ['moons', 'sun'],
       page: {offset: 0, limit: 10}}
  })
);

The optional named arguments for a fetch expression might only be understood by the JSONAPISource.

---

As per #202, each Source that's Queryable will just have to support the single query method, and can of course choose to support only the level of query complexity that's appropriate.

It's envisioned that the new Store will provide convenience methods for performing simple queries, such as findRecords(type) and findRecord(type, id).

[dgeb]

I've updated this proposal so that expression arguments are simply an array of values, which allows for more natural and concise expressions than requiring that each argument be named. Of course, it's still possible for expressions to require a hash for one or more arguments, but it's no longer required.

Also, I've renamed the proposal from "OrbitQL" to "Orbit Query Expressions". We could potentially introduce OrbitQL later as a higher order representation of query expressions.

[dgeb]

As a thought experiment about "OrbitQL", it would be possible to distill a query expression such as the following:

query(
  exp('filter',
      'planet',
      exp('&&',
          exp('===', exp('get', 'relationships/planet'), 'planet:jupiter'),
          exp('>', exp('get', 'attributes/mass'), 9000))
  })
});

Into the following OrbitQL query string:

query(
  `filter('planet',
          and( strictEqual( get('relationships/planet'), 'planet:jupiter' ),
               gt( get('attributes/mass'), 9000) ))`
});

This leads me to think that expression operations should be words instead of symbols.

Anyone want to write a parser? 😛

[opsb]

Hmm, the OrbitQL query definitely looks appealing. I think a useful exercise for me would be to try mapping some expressions from ReQL to OrbitQL and see how they align. ReQL queries are composable so the mapping should be fairly tight. The area I'm most interested in looking at though is joins. Looking at https://rethinkdb.com/docs/table-joins/ I think something like this might work:

const planetCriteria = and( 
  strictEqual( get('relationships/planet'), 'planet:jupiter' ),
  gt( get('attributes/mass'), 9000) 
);

query(
  join(
    filter('planet', planetCriteria),
    get('moon'),
    'planetId'
  )
});

Note I'm imaging that OrbitQL expressions are javascript functions. Would make it simple to construct to queries, we'd need to namespace or scope the functions somehow though to avoid collisions.

[dgeb]

WIP coming to the rethink branch soon.

[nathanhammond]

Only one comment: we need to understand every single action the user can perform in the query expressions so that it is fully serializable. This means that you only get some subset of behaviors (&&, ||, !, +, thing.prop.value, etc.) unless that portion of the query is designed to be understood and executable by the recipient of the query. For safety reasons however it probably can't allow for arbitrary code ("this area can be any JS!") since executing it could result in a denial of service attack.

Starting to sound like SQL...

[dgeb]

Only one comment: we need to understand every single action the user can perform in the query expressions so that it is fully serializable.

@nathanhammond I agree. Orbit sources need to be able to deconstruct query expressions fully into an understandable and serializable form, or else they should raise an exception indicating that they don't support a particular query.

[benaubin]

For the jsonapi adapter, can we use filters? http://jsonapi.org/format/#fetching-filtering

[dgeb]

Orbit query expressions / builders introduced via #299