Proposal: Getting rid of Neighbors in Path Queries

[neilkakkar]

Changes

I very much dislike neighbors function because: https://clickhouse.com/docs/en/sql-reference/functions/other-functions/#neighbor

1. It's implementation depends on a global ORDER BY clause in the subquery
2. Even then, if the data is not in the same block, this returns wrong results.

I'm proposing leveraging arrays to get the same job done. It uses a bit more memory than before (one more copy of the array for each person) - but gets rid of global sorting.

Another improvement this query makes: Ensures the groupArray works correctly.

From the docs: Values can be added to the array in any (indeterminate) order. To fix this, you order by the correct things in a subquery.

We're already ordering by timestamp in PathEventQuery, so don't need to order by personID in the outer clause.

Checklist

• All querysets/queries filter by Organization, by Team, and by User
• Django backend tests
• Jest frontend tests
• Cypress end-to-end tests
• Migrations are safe to run at scale (e.g. PostHog Cloud) – present proof if not obvious
• New/changed UI is decent on smartphones (viewport width around 360px)

[EDsCODE]

yeah good call