Bug: Unexpected behavior with dj.Top()

[mdmelin]

Bug Report

Description

Hi @dimitri-yatsenko,

I've noticed some strange behavior when trying to restrict a query with dj.Top(), and I'm not sure if it's intended or not. The returned query seems to change as the limit argument changes. (See attached screenshot)

Reproducibility

Include:

• OS: Windows
• Python Version: 3.10.19
• DataJoint Version: 0.14.6

Expected Behavior

I would expect the top of the returned query to be the same, regardless of the limit that is set (especially when order_by gets applied)

Screenshots

[mdmelin]

I understand that the limit probably gets applied first, but even when setting it to None, the order_by argument does not behave as expected.

[dimitri-yatsenko]

Hi @mdmelin, thanks for reporting this.

The root cause is in how the preview/display works in v0.14.x. When you view a query in a Jupyter notebook or call preview(), the display code internally calls fetch(limit=display_limit), which creates a new Top(limit=..., order_by="KEY") that overwrites the Top you applied to the query. So even though your query has dj.Top(10, order_by='some_col DESC'), the display shows rows in primary key order with a different limit — ignoring your ordering.

This has been resolved in DataJoint 2.0 across two changes:

1. FEAT: dj.Top orders the preview with order_by #1242 / PR feat: dj.Top order inheritance, part_integrity parameter, and storage fixes #1312 — fix(Top): allow order_by=None to inherit existing ordering. Top now supports order_by=None to inherit ordering from an existing Top, and multiple Top restrictions compose properly (merge when ordering matches, subquery when it conflicts).

2. PR DataJoint 2.0: Modern Fetch and Insert APIs #1304 — Modernize fetch API with composable output methods. The fetch/preview pipeline was rewritten so that preview() calls to_arrays(limit=...) which uses _apply_top() — this only creates a new Top when explicitly requested, and uses order_by=None (inherit) rather than overwriting with "KEY".

Together, these ensure that when you apply dj.Top(10, order_by='score DESC') to a query, the display preview respects your ordering and limit.

The fix requires upgrading to DataJoint 2.0. As a workaround in v0.14.6, don't rely on the display to verify ordering — use .fetch() directly with explicit order_by to confirm results:

(query & dj.Top(10, order_by='score DESC')).fetch(order_by='score DESC', format="frame")