fix(tests): make parquet select deterministic with order by by villebro · Pull Request #16570 · apache/superset

villebro · 2021-09-02T17:00:03Z

SUMMARY

A recently added unit test is causing flaky test results due to non-deterministic ordering of results.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

Has associated issue:
Required feature flags:
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

codecov · 2021-09-02T17:18:53Z

Codecov Report

Merging #16570 (442a5cb) into master (02798a3) will increase coverage by 0.51%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #16570      +/-   ##
==========================================
+ Coverage   76.19%   76.71%   +0.51%     
==========================================
  Files        1002     1002              
  Lines       53813    53815       +2     
  Branches     7287     7287              
==========================================
+ Hits        41005    41282     +277     
+ Misses      12568    12293     -275     
  Partials      240      240

Flag	Coverage Δ
hive	`81.12% <ø> (+0.03%)`	⬆️
mysql	`81.54% <ø> (?)`
postgres	`81.57% <ø> (?)`
presto	`81.40% <ø> (?)`
python	`82.07% <ø> (+0.99%)`	⬆️
sqlite	`81.17% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
superset/db_engines/hive.py	`82.14% <0.00%> (-3.05%)`	⬇️
superset/views/base_api.py	`98.31% <0.00%> (+0.42%)`	⬆️
superset/utils/core.py	`89.89% <0.00%> (+0.48%)`	⬆️
superset/security/manager.py	`91.84% <0.00%> (+0.52%)`	⬆️
superset/db_engine_specs/postgres.py	`97.36% <0.00%> (+0.87%)`	⬆️
superset/utils/cache.py	`74.52% <0.00%> (+0.94%)`	⬆️
superset/connectors/sqla/models.py	`89.70% <0.00%> (+1.42%)`	⬆️
superset/result_set.py	`98.38% <0.00%> (+1.61%)`	⬆️
superset/common/query_context.py	`90.74% <0.00%> (+2.31%)`	⬆️
superset/db_engine_specs/base.py	`88.39% <0.00%> (+2.51%)`	⬆️
... and 16 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 02798a3...442a5cb. Read the comment docs.

…6570)

* add support for adhoc columns to api and sqla model * fix some types * fix duplicates in column names * fix more lint * fix schema and dedup * clean up some logic * first pass at fixing viz.py * Add frontend support for adhoc columns * Add title edit * Fix showing custom title * Use column name as default value in sql editor * fix: Adds a loading message when needed in the Select component (#16531) * fix(tests): make parquet select deterministic with order by (#16570) * bump emotion to help with cache clobbering (#16559) * fix: Support Jinja template functions in global async queries (#16412) * Support Jinja template functions in async queries * Pylint * Add tests for async tasks * Remove redundant has_request_context check * fix: impersonate user label/tooltip (#16573) * docs: update for small typos (#16568) * feat: Add Aurora Data API engine spec (#16535) * feat: Add Aurora Data API engine spec * Fix lint * refactor: sql_json view endpoint: encapsulate ctas parameters (#16548) * refactor sql_json view endpoint: encapsulate ctas parameters * fix failed tests * fix failed tests and ci issues * refactor sql_json view endpoint: separate concern into ad hod method (#16595) * feat: Experimental cross-filter plugins (#16594) * fix:fix get permission function * feat: add cross filter chart in charts gallery under FF * chore(deps): bump superset-ui to 0.18.2 (#16601) * update type guard references * fix imports * update series_columns schema * Add changes that got lost in rebase * Use current columns name or expression as sql editor init value * add integration test and do minor fixes * Bump superset-ui * fix linting issue * bump superset-ui to 0.18.22 * resolve merge conflict * lint * fix select filter infinite loop * bump superset-ui to 0.18.23 * Fix auto setting column popover title * Enable adhoc columns only if UX_BETA enabled * put back removed test * Move popover height and width to constants * Refactor big ternary expression Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com> Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com> Co-authored-by: Elizabeth Thompson <eschutho@gmail.com> Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com> Co-authored-by: Beto Dealmeida <roberto@dealmeida.net> Co-authored-by: joeADSP <75027008+joeADSP@users.noreply.github.com> Co-authored-by: ofekisr <35701650+ofekisr@users.noreply.github.com> Co-authored-by: simcha90 <56388545+simcha90@users.noreply.github.com>

…6570)

* add support for adhoc columns to api and sqla model * fix some types * fix duplicates in column names * fix more lint * fix schema and dedup * clean up some logic * first pass at fixing viz.py * Add frontend support for adhoc columns * Add title edit * Fix showing custom title * Use column name as default value in sql editor * fix: Adds a loading message when needed in the Select component (#16531) * fix(tests): make parquet select deterministic with order by (#16570) * bump emotion to help with cache clobbering (#16559) * fix: Support Jinja template functions in global async queries (#16412) * Support Jinja template functions in async queries * Pylint * Add tests for async tasks * Remove redundant has_request_context check * fix: impersonate user label/tooltip (#16573) * docs: update for small typos (#16568) * feat: Add Aurora Data API engine spec (#16535) * feat: Add Aurora Data API engine spec * Fix lint * refactor: sql_json view endpoint: encapsulate ctas parameters (#16548) * refactor sql_json view endpoint: encapsulate ctas parameters * fix failed tests * fix failed tests and ci issues * refactor sql_json view endpoint: separate concern into ad hod method (#16595) * feat: Experimental cross-filter plugins (#16594) * fix:fix get permission function * feat: add cross filter chart in charts gallery under FF * chore(deps): bump superset-ui to 0.18.2 (#16601) * update type guard references * fix imports * update series_columns schema * Add changes that got lost in rebase * Use current columns name or expression as sql editor init value * add integration test and do minor fixes * Bump superset-ui * fix linting issue * bump superset-ui to 0.18.22 * resolve merge conflict * lint * fix select filter infinite loop * bump superset-ui to 0.18.23 * Fix auto setting column popover title * Enable adhoc columns only if UX_BETA enabled * put back removed test * Move popover height and width to constants * Refactor big ternary expression Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com> Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com> Co-authored-by: Elizabeth Thompson <eschutho@gmail.com> Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com> Co-authored-by: Beto Dealmeida <roberto@dealmeida.net> Co-authored-by: joeADSP <75027008+joeADSP@users.noreply.github.com> Co-authored-by: ofekisr <35701650+ofekisr@users.noreply.github.com> Co-authored-by: simcha90 <56388545+simcha90@users.noreply.github.com>

…6570)

* add support for adhoc columns to api and sqla model * fix some types * fix duplicates in column names * fix more lint * fix schema and dedup * clean up some logic * first pass at fixing viz.py * Add frontend support for adhoc columns * Add title edit * Fix showing custom title * Use column name as default value in sql editor * fix: Adds a loading message when needed in the Select component (apache#16531) * fix(tests): make parquet select deterministic with order by (apache#16570) * bump emotion to help with cache clobbering (apache#16559) * fix: Support Jinja template functions in global async queries (apache#16412) * Support Jinja template functions in async queries * Pylint * Add tests for async tasks * Remove redundant has_request_context check * fix: impersonate user label/tooltip (apache#16573) * docs: update for small typos (apache#16568) * feat: Add Aurora Data API engine spec (apache#16535) * feat: Add Aurora Data API engine spec * Fix lint * refactor: sql_json view endpoint: encapsulate ctas parameters (apache#16548) * refactor sql_json view endpoint: encapsulate ctas parameters * fix failed tests * fix failed tests and ci issues * refactor sql_json view endpoint: separate concern into ad hod method (apache#16595) * feat: Experimental cross-filter plugins (apache#16594) * fix:fix get permission function * feat: add cross filter chart in charts gallery under FF * chore(deps): bump superset-ui to 0.18.2 (apache#16601) * update type guard references * fix imports * update series_columns schema * Add changes that got lost in rebase * Use current columns name or expression as sql editor init value * add integration test and do minor fixes * Bump superset-ui * fix linting issue * bump superset-ui to 0.18.22 * resolve merge conflict * lint * fix select filter infinite loop * bump superset-ui to 0.18.23 * Fix auto setting column popover title * Enable adhoc columns only if UX_BETA enabled * put back removed test * Move popover height and width to constants * Refactor big ternary expression Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com> Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com> Co-authored-by: Elizabeth Thompson <eschutho@gmail.com> Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com> Co-authored-by: Beto Dealmeida <roberto@dealmeida.net> Co-authored-by: joeADSP <75027008+joeADSP@users.noreply.github.com> Co-authored-by: ofekisr <35701650+ofekisr@users.noreply.github.com> Co-authored-by: simcha90 <56388545+simcha90@users.noreply.github.com>

fix(tests): make parquet select deterministic with order by

442a5cb

superset-github-bot Bot added the preset-io label Sep 2, 2021

pull-request-size Bot added the size/XS label Sep 2, 2021

villebro requested a review from michael-s-molina September 2, 2021 17:00

michael-s-molina approved these changes Sep 2, 2021

View reviewed changes

michael-s-molina merged commit 070fdbe into apache:master Sep 2, 2021

villebro added a commit to preset-io/superset that referenced this pull request Sep 6, 2021

fix(tests): make parquet select deterministic with order by (apache#1…

661e499

…6570)

opus-42 pushed a commit to opus-42/incubator-superset that referenced this pull request Nov 14, 2021

fix(tests): make parquet select deterministic with order by (apache#1…

62a5b83

…6570)

QAlexBall pushed a commit to QAlexBall/superset that referenced this pull request Dec 28, 2021

fix(tests): make parquet select deterministic with order by (apache#1…

2438d28

…6570)

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.4.0 First shipped in 1.4.0 labels Mar 13, 2024

qfcwell pushed a commit to qfcwell/superset that referenced this pull request May 12, 2026

fix(tests): make parquet select deterministic with order by (apache#1…

229138c

…6570)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(tests): make parquet select deterministic with order by#16570

fix(tests): make parquet select deterministic with order by#16570
michael-s-molina merged 1 commit into
apache:masterfrom
preset-io:villebro/fix-parquet-test

villebro commented Sep 2, 2021

Uh oh!

codecov Bot commented Sep 2, 2021 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

villebro commented Sep 2, 2021

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

Uh oh!

codecov Bot commented Sep 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov Bot commented Sep 2, 2021 •

edited

Loading