Skip to content

chore: Refactor ExploreMixin to power both Datasets (SqlaTable) and Query models#22853

Merged
hughhhh merged 35 commits into
masterfrom
fix-explore-mixin
Apr 10, 2023
Merged

chore: Refactor ExploreMixin to power both Datasets (SqlaTable) and Query models#22853
hughhhh merged 35 commits into
masterfrom
fix-explore-mixin

Conversation

@hughhhh
Copy link
Copy Markdown
Member

@hughhhh hughhhh commented Jan 25, 2023

SUMMARY

Cleaning up tech debt from intial SIP-68 work to move towards having newer models (Table, Dataset) to run in explore. Since we are deprioritizing that work, refactoring the ExploreMixin to power SqlaTable and Query which are legacy models. Main reason is to have one codepath on executing queries in explore vs. having divergent paths at the moment.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@rusackas
Copy link
Copy Markdown
Member

Pinging @villebro since I know he's keen on this. Might wanna fill out the PR description though, for the rest of the world ;)

@codecov
Copy link
Copy Markdown

codecov Bot commented Jan 25, 2023

Codecov Report

Merging #22853 (548682e) into master (8ef49a5) will increase coverage by 0.23%.
The diff coverage is 68.22%.

❗ Current head 548682e differs from pull request most recent head dfe7191. Consider uploading reports for the commit dfe7191 to get more accurate results

@@            Coverage Diff             @@
##           master   #22853      +/-   ##
==========================================
+ Coverage   67.70%   67.94%   +0.23%     
==========================================
  Files        1918     1918              
  Lines       74133    73873     -260     
  Branches     8052     8052              
==========================================
- Hits        50193    50190       -3     
+ Misses      21887    21630     -257     
  Partials     2053     2053              
Flag Coverage Δ
hive 53.16% <54.49%> (+0.42%) ⬆️
javascript 53.89% <0.00%> (ø)
mysql 79.21% <68.78%> (+0.69%) ⬆️
postgres 79.29% <68.78%> (+0.69%) ⬆️
presto ?
python 83.03% <69.31%> (+0.59%) ⬆️
sqlite 77.78% <68.78%> (+0.68%) ⬆️
unit 53.02% <40.21%> (+0.38%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...d/src/SqlLab/components/SaveDatasetModal/index.tsx 52.27% <0.00%> (ø)
superset-frontend/src/SqlLab/fixtures.ts 100.00% <ø> (ø)
...src/dashboard/components/PropertiesModal/index.tsx 64.03% <ø> (ø)
...nents/controls/MetricControl/AdhocMetricOption.jsx 77.77% <0.00%> (ø)
superset/migrations/env.py 0.00% <0.00%> (ø)
superset/views/dashboard/mixin.py 95.00% <ø> (ø)
superset/models/sql_lab.py 77.86% <42.85%> (+2.86%) ⬆️
superset/connectors/sqla/models.py 86.32% <63.63%> (-3.29%) ⬇️
superset/utils/core.py 90.99% <66.66%> (+0.08%) ⬆️
superset/models/helpers.py 68.69% <69.74%> (+30.04%) ⬆️
... and 4 more

... and 3 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@hughhhh hughhhh force-pushed the fix-explore-mixin branch 4 times, most recently from 36346e1 to 2b1414c Compare January 25, 2023 19:19
@hughhhh hughhhh changed the title chore: Refactor ExploreMixin to power both Datasets (SqlaTable) and Query models chore(WIP): Refactor ExploreMixin to power both Datasets (SqlaTable) and Query models Jan 25, 2023
@hughhhh hughhhh force-pushed the fix-explore-mixin branch 8 times, most recently from 743a478 to 3d7ae3d Compare January 26, 2023 16:47
@hughhhh
Copy link
Copy Markdown
Member Author

hughhhh commented Jan 26, 2023

/test_env up

@hughhhh
Copy link
Copy Markdown
Member Author

hughhhh commented Jan 26, 2023

/testenv up

@github-actions
Copy link
Copy Markdown
Contributor

@hughhhh Ephemeral environment spinning up at http://34.221.152.250:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

@jinghua-qa
Copy link
Copy Markdown
Member

issue 1:
When power=query, missing data type in the left data panel in explore
Screen Shot 2023-01-27 at 8 45 56 AM
issue 2:
Column dropdown is empty in Metric when power=query

Screen.Recording.2023-01-27.at.8.46.46.AM.mov

issue 3:
Not able to remove metrics in explore when power = dataset

Screen.Recording.2023-01-27.at.8.52.24.AM.mov

@andrey-zayats
Copy link
Copy Markdown

andrey-zayats commented Jan 30, 2023

issue 4:
chart in explore returns a syntax error when the user applies time range (can't repro with ALL datasets).
on which I was able to repro (maybe they are broken):

  • video_game_sales
  • FCC 2018 Survey
  • birth_france_by_region
    image

issue 5:
aggregate field is not pre-filled in metric control when d&d any value from the list to the Metric section.
The issue is relevant for:

  • filters control (operator field should be pre-filled in with IN operator)
  • sort by control
aggregate.mp4

issue 6:

User is not able to create dataset when chart = power query.

create.dataset.mp4

Also user can't create chart using chart power query (the same error^).

save.chart.mp4

preset-machine pushed a commit to preset-io/superset that referenced this pull request Apr 10, 2023
hughhhh added a commit that referenced this pull request Apr 10, 2023
jinghua-qa added a commit to preset-io/superset that referenced this pull request Apr 13, 2023
jinghua-qa added a commit to preset-io/superset that referenced this pull request Apr 14, 2023
sebastianliebscher pushed a commit to sebastianliebscher/superset that referenced this pull request Apr 28, 2023
sebastianliebscher pushed a commit to sebastianliebscher/superset that referenced this pull request Apr 28, 2023
timestamp = dttm_col.get_timestamp_expression(
time_grain=time_grain, template_processor=template_processor
)
timestamp = dttm_col.get_timestamp_expression(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hughhhh this logic causes an error if the dttm_col is associated with a query as opposed to a table as dttm_col which has been coerced to be of type TableColumn has no table associated with it, i.e., dttm_col.table is None and thus this throws an exception.

john-bodley added a commit to john-bodley/superset that referenced this pull request May 31, 2023
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 3.0.0 First shipped in 3.0.0 labels Mar 13, 2024
@mistercrunch mistercrunch deleted the fix-explore-mixin branch March 26, 2024 17:55
qfcwell pushed a commit to qfcwell/superset that referenced this pull request May 12, 2026
qfcwell pushed a commit to qfcwell/superset that referenced this pull request May 12, 2026
qfcwell pushed a commit to qfcwell/superset that referenced this pull request May 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels preset:2023.13 risk:breaking-change Issues or PRs that will introduce breaking changes size/XXL 🚢 3.0.0 First shipped in 3.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants