refactor: sql_json view endpoint: encapsulate ctas parameters by ofekisr · Pull Request #16548 · apache/superset

ofekisr · 2021-09-01T16:33:45Z

SUMMARY
The sql_json view code in superset core view without any "clean code" standard and it does not adopt any software development principle.
This is the fourth PR in the sequence of future PRs (previous PR) try to solve it by refactoring the code.
The PR focus of encapsulating ctas (create table as select) parameters
TESTING INSTRUCTIONS
there are no logic changes so it implies on the current tests.

codecov · 2021-09-05T12:27:33Z

Codecov Report

Merging #16548 (e9b794b) into master (e947f8a) will increase coverage by 0.02%.
The diff coverage is 94.00%.

@@            Coverage Diff             @@
##           master   #16548      +/-   ##
==========================================
+ Coverage   76.71%   76.74%   +0.02%     
==========================================
  Files        1002     1003       +1     
  Lines       53801    53856      +55     
  Branches     6858     6858              
==========================================
+ Hits        41276    41330      +54     
- Misses      12288    12289       +1     
  Partials      237      237

Flag	Coverage Δ
hive	`81.11% <94.00%> (+0.03%)`	⬆️
mysql	`81.57% <93.75%> (+0.03%)`	⬆️
postgres	`81.60% <94.00%> (+0.03%)`	⬆️
presto	`81.40% <94.00%> (+0.03%)`	⬆️
python	`82.11% <94.00%> (+0.03%)`	⬆️
sqlite	`81.25% <93.75%> (+0.07%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
superset/views/core.py	`75.57% <75.00%> (-0.02%)`	⬇️
superset/utils/sqllab_execution_context.py	`91.81% <95.65%> (+1.67%)`	⬆️
superset/jinja_context.py	`89.36% <0.00%> (ø)`
superset/db_engine_specs/__init__.py	`50.00% <0.00%> (ø)`
superset/db_engine_specs/aurora.py	`100.00% <0.00%> (ø)`
superset/tasks/async_queries.py	`96.96% <0.00%> (+0.19%)`	⬆️
superset/views/utils.py	`83.57% <0.00%> (+0.24%)`	⬆️
superset/db_engine_specs/rockset.py	`53.12% <0.00%> (+1.60%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e947f8a...e9b794b. Read the comment docs.

amitmiran137 · 2021-09-05T13:21:38Z



+@pytest.mark.sqllab
 class TestSqlLab(SupersetTestCase):


Remove that custom mark

amitmiran137

A few minor comments

amitmiran137

Another small refactor
Look good
Minor comments will be addressed in the following PR

…#16548) * refactor sql_json view endpoint: encapsulate ctas parameters * fix failed tests * fix failed tests and ci issues

* add support for adhoc columns to api and sqla model * fix some types * fix duplicates in column names * fix more lint * fix schema and dedup * clean up some logic * first pass at fixing viz.py * Add frontend support for adhoc columns * Add title edit * Fix showing custom title * Use column name as default value in sql editor * fix: Adds a loading message when needed in the Select component (#16531) * fix(tests): make parquet select deterministic with order by (#16570) * bump emotion to help with cache clobbering (#16559) * fix: Support Jinja template functions in global async queries (#16412) * Support Jinja template functions in async queries * Pylint * Add tests for async tasks * Remove redundant has_request_context check * fix: impersonate user label/tooltip (#16573) * docs: update for small typos (#16568) * feat: Add Aurora Data API engine spec (#16535) * feat: Add Aurora Data API engine spec * Fix lint * refactor: sql_json view endpoint: encapsulate ctas parameters (#16548) * refactor sql_json view endpoint: encapsulate ctas parameters * fix failed tests * fix failed tests and ci issues * refactor sql_json view endpoint: separate concern into ad hod method (#16595) * feat: Experimental cross-filter plugins (#16594) * fix:fix get permission function * feat: add cross filter chart in charts gallery under FF * chore(deps): bump superset-ui to 0.18.2 (#16601) * update type guard references * fix imports * update series_columns schema * Add changes that got lost in rebase * Use current columns name or expression as sql editor init value * add integration test and do minor fixes * Bump superset-ui * fix linting issue * bump superset-ui to 0.18.22 * resolve merge conflict * lint * fix select filter infinite loop * bump superset-ui to 0.18.23 * Fix auto setting column popover title * Enable adhoc columns only if UX_BETA enabled * put back removed test * Move popover height and width to constants * Refactor big ternary expression Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com> Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com> Co-authored-by: Elizabeth Thompson <eschutho@gmail.com> Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com> Co-authored-by: Beto Dealmeida <roberto@dealmeida.net> Co-authored-by: joeADSP <75027008+joeADSP@users.noreply.github.com> Co-authored-by: ofekisr <35701650+ofekisr@users.noreply.github.com> Co-authored-by: simcha90 <56388545+simcha90@users.noreply.github.com>

…#16548) * refactor sql_json view endpoint: encapsulate ctas parameters * fix failed tests * fix failed tests and ci issues

* add support for adhoc columns to api and sqla model * fix some types * fix duplicates in column names * fix more lint * fix schema and dedup * clean up some logic * first pass at fixing viz.py * Add frontend support for adhoc columns * Add title edit * Fix showing custom title * Use column name as default value in sql editor * fix: Adds a loading message when needed in the Select component (#16531) * fix(tests): make parquet select deterministic with order by (#16570) * bump emotion to help with cache clobbering (#16559) * fix: Support Jinja template functions in global async queries (#16412) * Support Jinja template functions in async queries * Pylint * Add tests for async tasks * Remove redundant has_request_context check * fix: impersonate user label/tooltip (#16573) * docs: update for small typos (#16568) * feat: Add Aurora Data API engine spec (#16535) * feat: Add Aurora Data API engine spec * Fix lint * refactor: sql_json view endpoint: encapsulate ctas parameters (#16548) * refactor sql_json view endpoint: encapsulate ctas parameters * fix failed tests * fix failed tests and ci issues * refactor sql_json view endpoint: separate concern into ad hod method (#16595) * feat: Experimental cross-filter plugins (#16594) * fix:fix get permission function * feat: add cross filter chart in charts gallery under FF * chore(deps): bump superset-ui to 0.18.2 (#16601) * update type guard references * fix imports * update series_columns schema * Add changes that got lost in rebase * Use current columns name or expression as sql editor init value * add integration test and do minor fixes * Bump superset-ui * fix linting issue * bump superset-ui to 0.18.22 * resolve merge conflict * lint * fix select filter infinite loop * bump superset-ui to 0.18.23 * Fix auto setting column popover title * Enable adhoc columns only if UX_BETA enabled * put back removed test * Move popover height and width to constants * Refactor big ternary expression Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com> Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com> Co-authored-by: Elizabeth Thompson <eschutho@gmail.com> Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com> Co-authored-by: Beto Dealmeida <roberto@dealmeida.net> Co-authored-by: joeADSP <75027008+joeADSP@users.noreply.github.com> Co-authored-by: ofekisr <35701650+ofekisr@users.noreply.github.com> Co-authored-by: simcha90 <56388545+simcha90@users.noreply.github.com>

…#16548) * refactor sql_json view endpoint: encapsulate ctas parameters * fix failed tests * fix failed tests and ci issues

* add support for adhoc columns to api and sqla model * fix some types * fix duplicates in column names * fix more lint * fix schema and dedup * clean up some logic * first pass at fixing viz.py * Add frontend support for adhoc columns * Add title edit * Fix showing custom title * Use column name as default value in sql editor * fix: Adds a loading message when needed in the Select component (apache#16531) * fix(tests): make parquet select deterministic with order by (apache#16570) * bump emotion to help with cache clobbering (apache#16559) * fix: Support Jinja template functions in global async queries (apache#16412) * Support Jinja template functions in async queries * Pylint * Add tests for async tasks * Remove redundant has_request_context check * fix: impersonate user label/tooltip (apache#16573) * docs: update for small typos (apache#16568) * feat: Add Aurora Data API engine spec (apache#16535) * feat: Add Aurora Data API engine spec * Fix lint * refactor: sql_json view endpoint: encapsulate ctas parameters (apache#16548) * refactor sql_json view endpoint: encapsulate ctas parameters * fix failed tests * fix failed tests and ci issues * refactor sql_json view endpoint: separate concern into ad hod method (apache#16595) * feat: Experimental cross-filter plugins (apache#16594) * fix:fix get permission function * feat: add cross filter chart in charts gallery under FF * chore(deps): bump superset-ui to 0.18.2 (apache#16601) * update type guard references * fix imports * update series_columns schema * Add changes that got lost in rebase * Use current columns name or expression as sql editor init value * add integration test and do minor fixes * Bump superset-ui * fix linting issue * bump superset-ui to 0.18.22 * resolve merge conflict * lint * fix select filter infinite loop * bump superset-ui to 0.18.23 * Fix auto setting column popover title * Enable adhoc columns only if UX_BETA enabled * put back removed test * Move popover height and width to constants * Refactor big ternary expression Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com> Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com> Co-authored-by: Elizabeth Thompson <eschutho@gmail.com> Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com> Co-authored-by: Beto Dealmeida <roberto@dealmeida.net> Co-authored-by: joeADSP <75027008+joeADSP@users.noreply.github.com> Co-authored-by: ofekisr <35701650+ofekisr@users.noreply.github.com> Co-authored-by: simcha90 <56388545+simcha90@users.noreply.github.com>

refactor sql_json view endpoint: encapsulate ctas parameters

20d784b

pull-request-size Bot added the size/L label Sep 1, 2021

junlincc requested review from betodealmeida and eschutho September 1, 2021 16:34

ofekisr added 2 commits September 1, 2021 20:06

fix failed tests

581bbc8

fix failed tests and ci issues

e9b794b

amitmiran137 reviewed Sep 5, 2021

View reviewed changes

amitmiran137 approved these changes Sep 5, 2021

View reviewed changes

amitmiran137 merged commit be77ad2 into apache:master Sep 5, 2021

ofekisr mentioned this pull request Sep 5, 2021

refactor: sql_json view endpoint: separate concern into ad hod method #16595

Merged

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.4.0 First shipped in 1.4.0 labels Mar 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: sql_json view endpoint: encapsulate ctas parameters#16548

refactor: sql_json view endpoint: encapsulate ctas parameters#16548
amitmiran137 merged 3 commits into
apache:masterfrom
nielsen-oss:refactor/sql_json_view4

ofekisr commented Sep 1, 2021 •

edited

Loading

Uh oh!

codecov Bot commented Sep 5, 2021 •

edited

Loading

Uh oh!

amitmiran137 Sep 5, 2021

Uh oh!

amitmiran137 left a comment

Uh oh!

amitmiran137 left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ofekisr commented Sep 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Sep 5, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

amitmiran137 Sep 5, 2021

Choose a reason for hiding this comment

Uh oh!

amitmiran137 left a comment

Choose a reason for hiding this comment

Uh oh!

amitmiran137 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ofekisr commented Sep 1, 2021 •

edited

Loading

codecov Bot commented Sep 5, 2021 •

edited

Loading