fix: import ZIP files that have been modified by betodealmeida · Pull Request #12425 · apache/superset

betodealmeida · 2021-01-11T23:09:00Z

SUMMARY

If the user modifies the ZIP file from an exported object, Mac OS will add system files to the ZIP, breaking the import.

This PR makes the import mechanism ignore those system files.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A

TEST PLAN

Exported chart
Unzipped file and modified chart YAML
Compressed directory into a new ZIP file
Import fails; with this PR it succeeds.

ADDITIONAL INFORMATION

codecov-io · 2021-01-12T00:42:04Z

Codecov Report

Merging #12425 (08a7db4) into master (ddbcf5b) will increase coverage by 0.23%.
The diff coverage is 87.50%.

@@            Coverage Diff             @@
##           master   #12425      +/-   ##
==========================================
+ Coverage   62.53%   62.76%   +0.23%     
==========================================
  Files        1014     1015       +1     
  Lines       49509    49537      +28     
  Branches     5078     5080       +2     
==========================================
+ Hits        30962    31094     +132     
+ Misses      18337    18234     -103     
+ Partials      210      209       -1

Flag	Coverage Δ
javascript	`60.75% <ø> (+0.03%)`	⬆️
python	`64.08% <87.50%> (+0.35%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
superset/commands/importers/v1/utils.py	`86.04% <85.71%> (-0.07%)`	⬇️
superset/charts/api.py	`81.51% <100.00%> (+0.05%)`	⬆️
superset/utils/core.py	`88.56% <0.00%> (-1.31%)`	⬇️
superset-frontend/src/components/Select/styles.tsx	`61.33% <0.00%> (-1.17%)`	⬇️
...rontend/src/explore/components/DatasourcePanel.tsx	`85.71% <0.00%> (-0.34%)`	⬇️
superset/viz_sip38.py	`0.00% <0.00%> (ø)`
superset/css_templates/api.py	`95.74% <0.00%> (ø)`
superset/annotation_layers/api.py	`83.19% <0.00%> (ø)`
...set-frontend/src/common/components/Modal/Modal.tsx	`100.00% <0.00%> (ø)`
...end/src/components/Select/SupersetStyledSelect.tsx	`88.29% <0.00%> (ø)`
... and 17 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ddbcf5b...08a7db4. Read the comment docs.

hughhhh

👍

* fix: import ZIP files that have been modified * Add unit test

* release: bump to 1.0.0 and CHANGELOG * fix(explore): long metric name display (#12387) * fix(explore): long metric name display * add tooltip to control * chore: Show datasets when search input is empty (#12391) * chore: Fix typo “Rest” to “Reset” (#12392) * chore: upgrade eslint, babel, and prettier (#12393) * feat(explore): add tooltip to timepicker label (#12401) * chore: change Datasource to Dataset in Explore ui (#12402) * chore(explore):change dataset to datasource in ui * modal * Add space * Changing it back🤦🏾‍♀️ * Chargeback * fix: Refresh Interval Modal dropdown (#12406) * fix(native-filters): incorrect queriesData state (#12409) * refactor: from superset.utils.core break down date_parser (#12408) * Fixes control panel fields styling (#12236) (#12326) * feat: Resizable dataset and controls panels on Explore view (#12411) * Implement resizable panels on explore view * Optimize chart rendering while resizing * Make dataset column narrower Co-authored-by: Evan Rusackas <evan@preset.io> * fix(dashboard): artefacts shown while drag and dropping deck.gl charts (#12418) * [12181] Fix artifacts while drag and dropping deck.gl charts. * Run prettier * bump superset-ui packages for rolling window change (#12426) * chore: bump superset-ui deckgl plugin (#12466) * fix: do not show vertical scrollbar for charts in dashboard (#12478) * fix: do not show vertical scrollbar for charts in dashboard * Proper fix for #11419 Co-authored-by: Jesse Yang <jesse.yang@airbnb.com> * fix(dashboard): use datasource id from slice metadata (#12483) * fix(timepicker): make pyparsing thread safe (#12489) * fix: make pyparsing thread safe * remove parenthesis for decorator * fix (SQL Lab): disappearing results on tab switch (#12472) * fix (SQL Lab): disappearing results on tab switch * Remove state * Fix test * fix: import ZIP files that have been modified (#12425) * fix: import ZIP files that have been modified * Add unit test * update changelog with rc2 entries * fix: impose dataset ownership check on old API (#12491) * fix: impose dataset ownership check on old API * update UPDATING.md * partially protect the old MVC also * prevent metric and column add and update * ci: remove refs/tags from docker tags on a release (#12518) * ci: remove refs/tags from docker tags on a release * wider head * fix: lowercase all columns in examples (#12530) * fix(explore): time table control panel (#12532) * fix(explore): Add Time section back to FilterBox (#12537) * Fixing Pinot queries for time granularities: WEEKS/MONTHS/QUARTERS/YEARS (#12536) * fix: Select options overflowing Save chart modal on Explore view (#12522) * Fix select options overflowing modal * fix unit test Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com> * Fix list filters vertical alignment (#12497) * feat(db-engine): Add support for Apache Solr (#12403) * [db engine] Add support for Apache Solr * Fixing typo * chore: rename docker image in build_docker_image.sh, docker-compose.yml and helm values.yaml (#12337) * add rc3 changelog entries * fix: Popover closes on change of dropdowns values (#12410) * fix: Add MAX_SQL_ROW value to LIMIT_DROPDOWN (#12555) * fix(viz): missing groupby and broken adhoc metrics for boxplot (#12556) * fix: height on grid results (#12558) * fix: case expression should not have double quotes (#12562) * Fix 500 error when loading dashboards with slice having deleted dataset (#12535) * add rc4 changelog entries * Fixed typo on line 348 * Added files Co-authored-by: Daniel Gaspar <danielvazgaspar@gmail.com> Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com> Co-authored-by: Geido <60598000+geido@users.noreply.github.com> Co-authored-by: Junlin Chen <junlin@preset.io> Co-authored-by: Jesse Yang <jesse.yang@airbnb.com> Co-authored-by: Agata Stawarz <47450693+agatapst@users.noreply.github.com> Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com> Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com> Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com> Co-authored-by: Evan Rusackas <evan@preset.io> Co-authored-by: Kasia Kucharczyk <2536609+kkucharc@users.noreply.github.com> Co-authored-by: Phillip Kelley-Dotson <pkelleydotson@yahoo.com> Co-authored-by: Grace Guo <grace.guo@airbnb.com> Co-authored-by: Beto Dealmeida <roberto@dealmeida.net> Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com> Co-authored-by: Xiang Fu <fx19880617@gmail.com> Co-authored-by: Ahmed Adel <github@aadel.io> Co-authored-by: Amit Miran <47772523+amitmiran137@users.noreply.github.com> Co-authored-by: Hugh A. Miles II <hughmil3s@gmail.com> Co-authored-by: Shuyao Bi <shuyaob@andrew.cmu.edu> Co-authored-by: Lyndsi Kay Williams <lyndsikaywilliams@Lyndsis-MacBook-Pro.local>

* fix: import ZIP files that have been modified * Add unit test

* release: bump to 1.0.0 and CHANGELOG * fix(explore): long metric name display (apache#12387) * fix(explore): long metric name display * add tooltip to control * chore: Show datasets when search input is empty (apache#12391) * chore: Fix typo “Rest” to “Reset” (apache#12392) * chore: upgrade eslint, babel, and prettier (apache#12393) * feat(explore): add tooltip to timepicker label (apache#12401) * chore: change Datasource to Dataset in Explore ui (apache#12402) * chore(explore):change dataset to datasource in ui * modal * Add space * Changing it back🤦🏾‍♀️ * Chargeback * fix: Refresh Interval Modal dropdown (apache#12406) * fix(native-filters): incorrect queriesData state (apache#12409) * refactor: from superset.utils.core break down date_parser (apache#12408) * Fixes control panel fields styling (apache#12236) (apache#12326) * feat: Resizable dataset and controls panels on Explore view (apache#12411) * Implement resizable panels on explore view * Optimize chart rendering while resizing * Make dataset column narrower Co-authored-by: Evan Rusackas <evan@preset.io> * fix(dashboard): artefacts shown while drag and dropping deck.gl charts (apache#12418) * [12181] Fix artifacts while drag and dropping deck.gl charts. * Run prettier * bump superset-ui packages for rolling window change (apache#12426) * chore: bump superset-ui deckgl plugin (apache#12466) * fix: do not show vertical scrollbar for charts in dashboard (apache#12478) * fix: do not show vertical scrollbar for charts in dashboard * Proper fix for apache#11419 Co-authored-by: Jesse Yang <jesse.yang@airbnb.com> * fix(dashboard): use datasource id from slice metadata (apache#12483) * fix(timepicker): make pyparsing thread safe (apache#12489) * fix: make pyparsing thread safe * remove parenthesis for decorator * fix (SQL Lab): disappearing results on tab switch (apache#12472) * fix (SQL Lab): disappearing results on tab switch * Remove state * Fix test * fix: import ZIP files that have been modified (apache#12425) * fix: import ZIP files that have been modified * Add unit test * update changelog with rc2 entries * fix: impose dataset ownership check on old API (apache#12491) * fix: impose dataset ownership check on old API * update UPDATING.md * partially protect the old MVC also * prevent metric and column add and update * ci: remove refs/tags from docker tags on a release (apache#12518) * ci: remove refs/tags from docker tags on a release * wider head * fix: lowercase all columns in examples (apache#12530) * fix(explore): time table control panel (apache#12532) * fix(explore): Add Time section back to FilterBox (apache#12537) * Fixing Pinot queries for time granularities: WEEKS/MONTHS/QUARTERS/YEARS (apache#12536) * fix: Select options overflowing Save chart modal on Explore view (apache#12522) * Fix select options overflowing modal * fix unit test Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com> * Fix list filters vertical alignment (apache#12497) * feat(db-engine): Add support for Apache Solr (apache#12403) * [db engine] Add support for Apache Solr * Fixing typo * chore: rename docker image in build_docker_image.sh, docker-compose.yml and helm values.yaml (apache#12337) * add rc3 changelog entries * fix: Popover closes on change of dropdowns values (apache#12410) * fix: Add MAX_SQL_ROW value to LIMIT_DROPDOWN (apache#12555) * fix(viz): missing groupby and broken adhoc metrics for boxplot (apache#12556) * fix: height on grid results (apache#12558) * fix: case expression should not have double quotes (apache#12562) * Fix 500 error when loading dashboards with slice having deleted dataset (apache#12535) * add rc4 changelog entries * Fixed typo on line 348 * Added files Co-authored-by: Daniel Gaspar <danielvazgaspar@gmail.com> Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com> Co-authored-by: Geido <60598000+geido@users.noreply.github.com> Co-authored-by: Junlin Chen <junlin@preset.io> Co-authored-by: Jesse Yang <jesse.yang@airbnb.com> Co-authored-by: Agata Stawarz <47450693+agatapst@users.noreply.github.com> Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com> Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com> Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com> Co-authored-by: Evan Rusackas <evan@preset.io> Co-authored-by: Kasia Kucharczyk <2536609+kkucharc@users.noreply.github.com> Co-authored-by: Phillip Kelley-Dotson <pkelleydotson@yahoo.com> Co-authored-by: Grace Guo <grace.guo@airbnb.com> Co-authored-by: Beto Dealmeida <roberto@dealmeida.net> Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com> Co-authored-by: Xiang Fu <fx19880617@gmail.com> Co-authored-by: Ahmed Adel <github@aadel.io> Co-authored-by: Amit Miran <47772523+amitmiran137@users.noreply.github.com> Co-authored-by: Hugh A. Miles II <hughmil3s@gmail.com> Co-authored-by: Shuyao Bi <shuyaob@andrew.cmu.edu> Co-authored-by: Lyndsi Kay Williams <lyndsikaywilliams@Lyndsis-MacBook-Pro.local>

fix: import ZIP files that have been modified

758d726

betodealmeida requested a review from dpgaspar January 11, 2021 23:09

pull-request-size Bot added the size/S label Jan 11, 2021

betodealmeida requested a review from hughhhh January 11, 2021 23:09

hughhhh reviewed Jan 12, 2021

View reviewed changes

Comment thread superset/commands/importers/v1/utils.py

Add unit test

08a7db4

hughhhh self-requested a review January 12, 2021 19:43

hughhhh approved these changes Jan 12, 2021

View reviewed changes

betodealmeida merged commit e47350e into apache:master Jan 12, 2021

villebro pushed a commit that referenced this pull request Jan 13, 2021

fix: import ZIP files that have been modified (#12425)

78e4b02

* fix: import ZIP files that have been modified * Add unit test

villebro added the v1.0 label Jan 14, 2021

betodealmeida mentioned this pull request Jan 14, 2021

fix: ignore system files on import #12526

Merged

6 tasks

amitmiran137 pushed a commit to nielsen-oss/superset that referenced this pull request Jan 14, 2021

fix: import ZIP files that have been modified (apache#12425)

b83bf58

* fix: import ZIP files that have been modified * Add unit test

villebro pushed a commit to preset-io/superset that referenced this pull request Jan 15, 2021

fix: import ZIP files that have been modified (apache#12425)

b21c90d

* fix: import ZIP files that have been modified * Add unit test

junlincc added the preset-io label Mar 18, 2021

mistercrunch added 🍒 1.0.0 Cherry-picked to 1.0.0 🍒 1.0.1 Cherry-picked to 1.0.1 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.2.0 First shipped in 1.2.0 labels Mar 12, 2024

qfcwell pushed a commit to qfcwell/superset that referenced this pull request May 12, 2026

fix: import ZIP files that have been modified (apache#12425)

320b4ab

* fix: import ZIP files that have been modified * Add unit test

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: import ZIP files that have been modified#12425

fix: import ZIP files that have been modified#12425
betodealmeida merged 2 commits into
apache:masterfrom
betodealmeida:fix_import_modified_zip

betodealmeida commented Jan 11, 2021

Uh oh!

codecov-io commented Jan 12, 2021 •

edited

Loading

Uh oh!

Uh oh!

hughhhh left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

betodealmeida commented Jan 11, 2021

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN

ADDITIONAL INFORMATION

Uh oh!

codecov-io commented Jan 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

hughhhh left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

codecov-io commented Jan 12, 2021 •

edited

Loading