[connectors] Make names non-nullable

[john-bodley]

This PR ensures that the various database values cannot be NULL which surfaced as a result #5445 where it became apparent there could be ill-defined and/or erroneous database values. This is also reflected in the CRUD views which is handled automatically by Flask-AppBuilder. Specifically the following additional fields are now non-NULL:

Druid

• Datasource: datasource_name
• Column: column_name
• Metric: metric_name, json

SQLAlchemy

• Table: table_name
• Column: column_name
• Metric: name, expression

The attached screenshot provides an example of a form comprising of a required field (*) and form validation upon saving.

Note this PR also includes a migration to update the schema to include the non-nullable fields. In order for this to happen however erroneous records need to be removed and ill-defined records need to be mutated.

Erroneous records are defined when:

• A Druid or SQLAlchemy column name and column expression does not exist.
• A Druid or SQLAlchemy metric name and metric expression does not exist.

Ill-defined records are defined when:

• A Druid datasource or SQLAlchemy table name does not exist.
• A Druid or SQLAlchemy column name does not exist however there is a column expression defined.
• A Druid or SQLAlchemy metric name xor metric expression exists.

Given that the CRUD model is broken with NULL values the migration mutates the relevant fields to contain the migration_40f150716b13_ prefix. This allows the admin to then provide a cleanup pass if required to remove or update the violating records.

to: @graceguo-supercat @michellethomas @mistercrunch @xtinec

[codecov-io]

Codecov Report

Merging #7067 into master will increase coverage by 0.02%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master   #7067      +/-   ##
=========================================
+ Coverage   64.48%   64.5%   +0.02%     
=========================================
  Files         421     421              
  Lines       20547   20559      +12     
  Branches     2250    2250              
=========================================
+ Hits        13249   13261      +12     
  Misses       7171    7171              
  Partials      127     127

Impacted Files	Coverage Δ
superset/connectors/base/models.py	90.37% <100%> (ø)	⬆️
superset/connectors/druid/models.py	82.71% <100%> (ø)	⬆️
superset/connectors/druid/views.py	67.53% <100%> (+1.31%)	⬆️
superset/connectors/sqla/models.py	81.86% <100%> (ø)	⬆️
superset/connectors/sqla/views.py	65% <100%> (+1.84%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 30f88ca...3250807. Read the comment docs.

[mistercrunch]

Outside of typos LGTM

[john-bodley]

@mistercrunch apologies for the copy-and-paste typos. I realized that the edit_form_extra_fields is not required as Flask-AppBuilder handles this for non-nullable fields.

[mistercrunch]

Sweet, well at least the typo made us discover that it wasn't necessary :)

LGTM

[john-bodley]

Closing this in favor of resurrecting (and augmenting) the following PRs:

• [schema] Updating the tables schema #5449
• [schema] Updating the datasources schema #5451
• [schema] Updating the base column schema #5452
• [schema] Updating the base metric schema #5453 (needs to be augmented for non-nullable metric expressions)