Skip to content

[schema] Ensuring that the slice datasource fields are non-nullable#7114

Closed
john-bodley wants to merge 1 commit into
apache:masterfrom
john-bodley:john-bodley--update-slices
Closed

[schema] Ensuring that the slice datasource fields are non-nullable#7114
john-bodley wants to merge 1 commit into
apache:masterfrom
john-bodley:john-bodley--update-slices

Conversation

@john-bodley
Copy link
Copy Markdown
Member

We've noticed a number of anomalies in our database caused by ill-defined forms and/or table schema definitions. This PR resolves a number of issues related to the slices table including:

  • Ensures that the datasource_id column is not-NULL.
  • Ensures that the datasource_name column is not-NULL.
  • Ensures that the datasource_type column is non-NULL.

Long terms we should look at trying to deprecate the denormalized datasource_name column which can (and should) be inferred from the (datasource_id, datasource_type) tuple. I believe the reason for its existence is mostly legacy however it may be required for importing/exporting of slices where one must identify entities by name rather than ID.

Note this migration will fail if the slices.datasource_id, slices.datasource_name, or slices.datasource_type column is NULL. One must manually fix these records as programmatically trying to remedy these invalid records is difficult as it may not be apparent what the corresponding datasource should be given the possibly lack of information. The following query determines which records are problematic:

SELECT 
    * 
FROM 
    slices
WHERE 
    datasource_id IS NULL OR 
    datasource_name IS NULL OR 
    datasource_type IS NULL

Note this PR is gated by #5445 and #7084 which ensure that empty strings associated with form-data wont persist in the database and is necessary for ensuring that the relevant entries are non-NULL.

to: @graceguo-supercat @michellethomas @mistercrunch

@john-bodley john-bodley force-pushed the john-bodley--update-slices branch from a14068f to 5ebfe9b Compare March 25, 2019 18:03
@john-bodley john-bodley added the risk:db-migration PRs that require a DB migration label Mar 25, 2019
@stale
Copy link
Copy Markdown

stale Bot commented May 24, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

@stale stale Bot added the inactive Inactive for >= 30 days label May 24, 2019
@stale stale Bot closed this May 31, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

inactive Inactive for >= 30 days risk:db-migration PRs that require a DB migration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant