Skip to content

[FEATURE] Pipeline name for synced table #5280

@malthe

Description

@malthe

Use-cases

When a synced table is created using the provider it is named:

Synced table: <catalog>.<schema>.<table> <random id>

This is perhaps fine, but it gets weird if we then add multiple table syncs to the same pipeline.

Attempted Solutions

You can change this using the UI.

Proposal

There's something strange about the current API and it's also not congruent with the API that's used by the UI.

  1. It should be possible to specify the pipeline name.

  2. Like the /ajax-api/2.0/pipelines/<id> endpoint (used by the UI), each table sync should be defined within the same pipeline definition, obviating the need for awkward new/existing-pipeline chaining (which also means the pipeline gets assigned a name based on the initial entry only):

    ...
    managed_definition:
      database_table_sync:
         sinks:
         - dest_table: ...
           primary_key: ...

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions