Test failure: `test_running_real_migrate_groups_job`

[github-actions]

❌ test_running_real_migrate_groups_job: databricks.sdk.errors.sdk.OperationFailed: failed to reach TERMINATED or SKIPPED, got RunLifeCycleState.INTERNAL_ERROR: Task apply_permissions failed with message: Workload failed, see run output for details. (3m10.621s)

databricks.sdk.errors.sdk.OperationFailed: failed to reach TERMINATED or SKIPPED, got RunLifeCycleState.INTERNAL_ERROR: Task apply_permissions failed with message: Workload failed, see run output for details.
[gw4] linux -- Python 3.10.19 /home/runner/work/ucx/ucx/.venv/bin/python
05:33 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.6Z5t/config.yml) doesn't exist.
05:33 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:33 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
05:33 INFO [databricks.labs.ucx.install] Fetching installations...
05:33 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:33 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:33 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:33 INFO [databricks.labs.ucx.install] Installing UCX v0.60.2+320251210053311
05:33 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assess-workflows
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=convert-wasbs-to-adls-gen2-experimental
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:33 INFO [databricks.labs.ucx.install] Creating dashboards...
05:33 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:33 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:33 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:33 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:33 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:33 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:33 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:33 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:33 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:33 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:33 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:33 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:33 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:33 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:33 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:33 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:33 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:33 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:33 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.6Z5t/README for the next steps.
05:33 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_slvnsqvvh.tables] ignoring any existing tables inventory; refresh is forced.
05:33 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_slvnsqvvh.tables] crawling new set of snapshot data for tables
05:33 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_skscdpruq] listing tables and views
05:33 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_skscdpruq.dummy_tfoa4cxh5] fetching table metadata
05:33 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_slvnsqvvh.tables] found 1 new records for tables
05:33 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-groups job: https://DATABRICKS_HOST#job/179877002609110
05:33 INFO [databricks.labs.ucx.installer.workflows] Named parameters for migrate-groups job: None
05:33 INFO [databricks.labs.ucx.installer.workflows] Started migrate-groups job: https://DATABRICKS_HOST#job/179877002609110/runs/227729582225775
05:33 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-groups workflow: https://DATABRICKS_HOST#job/179877002609110
05:33 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
05:33 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.6Z5t/config.yml) doesn't exist.
05:33 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:33 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
05:33 INFO [databricks.labs.ucx.install] Fetching installations...
05:33 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:33 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:33 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:33 INFO [databricks.labs.ucx.install] Installing UCX v0.60.2+320251210053311
05:33 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assess-workflows
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=convert-wasbs-to-adls-gen2-experimental
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:33 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:33 INFO [databricks.labs.ucx.install] Creating dashboards...
05:33 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:33 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:33 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:33 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:33 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:33 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:33 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:33 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:33 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:33 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:33 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:33 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:33 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:33 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:33 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:33 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:33 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:33 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:33 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.6Z5t/README for the next steps.
05:33 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_slvnsqvvh.tables] ignoring any existing tables inventory; refresh is forced.
05:33 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_slvnsqvvh.tables] crawling new set of snapshot data for tables
05:33 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_skscdpruq] listing tables and views
05:33 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_skscdpruq.dummy_tfoa4cxh5] fetching table metadata
05:33 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_slvnsqvvh.tables] found 1 new records for tables
05:33 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-groups job: https://DATABRICKS_HOST#job/179877002609110
05:33 INFO [databricks.labs.ucx.installer.workflows] Named parameters for migrate-groups job: None
05:33 INFO [databricks.labs.ucx.installer.workflows] Started migrate-groups job: https://DATABRICKS_HOST#job/179877002609110/runs/227729582225775
05:33 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-groups workflow: https://DATABRICKS_HOST#job/179877002609110
05:33 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
05:35 INFO [databricks.labs.ucx.install] Deleting UCX v0.60.2+320251210053311 from https://DATABRICKS_HOST
05:35 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_slvnsqvvh
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=954396356924000, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1045952841464602, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=179877002609110, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=97321754688624, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=865668858281582, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=665934846229510, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=392990171853524, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=798952752309107, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1124246032315130, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=852166039836891, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=591717170519753, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=361161297492799, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=528598738421820, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1113639923030641, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=463243821018118, as it is no longer needed
05:35 INFO [databricks.labs.ucx.install] Deleting cluster policy
05:35 INFO [databricks.labs.ucx.install] Deleting secret scope
05:35 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw4] linux -- Python 3.10.19 /home/runner/work/ucx/ucx/.venv/bin/python

_{Running from nightly #718}

[github-actions]

❌ test_running_real_migrate_groups_job: databricks.sdk.errors.sdk.OperationFailed: failed to reach TERMINATED or SKIPPED, got RunLifeCycleState.INTERNAL_ERROR: Task apply_permissions failed with message: Workload failed, see run output for details. (3m11.589s)

databricks.sdk.errors.sdk.OperationFailed: failed to reach TERMINATED or SKIPPED, got RunLifeCycleState.INTERNAL_ERROR: Task apply_permissions failed with message: Workload failed, see run output for details.
05:34 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:34 INFO [databricks.sdk] Using Databricks Metadata Service authentication
[gw7] linux -- Python 3.10.19 /home/runner/work/ucx/ucx/.venv/bin/python
05:34 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:34 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:34 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy-hqrbxxcm-ra78c275fb@example.com: https://DATABRICKS_HOST/#settings/workspace/identity-and-access/users/145850868666039
05:34 INFO [databricks.labs.pytester.fixtures.iam] Workspace group ucx-rByP-ra78c275fb: https://DATABRICKS_HOST#setting/accounts/groups/2126296680645909
05:34 INFO [databricks.labs.pytester.fixtures.iam] Account group ucx-rByP-ra78c275fb: https://accounts.CLOUD_ENVdatabricks.net/users/groups/152703829160474/members
05:34 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy-928U3UOU-ra78c275fb: https://DATABRICKS_HOST/setting/clusters/cluster-policies/view/0013414ED81BF697
05:34 INFO [databricks.labs.pytester.fixtures.baseline] Created hive_metastore.dummy_sninznhrh schema: https://DATABRICKS_HOST/#explore/data/hive_metastore/dummy_sninznhrh
05:34 INFO [databricks.labs.pytester.fixtures.baseline] Created hive_metastore.dummy_sninznhrh.dummy_tjgpdc1dw schema: https://DATABRICKS_HOST/#explore/data/hive_metastore/dummy_sninznhrh/dummy_tjgpdc1dw
05:34 INFO [databricks.labs.pytester.fixtures.baseline] Created hive_metastore.dummy_smbaflapz schema: https://DATABRICKS_HOST/#explore/data/hive_metastore/dummy_smbaflapz
05:34 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.4wEc/config.yml) doesn't exist.
05:34 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Inventory Database stored in hive_metastore
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Catalog to store UCX artifacts in
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Log level
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Number of threads
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Backup prefix
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: �[1mChoose how to map the workspace groups:�[0m
�[1m[0]�[0m �[36mMatch by Name�[0m
�[1m[1]�[0m �[36mApply a Prefix�[0m
�[1m[2]�[0m �[36mApply a Suffix�[0m
�[1m[3]�[0m �[36mMatch by External ID�[0m
�[1m[4]�[0m �[36mRegex Substitution�[0m
�[1m[5]�[0m �[36mRegex Matching�[0m
Enter a number between 0 and 5
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Enter a prefix to add to the workspace group name
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Comma-separated list of workspace group names to migrate. If not specified, we'll use all account-level groups with matching names to workspace-level groups
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Comma-separated list of databases to migrate. If not specified, we'll use all databases in hive_metastore
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Do you want to skip TACL migration when migrating tables?
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Do you want to define a TEST_SCHEMA owner group for all tables and schemas? 
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Does given workspace 841570839244994 block Internet access?
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Do you want to trigger assessment job after installation?
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Reconciliation threshold, in percentage
05:34 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
05:34 INFO [databricks.labs.ucx.install] Fetching installations...
05:34 INFO [databricks.labs.blueprint.parallel] finding 4wEc installations 100/116, rps: 87.351/sec
05:34 INFO [databricks.labs.blueprint.parallel] finding 4wEc installations 116/116, rps: 92.543/sec
05:34 INFO [databricks.labs.blueprint.parallel] Finished 'finding 4wEc installations' tasks: 0% results available (0/116). Took 0:00:01.254050
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: �[1mSelect PRO or SERVERLESS SQL warehouse to run assessment dashboards on�[0m
�[1m[0]�[0m �[36m[Create new PRO SQL warehouse]�[0m
�[1m[1]�[0m �[36mDEFAULT Test Warehouse (TEST_DEFAULT_WAREHOUSE_ID, SERVERLESS, RUNNING)�[0m
Enter a number between 0 and 1
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Instance pool id to be set in cluster policy for all workflow clusters
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: We have identified one or more cluster policies set up for an external metastore. Would you like to set UCX to connect to the external metastore?
05:34 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: �[1mIf hive_metastore contains managed table with external location paths (not dbfs root), please select which migration option to choose from�[0m
�[1m[0]�[0m �[36mConvert MANAGED HMS table to EXTERNAL HMS table and migrate as EXTERNAL UC table. Once the relevant HMS tables are deleted, the underlying data won't get deleted anymore, consider the impact of this change on your data workloads�[0m
�[1m[1]�[0m �[36mCopy data from MANAGED HMS to MANAGED UC table�[0m
�[1m[2]�[0m �[36mMigrate MANAGED HMS table as EXTERNAL UC table. This option would require you to convert MANAGED HMS tables to EXTERNAL HMS tables once UC migration is complete, otherwise deleting HMS MANAGED table would delete the migrated UC table�[0m
Enter a number between 0 and 2
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Parallelism for migrating dbfs root delta tables with deep clone
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Min workers for auto-scale job cluster for table migration
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Max workers for auto-scale job cluster for table migration
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Open config file in the browser and continue installing? https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.4wEc/config.yml
05:34 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:34 INFO [databricks.labs.blueprint.parallel] ensure clusters running 3/3, rps: 30.716/sec
05:34 INFO [databricks.labs.blueprint.parallel] Finished 'ensure clusters running' tasks: 0% results available (0/3). Took 0:00:00.097908
05:34 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:34 INFO [databricks.labs.pytester.fixtures.baseline] Created ucx_ZLcvQwr84uKpuCLP catalog: https://DATABRICKS_HOST/#explore/data/ucx_ZLcvQwr84uKpuCLP
05:34 INFO [databricks.labs.ucx.install] Installing UCX v0.60.2+320260101053449
05:34 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz database exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.CLOUD_ENV_service_principals table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.clusters table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.global_init_scripts table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.jobs table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.pipelines table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.external_locations table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.mounts table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.grants table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.groups table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.tables table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.table_size table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.table_failures table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.workspace_objects table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.permissions table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.submit_runs table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.policies table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.migration_status table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.workflow_problems table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.query_problems table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.udfs table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.logs table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.recon_results table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.directfs_in_paths table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.directfs_in_queries table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.used_tables_in_paths table exists
05:34 INFO [databricks.labs.blueprint.parallel] deploy tables 10/29, rps: 4.945/sec
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.used_tables_in_queries table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.inferred_grants table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.redash_dashboards table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.lakeview_dashboards table exists
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=convert-wasbs-to-adls-gen2-experimental
05:34 INFO [databricks.labs.blueprint.parallel] deploy tables 20/29, rps: 5.616/sec
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:34 INFO [databricks.labs.blueprint.parallel] deploy tables 29/29, rps: 7.179/sec
05:34 INFO [databricks.labs.blueprint.parallel] Finished 'deploy tables' tasks: 0% results available (0/29). Took 0:00:04.040239
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.grant_detail view matches queries/views/grant_detail.sql contents
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.objects view matches queries/views/objects.sql contents
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assess-workflows
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.table_estimates view matches queries/views/table_estimates.sql contents
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.misc_patterns view matches queries/views/misc_patterns.sql contents
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.code_patterns view matches queries/views/code_patterns.sql contents
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.reconciliation_results view matches queries/views/reconciliation_results.sql contents
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.directfs view matches queries/views/directfs.sql contents
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:35 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.used_tables view matches queries/views/used_tables.sql contents
05:35 INFO [databricks.labs.ucx.install] Creating dashboards...
05:35 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:35 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:35 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:35 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:35 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:35 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:35 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:35 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:35 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:35 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:35 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:35 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:35 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:35 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:35 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:35 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:35 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:35 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:35 INFO [databricks.labs.blueprint.parallel] installing dashboards 7/7, rps: 2.366/sec
05:35 INFO [databricks.labs.blueprint.parallel] Finished 'installing dashboards' tasks: 0% results available (0/7). Took 0:00:02.958669
05:35 INFO [databricks.labs.blueprint.parallel] installing components 2/2, rps: 0.133/sec
05:35 INFO [databricks.labs.blueprint.parallel] Finished 'installing components' tasks: 0% results available (0/2). Took 0:00:15.002105
05:35 INFO [databricks.labs.blueprint.tui] Asking prompt: Open job overview in your browser? https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.4wEc/README
05:35 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.4wEc/README for the next steps.
05:35 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_smbaflapz.tables] ignoring any existing tables inventory; refresh is forced.
05:35 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_smbaflapz.tables] crawling new set of snapshot data for tables
05:35 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_sninznhrh] listing tables and views
05:35 INFO [databricks.labs.blueprint.parallel] listing tables 1/1, rps: 2.114/sec
05:35 INFO [databricks.labs.blueprint.parallel] Finished 'listing tables' tasks: 100% results available (1/1). Took 0:00:00.473284
05:35 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_sninznhrh.dummy_tjgpdc1dw] fetching table metadata
05:35 INFO [databricks.labs.blueprint.parallel] describing tables in hive_metastore 1/1, rps: 2.038/sec
05:35 INFO [databricks.labs.blueprint.parallel] Finished 'describing tables in hive_metastore' tasks: 100% results available (1/1). Took 0:00:00.490909
05:35 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_smbaflapz.tables] found 1 new records for tables
05:35 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-groups job: https://DATABRICKS_HOST#job/866701412553282
05:35 INFO [databricks.labs.ucx.installer.workflows] Named parameters for migrate-groups job: None
05:35 INFO [databricks.labs.ucx.installer.workflows] Started migrate-groups job: https://DATABRICKS_HOST#job/866701412553282/runs/917022371200947
05:35 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-groups workflow: https://DATABRICKS_HOST#job/866701412553282
05:35 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
05:34 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:34 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:34 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy-hqrbxxcm-ra78c275fb@example.com: https://DATABRICKS_HOST/#settings/workspace/identity-and-access/users/145850868666039
05:34 INFO [databricks.labs.pytester.fixtures.iam] Workspace group ucx-rByP-ra78c275fb: https://DATABRICKS_HOST#setting/accounts/groups/2126296680645909
05:34 INFO [databricks.labs.pytester.fixtures.iam] Account group ucx-rByP-ra78c275fb: https://accounts.CLOUD_ENVdatabricks.net/users/groups/152703829160474/members
05:34 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy-928U3UOU-ra78c275fb: https://DATABRICKS_HOST/setting/clusters/cluster-policies/view/0013414ED81BF697
05:34 INFO [databricks.labs.pytester.fixtures.baseline] Created hive_metastore.dummy_sninznhrh schema: https://DATABRICKS_HOST/#explore/data/hive_metastore/dummy_sninznhrh
05:34 INFO [databricks.labs.pytester.fixtures.baseline] Created hive_metastore.dummy_sninznhrh.dummy_tjgpdc1dw schema: https://DATABRICKS_HOST/#explore/data/hive_metastore/dummy_sninznhrh/dummy_tjgpdc1dw
05:34 INFO [databricks.labs.pytester.fixtures.baseline] Created hive_metastore.dummy_smbaflapz schema: https://DATABRICKS_HOST/#explore/data/hive_metastore/dummy_smbaflapz
05:34 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.4wEc/config.yml) doesn't exist.
05:34 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Inventory Database stored in hive_metastore
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Catalog to store UCX artifacts in
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Log level
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Number of threads
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Backup prefix
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: �[1mChoose how to map the workspace groups:�[0m
�[1m[0]�[0m �[36mMatch by Name�[0m
�[1m[1]�[0m �[36mApply a Prefix�[0m
�[1m[2]�[0m �[36mApply a Suffix�[0m
�[1m[3]�[0m �[36mMatch by External ID�[0m
�[1m[4]�[0m �[36mRegex Substitution�[0m
�[1m[5]�[0m �[36mRegex Matching�[0m
Enter a number between 0 and 5
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Enter a prefix to add to the workspace group name
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Comma-separated list of workspace group names to migrate. If not specified, we'll use all account-level groups with matching names to workspace-level groups
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Comma-separated list of databases to migrate. If not specified, we'll use all databases in hive_metastore
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Do you want to skip TACL migration when migrating tables?
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Do you want to define a TEST_SCHEMA owner group for all tables and schemas? 
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Does given workspace 841570839244994 block Internet access?
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Do you want to trigger assessment job after installation?
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Reconciliation threshold, in percentage
05:34 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
05:34 INFO [databricks.labs.ucx.install] Fetching installations...
05:34 INFO [databricks.labs.blueprint.parallel] finding 4wEc installations 100/116, rps: 87.351/sec
05:34 INFO [databricks.labs.blueprint.parallel] finding 4wEc installations 116/116, rps: 92.543/sec
05:34 INFO [databricks.labs.blueprint.parallel] Finished 'finding 4wEc installations' tasks: 0% results available (0/116). Took 0:00:01.254050
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: �[1mSelect PRO or SERVERLESS SQL warehouse to run assessment dashboards on�[0m
�[1m[0]�[0m �[36m[Create new PRO SQL warehouse]�[0m
�[1m[1]�[0m �[36mDEFAULT Test Warehouse (TEST_DEFAULT_WAREHOUSE_ID, SERVERLESS, RUNNING)�[0m
Enter a number between 0 and 1
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Instance pool id to be set in cluster policy for all workflow clusters
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: We have identified one or more cluster policies set up for an external metastore. Would you like to set UCX to connect to the external metastore?
05:34 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: �[1mIf hive_metastore contains managed table with external location paths (not dbfs root), please select which migration option to choose from�[0m
�[1m[0]�[0m �[36mConvert MANAGED HMS table to EXTERNAL HMS table and migrate as EXTERNAL UC table. Once the relevant HMS tables are deleted, the underlying data won't get deleted anymore, consider the impact of this change on your data workloads�[0m
�[1m[1]�[0m �[36mCopy data from MANAGED HMS to MANAGED UC table�[0m
�[1m[2]�[0m �[36mMigrate MANAGED HMS table as EXTERNAL UC table. This option would require you to convert MANAGED HMS tables to EXTERNAL HMS tables once UC migration is complete, otherwise deleting HMS MANAGED table would delete the migrated UC table�[0m
Enter a number between 0 and 2
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Parallelism for migrating dbfs root delta tables with deep clone
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Min workers for auto-scale job cluster for table migration
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Max workers for auto-scale job cluster for table migration
05:34 INFO [databricks.labs.blueprint.tui] Asking prompt: Open config file in the browser and continue installing? https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.4wEc/config.yml
05:34 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:34 INFO [databricks.labs.blueprint.parallel] ensure clusters running 3/3, rps: 30.716/sec
05:34 INFO [databricks.labs.blueprint.parallel] Finished 'ensure clusters running' tasks: 0% results available (0/3). Took 0:00:00.097908
05:34 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:34 INFO [databricks.labs.pytester.fixtures.baseline] Created ucx_ZLcvQwr84uKpuCLP catalog: https://DATABRICKS_HOST/#explore/data/ucx_ZLcvQwr84uKpuCLP
05:34 INFO [databricks.labs.ucx.install] Installing UCX v0.60.2+320260101053449
05:34 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz database exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.CLOUD_ENV_service_principals table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.clusters table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.global_init_scripts table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.jobs table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.pipelines table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.external_locations table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.mounts table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.grants table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.groups table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.tables table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.table_size table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.table_failures table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.workspace_objects table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.permissions table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.submit_runs table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.policies table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.migration_status table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.workflow_problems table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.query_problems table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.udfs table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.logs table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.recon_results table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.directfs_in_paths table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.directfs_in_queries table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.used_tables_in_paths table exists
05:34 INFO [databricks.labs.blueprint.parallel] deploy tables 10/29, rps: 4.945/sec
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.used_tables_in_queries table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.inferred_grants table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.redash_dashboards table exists
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.lakeview_dashboards table exists
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=convert-wasbs-to-adls-gen2-experimental
05:34 INFO [databricks.labs.blueprint.parallel] deploy tables 20/29, rps: 5.616/sec
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:34 INFO [databricks.labs.blueprint.parallel] deploy tables 29/29, rps: 7.179/sec
05:34 INFO [databricks.labs.blueprint.parallel] Finished 'deploy tables' tasks: 0% results available (0/29). Took 0:00:04.040239
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.grant_detail view matches queries/views/grant_detail.sql contents
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.objects view matches queries/views/objects.sql contents
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assess-workflows
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.table_estimates view matches queries/views/table_estimates.sql contents
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.misc_patterns view matches queries/views/misc_patterns.sql contents
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.code_patterns view matches queries/views/code_patterns.sql contents
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.reconciliation_results view matches queries/views/reconciliation_results.sql contents
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:34 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.directfs view matches queries/views/directfs.sql contents
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:35 INFO [databricks.labs.lsql.deployment] Ensuring hive_metastore.dummy_smbaflapz.used_tables view matches queries/views/used_tables.sql contents
05:35 INFO [databricks.labs.ucx.install] Creating dashboards...
05:35 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:35 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:35 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:35 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:35 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:35 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:35 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:35 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:35 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:35 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:35 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:35 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:35 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:35 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:35 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:35 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:35 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:35 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:35 INFO [databricks.labs.blueprint.parallel] installing dashboards 7/7, rps: 2.366/sec
05:35 INFO [databricks.labs.blueprint.parallel] Finished 'installing dashboards' tasks: 0% results available (0/7). Took 0:00:02.958669
05:35 INFO [databricks.labs.blueprint.parallel] installing components 2/2, rps: 0.133/sec
05:35 INFO [databricks.labs.blueprint.parallel] Finished 'installing components' tasks: 0% results available (0/2). Took 0:00:15.002105
05:35 INFO [databricks.labs.blueprint.tui] Asking prompt: Open job overview in your browser? https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.4wEc/README
05:35 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.4wEc/README for the next steps.
05:35 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_smbaflapz.tables] ignoring any existing tables inventory; refresh is forced.
05:35 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_smbaflapz.tables] crawling new set of snapshot data for tables
05:35 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_sninznhrh] listing tables and views
05:35 INFO [databricks.labs.blueprint.parallel] listing tables 1/1, rps: 2.114/sec
05:35 INFO [databricks.labs.blueprint.parallel] Finished 'listing tables' tasks: 100% results available (1/1). Took 0:00:00.473284
05:35 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_sninznhrh.dummy_tjgpdc1dw] fetching table metadata
05:35 INFO [databricks.labs.blueprint.parallel] describing tables in hive_metastore 1/1, rps: 2.038/sec
05:35 INFO [databricks.labs.blueprint.parallel] Finished 'describing tables in hive_metastore' tasks: 100% results available (1/1). Took 0:00:00.490909
05:35 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_smbaflapz.tables] found 1 new records for tables
05:35 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-groups job: https://DATABRICKS_HOST#job/866701412553282
05:35 INFO [databricks.labs.ucx.installer.workflows] Named parameters for migrate-groups job: None
05:35 INFO [databricks.labs.ucx.installer.workflows] Started migrate-groups job: https://DATABRICKS_HOST#job/866701412553282/runs/917022371200947
05:35 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-groups workflow: https://DATABRICKS_HOST#job/866701412553282
05:35 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
05:37 INFO [databricks.labs.blueprint.tui] Asking prompt: Do you want to uninstall ucx from the workspace too, this would remove ucx project folder, dashboards, queries and jobs
05:37 INFO [databricks.labs.ucx.install] Deleting UCX v0.60.2+320260101053449 from https://DATABRICKS_HOST
05:37 INFO [databricks.labs.blueprint.tui] Asking prompt: Do you want to delete the inventory database dummy_smbaflapz too?
05:37 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_smbaflapz
05:37 INFO [databricks.labs.lsql.deployment] Deleting hive_metastore.dummy_smbaflapz database
05:37 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=85874286248259, as it is no longer needed
05:37 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=911406116432469, as it is no longer needed
05:37 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=450289224638385, as it is no longer needed
05:37 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=845042826116635, as it is no longer needed
05:37 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=421926520444349, as it is no longer needed
05:37 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=411597176407977, as it is no longer needed
05:37 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1010016076637614, as it is no longer needed
05:37 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=701496574464304, as it is no longer needed
05:37 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=811796244368242, as it is no longer needed
05:37 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=983473269733114, as it is no longer needed
05:37 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1063062978690953, as it is no longer needed
05:37 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=204274593522243, as it is no longer needed
05:37 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=866701412553282, as it is no longer needed
05:37 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=286318120743940, as it is no longer needed
05:37 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=464643087955030, as it is no longer needed
05:37 INFO [databricks.labs.ucx.install] Deleting cluster policy
05:37 INFO [databricks.labs.ucx.install] Deleting secret scope
05:37 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw7] linux -- Python 3.10.19 /home/runner/work/ucx/ucx/.venv/bin/python

_{Running from nightly #740}