Skip to content

Conversation

@andrewnester
Copy link
Contributor

Changes

Fix for pip flags with equal sign being incorrectly treated as local file names

Why

Pip flags containing an equal sign and used in the environment dependencies section were incorrectly treated as local file names, leading to the bundle deploy error.

Tests

Added test case for this

@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Oct 15, 2025

Run: 18526081168

Env 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip
💚​ aws linux 1 321 545
💚​ aws windows 1 322 544
💚​ aws-ucws linux 1 437 441
💚​ aws-ucws windows 1 438 440
💚​ azure linux 1 321 544
💚​ azure windows 1 322 543
💚​ azure-ucws linux 1 437 440
💚​ azure-ucws windows 1 438 439
🔄​ gcp linux 3 1 317 546
🔄​ gcp windows 3 1 318 545
7 failing tests:
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
TestAccept/bundle/deployment/bind/cluster ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
TestAccept/bundle/deployment/bind/dashboard ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/resources/pipelines/update ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/resources/pipelines/update/DATABRICKS_BUNDLE_ENGINE=direct-exp ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/templates/default-python/integration_classic ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.9 ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p

// Alternatively it could be a flag with no argument and no space after it
// For example: -r myfile.txt or --index-url http://myindexurl.com or -i
re := regexp.MustCompile(`(^|\s+)--?[a-zA-Z0-9-]+(\s+|$)`)
re := regexp.MustCompile(`(^|\s+)--?[a-zA-Z0-9-]+(([\s|=]+)|$)`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I think you only need ([\s=]|$) -- either a space, an =, or the end of the string.

More than one space or = doesn't change the result.

@andrewnester andrewnester added this pull request to the merge queue Oct 15, 2025
Merged via the queue into main with commit 20dfac0 Oct 15, 2025
13 checks passed
@andrewnester andrewnester deleted the fix/env-spec-double-dash branch October 15, 2025 11:30
deco-sdk-tagging bot added a commit that referenced this pull request Oct 16, 2025
## Release v0.273.0

### Notable Changes

* (via Terraform v1.92.0) DABs will no longer try to update pipeline permissions upon pipeline deletion. This fixes PERMISSION\_ERROR upon 'bundle destroy'
  for pipelines that have run\_as setting enabled (described in https://community.databricks.com/t95/data-engineering/dab-dlt-destroy-fails-due-to-ownership-permissions-mismatch/td-p/132101)
  The downside is that if 'permissions:' block is removed from the resource, DABs will not try anymore to restore permissions to just the owner of the pipeline.

### CLI

* Add the `--configure-serverless` flag to `databricks auth login` to configure Databricks Connect to use serverless.

### Dependency updates
* Upgrade Go SDK to 0.82.0 ([#3769](#3769))
* Upgrade TF provider to 1.92.0 ([#3772](#3772))

### Bundles
* Updated the internal lakeflow-pipelines template to use an "src" layout ([#3671](#3671)).
* Fix for pip flags with equal sign being incorrectly treated as local file names ([#3766](#3766))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants