Skip to content

Fix!: serialize blueprint variables separately to leverage AST#4061

Merged
georgesittas merged 2 commits intomainfrom
jo/fix_blueprint_variable_representation
Apr 3, 2025
Merged

Fix!: serialize blueprint variables separately to leverage AST#4061
georgesittas merged 2 commits intomainfrom
jo/fix_blueprint_variable_representation

Conversation

@georgesittas
Copy link
Contributor

Fixes #4017

Given a model like this:

MODEL (
  name m.@{foo},
  blueprints (
    (foo := "s p a c e s"),
    (foo := 'c'),
  ),
);

SELECT
  @foo AS col

The blueprint variables that contain quotes are incorrectly rendered today:

from sqlmesh import Context
ctx = Context()
for fqn, model in ctx.models.items():
    print(f"Fully-qualified name: {fqn}, name: {model.name}, query: {model.render_query().sql()}")

# Fully-qualified name: "db"."m"."""s p a c e s""", name: m."""s p a c e s""", query: SELECT '"s p a c e s"' AS "col"
# Fully-qualified name: "db"."m"."'c'", name: m."'c'", query: SELECT '''c''' AS "col"

This PR addresses this bug with the help of this commit, by changing the representation of the blueprint values: their values are now the parsed ASTs, and not the corresponding SQL strings. This behavior is more intuitive, because for example quoted identifiers will be correctly interpolated in contexts such as model names, without dropping the quotes.

Since SQLGlot expressions aren't serialized today, making this fix wasn't as simple as pushing the blueprint variables to the SQLMESH_VARS dict, since that only contains Python literals that can be eval'd on the fly without additional context, when hydrating the Python environment.

Additionally, we need to serialize blueprint values because if, e.g., a model query references them, then the raw SQL with the variable reference is serialized and we need to have the variable's payload at deserialization time to ensure the query can be properly rendered.

This led to a new dictionary in the python env. that will contain only the blueprint variables, so that we can unambiguously serialize them as strings that are produced by sql(dialect=...) and then deserialize them in the macro evaluator by parsing with the same dialect.

@georgesittas georgesittas requested review from a team, izeigerman and tobymao March 31, 2025 18:02
@georgesittas georgesittas force-pushed the jo/fix_blueprint_variable_representation branch from 417eb37 to c516498 Compare March 31, 2025 18:12
@georgesittas georgesittas force-pushed the jo/fix_blueprint_variable_representation branch 3 times, most recently from 3813e17 to 257c1c8 Compare April 2, 2025 20:52
@georgesittas georgesittas force-pushed the jo/fix_blueprint_variable_representation branch from 257c1c8 to dcaa576 Compare April 2, 2025 22:33
@georgesittas georgesittas force-pushed the jo/fix_blueprint_variable_representation branch from 02bf7fe to ba6b042 Compare April 3, 2025 12:38
@georgesittas georgesittas force-pushed the jo/fix_blueprint_variable_representation branch from ba6b042 to a91f4ae Compare April 3, 2025 13:10
@georgesittas georgesittas requested a review from a team April 3, 2025 13:12
Copy link
Collaborator

@izeigerman izeigerman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice 👍

@georgesittas georgesittas merged commit 15866ba into main Apr 3, 2025
22 checks passed
@georgesittas georgesittas deleted the jo/fix_blueprint_variable_representation branch April 3, 2025 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SQL Model Blueprinting bug?

3 participants