Skip to content

fix: non-breaking detection for redundant cast projections#5851

Merged
mday-io merged 6 commits into
SQLMesh:mainfrom
1K5-TECH:new-main
Jul 1, 2026
Merged

fix: non-breaking detection for redundant cast projections#5851
mday-io merged 6 commits into
SQLMesh:mainfrom
1K5-TECH:new-main

Conversation

@albertosuman-1k5

Copy link
Copy Markdown
Contributor

Description

Summary

  • Fixes a false-positive Breaking categorization when adding a mid-list projection with a redundant same-type cast, e.g. new_col::STRING above an existing ::STRING projection.
  • Adds a conservative projection-additivity fallback for SqlModel.is_breaking_change() when SQLGlot’s AST diff emits spurious Move / Update edits.
  • Preserves conservative behavior for real changes such as projection rewrites, removals, reorders, filter/group changes, and cardinality-changing UDTFs.

Context

SQLGlot’s tree diff can cross-match structurally similar cast nodes, especially identical DataType leaves such as STRING. This can make a purely additive projection change look like a non-Insert diff, causing SQLMesh to return None from is_breaking_change(). Under AutoCategorizationMode.FULL, that becomes BREAKING.

The new fallback verifies directly that:

  • both rendered queries are top-level SELECTs,
  • the query skeleton is unchanged except for the projection list,
  • previous projections are preserved in order,
  • and added projections do not include cardinality-changing UDTFs.

Test plan

  • python -m pytest tests/core/test_snapshot.py -k "categorize_change_sql or categorize_change_seed" -q
  • python -m pytest cast_breaking_change_demo/test_cast_breaking_change.py -q
  • python -m pytest tests/core/test_snapshot.py tests/core/test_model.py::test_is_breaking_change tests/core/test_plan.py -q
  • pre-commit run --files sqlmesh/core/model/definition.py tests/core/test_snapshot.py cast_breaking_change_demo/test_cast_breaking_change.py

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO
@albertosuman-1k5

Copy link
Copy Markdown
Contributor Author

Can we trigger again the workflows? Pretty sure the first run failed because of the issue described in #5852

@mday-io

mday-io commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Can we trigger again the workflows? Pretty sure the first run failed because of the issue described in #5852

rerunning...

@mday-io mday-io self-requested a review June 23, 2026 13:50
Comment thread sqlmesh/core/model/definition.py
Comment thread sqlmesh/core/model/definition.py
@mday-io

mday-io commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Could you add tests for queries that reference SELECT-list positions from other clauses, such as ordinal ORDER BY / GROUP BY, to ensure those stay undetermined instead of being auto-categorized as non-breaking?

It would also be good to add coverage for aliased UDTF projections, since those may not appear as a top-level UDTF expression in the projection list but should still remain conservative.

@albertosuman-1k5

Copy link
Copy Markdown
Contributor Author

@mday-io added logic to avoid report as non breaking when order/group by uses positional arguments whose reference is past the introduced expression and to correctly mark aliased UDTF

@StuffbyYuki StuffbyYuki requested a review from mday-io July 1, 2026 14:55
@mday-io mday-io merged commit 13787d7 into SQLMesh:main Jul 1, 2026
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants