Skip to content

fix: Databricks column type identification#5848

Open
davem-bis wants to merge 1 commit into
SQLMesh:mainfrom
davem-bis:feature/DRM/pass-rendered-sql-to-cursor-for-databricks-column-type-derivation
Open

fix: Databricks column type identification#5848
davem-bis wants to merge 1 commit into
SQLMesh:mainfrom
davem-bis:feature/DRM/pass-rendered-sql-to-cursor-for-databricks-column-type-derivation

Conversation

@davem-bis

Copy link
Copy Markdown

Description

I have updated the call to the execution cursor in DatabricksEngineAdapter.column to have rendered SQL passed to it as opposed to the SQLGlot AST it was previously being supplied.

This should resolve #5841.

Test Plan

I have run this change against an existing environment in Databricks using:

  • databricks-sql-connector==4.0.5
  • thrift==0.20.0

This was run against a classic SQL Warehouse against Unity Catalog.

I have updated the unit test for this method to expect rendered SQL to be passed.

Checklist

  • I have run make style and fixed any issues

    Note: This is currently failing on main, however I don't believe this has introduced any new issues. ruff and ruff-format pass, whilst mypy fails on main.

  • I have added tests for my changes (if applicable)

  • All existing tests pass (make fast-test)

    Note: There are failing tests on main. Tests on DatabricksEngineAdapter.column pass.

  • My commits are signed off (git commit -s) per the DCO

… execution cursor as opposed to AST.

Signed-off-by: davem-bis <68955845+davem-bis@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Databricks runs fail to execute on 0.235.4 with thrift error

1 participant