Add Trino support for SQLAlchemyMetastoreLoader #1627
Description
Summary
Enable using Trino as a metastore source via SQLAlchemy's SQLAlchemyMetastoreLoader. This would allow Querybook to load table/schema metadata from Trino catalogs (Hive, Iceberg, etc.) using connection strings like trino://admin:password@trino.example.com:443/iceberg?SSL=true.
Proposed Technical Design
Add sqlalchemy-trino package:
- Provides SQLAlchemy dialect for Trino connections
- Enables
SQLAlchemyMetastoreLoaderto connect to Trino - Works with existing metastore loader infrastructure
Required dependency upgrade:
sqlalchemy-trinorequirestrino>=0.320.0- Current version:
trino==0.305.0 - Proposed version:
trino==0.336.0
Side effect - Query Engine cursor API fix:
The Trino client's cursor API changed in 0.336.0. The _query.fetch() method was replaced with _iterator. This requires a small fix in the query engine code.
Proposed Changes (Diff)
diff --git a/querybook/server/lib/query_executor/clients/trino.py b/querybook/server/lib/query_executor/clients/trino.py
index 35e9839d..658a91d9 100644
--- a/querybook/server/lib/query_executor/clients/trino.py
+++ b/querybook/server/lib/query_executor/clients/trino.py
@@ -22,7 +22,7 @@ class TrinoCursor(PrestoCursorMixin[trino.dbapi.Cursor, List[Any]], CursorBaseCl
def poll(self):
try:
- self.rows.extend(self._cursor._query.fetch())
+ self.rows.extend(self._cursor._iterator)
self._cursor._iterator = iter(self.rows)
poll_result = self._cursor.stats
completed = self._cursor._query._finished
diff --git a/requirements/engine/trino.txt b/requirements/engine/trino.txt
index 86cb0ed2..c3b91e72 100644
--- a/requirements/engine/trino.txt
+++ b/requirements/engine/trino.txt
@@ -1 +1,2 @@
-trino==0.305.0
+trino==0.336.0
+sqlalchemy-trino==0.5.0