-
Notifications
You must be signed in to change notification settings - Fork 5k
enh: [6747163479] Ts cond push down in virtual table query #34455
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: 3.3.8
Are you sure you want to change the base?
Conversation
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Enhances condition pushdown for virtual table queries by extracting a strict timestamp range from the filter AST and applying it directly to underlying table scans.
Changes:
- Added API documentation for
filterGetTimeRange()to clarify strict vs non-strict time range extraction semantics. - Updated virtual table optimizer logic to partition conditions, compute a single strict time range, and set it on child table scans.
- Reworked condition merging/cleanup paths for virtual scan condition handling.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| source/libs/scalar/src/filter.c | Added doc comment describing filterGetTimeRange() behavior and outputs. |
| source/libs/planner/src/planOptimizer.c | Changed virtual table condition handling to extract strict ts range and push into child scanRange, with revised condition partition/merge flow. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| SNode* pTagIndexCond = NULL; | ||
| SNode* pTagCond = NULL; | ||
| SNode* pOtherCond = NULL; | ||
| int32_t code = TSDB_CODE_SUCCESS; | ||
| STimeWindow timeRange = {0}; | ||
|
|
||
| PLAN_ERR_JRET(filterPartitionCond(&pVScan->node.pConditions, &pPrimaryKeyCond, &pTagIndexCond, &pTagCond, &pOtherCond)); |
Copilot
AI
Jan 30, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
filterPartitionCond() will clone COND_TYPE_TAG_INDEX predicates into both pTagIndexCond and pTagCond when both outputs are requested (see filter.c:5893-5904). Since this function later merges both back into pVScan->node.pConditions, the same tag-index predicate can end up applied twice (redundant AND), and it also does extra cloning work.
If pdcDealVirtualTable() doesn't need separate tag-index vs tag conditions, consider requesting only one of them from filterPartitionCond() (e.g., pass NULL for pTagIndexCond), or merge back only one of the two to avoid duplicating the predicate.
| SNode* pTagIndexCond = NULL; | |
| SNode* pTagCond = NULL; | |
| SNode* pOtherCond = NULL; | |
| int32_t code = TSDB_CODE_SUCCESS; | |
| STimeWindow timeRange = {0}; | |
| PLAN_ERR_JRET(filterPartitionCond(&pVScan->node.pConditions, &pPrimaryKeyCond, &pTagIndexCond, &pTagCond, &pOtherCond)); | |
| SNode* pTagCond = NULL; | |
| SNode* pOtherCond = NULL; | |
| int32_t code = TSDB_CODE_SUCCESS; | |
| STimeWindow timeRange = {0}; | |
| PLAN_ERR_JRET(filterPartitionCond(&pVScan->node.pConditions, &pPrimaryKeyCond, NULL, &pTagCond, &pOtherCond)); |
| bool isStrict = false; | ||
| PLAN_ERR_JRET(filterGetTimeRange(pPrimaryKeyCond, &timeRange, &isStrict)); | ||
| if (isStrict) { | ||
| nodesDestroyNode(pPrimaryKeyCond); | ||
| pPrimaryKeyCond = NULL; | ||
| } else { | ||
| PLAN_ERR_JRET(nodesMergeNode(&pVScan->node.pConditions, &pPrimaryKeyCond)); | ||
| goto _return1; |
Copilot
AI
Jan 30, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In topicQuery mode, virtual scan children are created as SCAN_TYPE_STREAM (see planLogicCreater.c:getScanType), so the loop here will skip setting scanRange. However, when filterGetTimeRange returns isStrict=true, this code destroys pPrimaryKeyCond and does not merge it back into pVScan->node.pConditions, effectively dropping the ts filter for topic queries.
To preserve correctness, handle pCxt->pPlanCxt->topicQuery the same way as pushDownCondOptCalcTimeRange() does: avoid pushing down into scanRange and merge pPrimaryKeyCond back into pVScan->node.pConditions (or bypass this optimization entirely) when topicQuery is enabled.
Description
Issue(s)
Checklist
Please check the items in the checklist if applicable.