pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java (22 lines): - line 212: // TODO: make this configurable - line 303: * TODO:For the backwards incompatible change, this is a - line 761: // TODO: move tenant related APIs here - line 895: // TODO: move table related APIs here - line 1268: // TODO: update idealStates & instanceZkMetadata - line 1290: // TODO: update idealStates & instanceZkMetadata - line 1294: // TODO: update idealStates & instanceZkMetadata - line 1385: // TODO: Consider throwing BadRequestException - line 1463: * TODO: refactor code to use this method over {@link #getAllInstancesForServerTenant(String)} if applicable to reuse - line 1480: * TODO: refactor code to use this method over {@link #getAllInstancesForBrokerTenant(String)} if applicable to reuse - line 1587: // TODO: Add the reason of the incompatibility - line 2737: // TODO: support to force download immutable segments from RealTime table. - line 2773: // TODO: support to force download immutable segments from RealTime table. - line 2805: // TODO: support to force download immutable segments from RealTime table. - line 2932: // TODO: these are static in Pinot's resource reset (for each resource type). - line 3022: // TODO: Would be nice if we can get the name of the instances to which messages were sent - line 3037: // TODO: Would be nice if we can get the name of the instances to which messages were sent - line 3061: // TODO: Would be nice if we can get the name of the instances to which messages were sent - line 3121: // TODO: Would be nice if we can get the name of the instances to which messages were sent - line 3619: // TODO: Handle the case when realtime segments are in OFFLINE state because there're some problem with - line 4009: // TODO: make the number of allowed snapshots configurable to allow users to keep at most N snapshots - line 4270: // TODO: need to expand the logic to revert multiple entries in one go when we support > 2 data snapshots pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java (13 lines): - line 154: * TODO: migrate code in this class to other places for better readability - line 173: // TODO: make this configurable with default set to 10 - line 369: // TODO: Consider using TableCache to read the table config - line 653: // TODO: also create the new partition groups here, instead of waiting till the {@link - line 765: // TODO: This code does not support size-based segment thresholds for tables with pauseless enabled. The - line 1019: // TODO: Find a better way to determine partition count and if the committing partition group is fully consumed. - line 1100: // TODO: https://github.com/apache/pinot/issues/12055 - line 1222: // FIXME: Right now, we assume topics are sharing same offset criteria - line 1280: // TODO : remove this handling after next release. - line 1452: * TODO: split this method into multiple smaller methods - line 1476: // TODO: Directly return map from StreamMetadataProvider - line 1802: * TODO: Add an on-demand way to upload LLC segment to deep store for a specific table. - line 1866: // TODO: add version check when persist segment ZK metadata pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java (12 lines): - line 341: // TODO each time this method is called, we print reason for stop. Good to print only once. - line 513: // TODO Issue 5359 Need to find a way to bump metrics without getting actual offset value. - line 515: // TODO: only LongMsgOffset supplies long offset value. - line 559: // TODO Although the metric is called real-time, updating it at this point is not really real-time. The choice of - line 637: // TODO: based on a config, decide whether the record should be silently dropped or stop further consumption on - line 761: // TODO: - line 1133: // TODO Use an auto-closeable object to delete temp resources. - line 1620: // TODO Validate configs - line 1789: // TODO: Revisit the non-pauseless handling - line 1896: // TODO: currentPartitionGroupConsumptionStatus should be fetched from idealState + segmentZkMetadata, - line 1991: * TODO: Revisit if we should preserve the offset info. - line 2000: // TODO Call this method when we are loading the segment, which we do from table datamanager afaik pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandler.java (11 lines): - line 258: // TODO: Use different global query id for OFFLINE and REALTIME table after releasing 0.12.0. See QueryIdUtils for - line 263: // TODO: how should we add the cid here? Maybe as a query param? - line 387: // TODO: Modify AccessControl interface to directly take PinotQuery - line 544: // TODO: Modify RoutingManager interface to directly take PinotQuery - line 679: // TODO: Replace ServerStats with ServerRoutingStatsEntry. - line 681: // TODO: Handle broker specific operations for explain plan queries such as: - line 688: // TODO: Assess if the Explain Plan Query should also be routed to REALTIME servers for HYBRID tables - line 699: // TODO: Even tracking the query as late as here, a potential race condition between calling cancel API and - line 809: // TODO: refine the exceptions here to early termination the queries won't requires to send to servers. - line 1822: * TODO: come up with other criteria for forcing a log and come up with better numbers - line 1974: * TODO: Directly take PinotQuery pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/BitSlicedRangeIndexReader.java (9 lines): - line 50: // TODO: Read max from header to prevent cases where max value is not available in the column metadata - line 62: // TODO: Handle this before reading the range index - line 71: // TODO: Handle this before reading the range index - line 80: // TODO: Handle this before reading the range index - line 89: // TODO: Handle this before reading the range index - line 124: // TODO: Handle this before reading the range index - line 133: // TODO: Handle this before reading the range index - line 142: // TODO: Handle this before reading the range index - line 151: // TODO: Handle this before reading the range index pinot-query-planner/src/main/java/org/apache/pinot/query/routing/WorkerManager.java (8 lines): - line 119: // TODO: Revisit this logic and see if we can generalize this - line 128: // TODO: Add a query test for LOOKUP join - line 286: // TODO: - line 323: // TODO: Short circuit it when no table needs to be scanned - line 430: // TODO: support optional segments for multi-stage engine. - line 507: // TODO: Support unavailable segments and optional segments for replicated leaf stage - line 513: * TODO: It doesn't handle unavailable segments. - line 576: // TODO: Currently we don't support the case when a partition doesn't contain any segment. The reason is that the pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/mutable/MutableSegmentImpl.java (7 lines): - line 176: // FIXME: There is a corner case for this approach which could cause inconsistency. When there is segment load during - line 314: // TODO (mutable-index-spi): The comment above was here, but no check was done. - line 371: // TODO this can be removed after forward index contents no longer depends on text index configs - line 598: // TODO fix this metadata mismatch scenario - line 1061: // TODO: Refactor virtual column provider to directly generate data source - line 1245: // FIXME: this for loop breaks for multi value dimensions. https://github.com/apache/pinot/issues/3867 - line 1265: * TODO: Eliminate the requirement on dictionary encoding for dimension and metric columns. pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotWindowExchangeNodeInsertRule.java (6 lines): - line 59: * TODO: - line 111: // TODO: Revisit whether we should use hash distribution - line 116: // TODO: ORDER BY only type queries need to be sorted on both sender and receiver side for better performance. - line 119: // TODO: Revisit whether we should use hash distribution - line 137: // TODO: ORDER BY only type queries need to be sorted only on the receiver side unless a hint is set indicating - line 295: * TODO: Explore an option to handle empty LogicalProject by actually projecting empty rows for each entry. This way pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/IngestionDelayTracker.java (6 lines): - line 84: * TODO: handle bug situations like the one where a partition is not allocated to a given server due to a bug. - line 122: // TODO: Consider removing this mechanism after releasing 1.2.0, and use {@link #stopTrackingPartitionIngestionDelay} - line 129: // TODO: Make thread pool a server/cluster level config - line 424: // TODO: Support other types of offsets - line 441: // TODO: Support other types of offsets - line 458: // TODO: Support other types of offsets pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java (6 lines): - line 92: // TODO: Check resource leaks - line 241: // TODO: Eventually pull the doc Id sorting logic out of Record Reader so that all row oriented logic can be - line 250: // TODO: _indexCreationInfoMap holds the reference to all unique values on heap (ColumnIndexCreationInfo -> - line 312: // TODO: Eventually pull the doc Id sorting logic out of Record Reader so that all row oriented logic can be - line 318: // TODO: _indexCreationInfoMap holds the reference to all unique values on heap (ColumnIndexCreationInfo -> - line 340: // TODO: Using column oriented, we can't catch incomplete records. Does that matter? pinot-query-planner/src/main/java/org/apache/pinot/query/planner/logical/EquivalentStagesFinder.java (6 lines): - line 125: // TODO: Distribution type not needed for equivalence in the first substituted send nodes. Their different - line 132: // TODO: Keys could probably be removed from the equivalence check, but would require to verify both - line 135: // TODO: Pre-partitioned and collations can probably be removed from the equivalence check, but would - line 148: // TODO: DataSchema equality checks enforce order between columns. This is probably not needed for equivalence - line 227: // TODO: Keys should probably be removed from the equivalence check, but would require to verify both - line 231: // TODO: Sort, sort on sender and collations can probably be removed from the equivalence check, but would pinot-query-planner/src/main/java/org/apache/pinot/query/planner/physical/v2/opt/rules/WorkerExchangeAssignmentRule.java (5 lines): - line 296: // TODO: Think if we need to treat random distribution as a constraint. - line 301: // TODO: Can optimize this to reduce fan out? - line 311: // TODO: Add broadcast to broadcast exchange. - line 359: // TODO: Add support for sub-partitioning or coalescing exchange here. - line 413: // TODO: Setup explicit metadata to force assume distribution constraint met. pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/StartServiceManagerCommand.java (5 lines): - line 64: // TODO: support forbids = {"-bootstrapConfigPaths", "-bootstrapServices"}) - line 67: // TODO: support forbids = {"-bootstrapConfigPaths", "-bootstrapServices"}) - line 71: // TODO: support forbids = {"-bootstrapConfigPaths", "-bootstrapServices"}) - line 76: // TODO: support forbids = {"-zkAddress", "-clusterName", "-port", "-bootstrapServices"}) - line 80: // TODO: support forbids = {"-zkAddress", "-clusterName", "-port", "-bootstrapConfigPaths"}) pinot-query-runtime/src/main/java/org/apache/pinot/query/service/dispatch/QueryDispatcher.java (5 lines): - line 173: // TODO: Consider always cancel when it returns (early terminate) - line 249: // TODO: Consider always cancel when it returns (early terminate) - line 360: // TODO: Cancel all dispatched requests if one of the dispatch errors out or deadline is breached. - line 644: // TODO: Improve the error handling, e.g. return partial response - line 666: // TODO: Improve the way the errors are compared pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotRuleUtils.java (5 lines): - line 52: // TODO: expansion is deprecated in Calcite; we need to move to the default value of false here which will be the - line 96: // TODO 1: optimize this part out as it is not efficient to scan the entire subtree for exchanges; - line 98: // TODO 2: this part is similar to how ServerPlanRequestVisitor determines leaf-stage boundary; - line 100: // TODO 3: for JoinNode, currently this only works towards left-side; - line 102: // TODO 4: for JoinNode, currently this only works for SEMI-JOIN, INNER-JOIN can bring in rows from both sides; pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/forward/ForwardIndexType.java (5 lines): - line 247: // TODO: Add support for updating RealtimeSegmentStatsHistory with average column value size for no dictionary - line 249: // TODO: Use the stats to get estimated average length - line 270: // TODO: Add support for variable width (bytes, string, big decimal) MV RAW column types - line 275: // TODO: Start with a smaller capacity on FixedByteMVForwardIndexReaderWriter and let it expand - line 288: // TODO: Start with a smaller capacity on FixedByteMVForwardIndexReaderWriter and let it expand pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/CastTransformFunction.java (5 lines): - line 81: // TODO: Support MV BIG_DECIMAL - line 141: // TODO: Add it to the interface - line 188: // TODO: Add it to the interface - line 303: // TODO: Add it to the interface - line 346: // TODO: Add it to the interface pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java (5 lines): - line 85: // TODO: Add the ability to configure the parser's maximum identifier length via configuration if needed in the future - line 235: // TODO: Explore if DISTINCT should be supported with GROUP BY - line 239: // TODO: Consider changing it to SELECTION query for LIMIT 0 - line 419: // TODO: convert to MySQL conformance once we retired most of the un-tested BABEL tokens - line 458: // TODO: explore support for GROUP BY with DISTINCT pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/GrpcBrokerRequestHandler.java (5 lines): - line 64: // TODO: Support TLS - line 94: // TODO: Routing bases on Map cannot be supported for logical tables. - line 99: // TODO: Add servers queried/responded stats - line 127: // TODO: support optional segments for GrpcQueryServer. - line 129: // TODO: enable throttling on per host bases. pinot-server/src/main/java/org/apache/pinot/server/predownload/PredownloadScheduler.java (5 lines): - line 61: // TODO: make download timeout configurable - line 95: // TODO: tune the value - line 264: // TODO: add future.orTimeout() to handle per segment downloading timeout - line 296: // TODO: increase rate limit here - line 386: // TODO: add download from peer logic pinot-core/src/main/java/org/apache/pinot/core/query/reduce/GapfillFilterHandler.java (5 lines): - line 47: // TODO: This won't work for certain aggregations because the column name in schema is not expression.toString(). - line 48: // TODO: Please refer to {@link PostAggregationHandler} on how to handle the index for aggregation queries. - line 51: // TODO: support proper null handling in GapfillFilterHandler. - line 76: // TODO: This does not handle transform properly (e.g. colA - colB where the gapfill selects colA and colB). - line 77: // TODO: This is handled within the PostAggregationValueExtractor, and we may also extract that out to be shared. pinot-controller/src/main/java/org/apache/pinot/controller/recommender/io/InputManager.java (5 lines): - line 174: // TODO: we should catch and log errors here so we don't fail queries on optimization. - line 607: return 0; //TODO: implement this after the complex is supported - line 611: // TODO: currently raw encoding is only applicable for SV columns, change this after it's supported for MV - line 627: return 0; //TODO: implement this after the complex is supported - line 641: //TODO: implement this after the complex is supported pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java (5 lines): - line 408: * TODO: refactor code to use this method over {@link #getInstancesWithTag(HelixManager, String)} if applicable to - line 474: * TODO: refactor code to use this method over {@link #getEnabledInstancesWithTag(HelixManager, String)} if applicable - line 497: * TODO: refactor code to use this method if applicable to reuse instance configs in order to reduce ZK accesses - line 506: * TODO: refactor code to use this method if applicable to reuse instance configs in order to reduce ZK accesses - line 525: * TODO: refactor code to use this method if applicable to reuse instance configs in order to reduce ZK accesses pinot-broker/src/main/java/org/apache/pinot/broker/routing/BrokerRoutingManager.java (5 lines): - line 98: * TODO: Expose RoutingEntry class to get a consistent view in the broker request handler and save the redundant map - line 505: // TODO: Support multiple partition columns - line 506: // TODO: Make partition pruner on top of the partition metadata manager to avoid keeping 2 copies of the metadata - line 661: // TODO: allow servers only with optional segments - line 666: // TODO: Report missing server metrics when we allow servers only with optional segments. pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentUploadDownloadRestletResource.java (5 lines): - line 253: // TODO: Consider validating the segment name and table name from the header against the actual segment - line 354: // TODO: remove this when we completely deprecate the table name from segment metadata - line 384: // TODO: Include the untarred segment size when using the METADATA push rest API. Currently we can only use the - line 444: // TODO: Consider validating the segment name and table name from the header against the actual segment - line 616: // TODO: Include the un-tarred segment size when using the METADATA push rest API. Currently we can only use the pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MultistageGroupByExecutor.java (4 lines): - line 200: // TODO: Change it to use top-K algorithm - line 215: // TODO: allocate new array row only if row enters set - line 388: // TODO: Fix it - line 416: // TODO: Fix it pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTenantRestletResource.java (4 lines): - line 164: // TODO: should be /tenant/{tenantName} - line 560: // TODO: FIXME: This API is horribly bad design doing too many unrelated operations and giving - line 592: // TODO: do not support drop. It's same as DELETE - line 694: // TODO decide on if the tenant rebalance should be database aware or not pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/TimeSeriesRequestHandler.java (4 lines): - line 139: // TODO: Implement this. - line 147: // TODO: Implement this. - line 155: // TODO: Implement this. - line 207: // TODO: Pass full raw query param string to the request pinot-core/src/main/java/org/apache/pinot/core/query/optimizer/filter/NumericalFilterOptimizer.java (4 lines): - line 71: * TODO: Add support for IN, and NOT IN operators. - line 178: // TODO: Add support for FLOAT_VALUE - line 289: // TODO: Add support for FLOAT_VALUE - line 369: // TODO: Consider unifying logic with rewriteRangeExpression pinot-query-planner/src/main/java/org/apache/pinot/query/planner/logical/RelToPlanNodeConverter.java (4 lines): - line 211: * TODO: Add support for exclude clauses ({@link org.apache.calcite.rex.RexWindowExclusion}) - line 257: // TODO: The constants are already extracted in the PinotWindowExchangeNodeInsertRule, we can remove them from - line 356: // TODO: Consider adding DYNAMIC_BROADCAST as a separate join strategy - line 442: * TODO: Revisit this method: pinot-query-planner/src/main/java/org/apache/pinot/query/QueryEnvironment.java (4 lines): - line 113: //TODO: We should consider splitting this class in two: One that is used for parsing and one that is used for - line 420: // TODO: add support for cost factory - line 488: // TODO: We can consider using HepMatchOrder.TOP_DOWN if we find cases where it would help. - line 691: // TODO: current code only assume one SubPlan per query, but we should support multiple SubPlans per query. pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotAggregateExchangeNodeInsertRule.java (4 lines): - line 109: * TODO: - line 248: // TODO: Currently it only handles one WITHIN GROUP collation across all AggregateCalls. - line 289: * TODO: Add optional INTERMEDIATE stage to reduce hotspot. - line 427: // TODO: Revisit the following logic: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java (4 lines): - line 87: // TODO: check resource leaks - line 89: // TODO Refactor class name to match interface name - line 177: // TODO: Dictionary creator holds all unique values on heap. Consider keeping dictionary instead of creator - line 215: // TODO: Remove this when values stored as ForwardIndex stop depending on TextIndex config pinot-controller/src/main/java/org/apache/pinot/controller/BaseControllerStarter.java (4 lines): - line 166: // TODO: rename this variable once it's full separated with Helix controller. - line 237: // TODO: Use the same instance id for controller and participant when leadControllerResource is always enabled after - line 520: // TODO: Need to put this inside HelixResourceManager when HelixControllerLeadershipManager is removed. - line 644: * TODO: Cleanup orphan table config and schema pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java (4 lines): - line 43: * TODO: Add support for scalar functions auto registration. - line 158: // TODO: once we support other types of multiset, we should make CARDINALITY its own function - line 214: // TODO: Revisit whether we should return BOOLEAN instead - line 227: // TODO: Once VECTOR type is defined, we should update here. pinot-core/src/main/java/org/apache/pinot/core/plan/FilterPlanNode.java (4 lines): - line 139: // TODO: handle nested geography/geometry conversion functions - line 175: // TODO: handle nested geography/geometry conversion functions - line 242: // TODO: ExpressionFilterOperator does not support predicate types without PredicateEvaluator (TEXT_MATCH) - line 289: if (jsonIndex == null) { //TODO: rework pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/defaultcolumn/BaseDefaultColumnHandler.java (4 lines): - line 392: // TODO: Support chained derived column - line 403: // TODO: Support creation of derived columns from forward index disabled columns - line 414: // TODO: Support forward index disabled derived column - line 583: * TODO: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java (4 lines): - line 143: // TODO: Consider making the timeoutMs below table rebalancer configurable - line 215: // TODO - Add rebalanceJobId to all log messages for easy tracking. - line 522: // TODO: Consider allowing segment state change from CONSUMING to ONLINE - line 859: // TODO: Add a metric to estimate the total time it will take to rebalance. Need some good heuristics on how pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/instance/FDAwareInstancePartitionSelector.java (4 lines): - line 200: // TODO: refine this and segment assignment to minimize movement during numInstancesPerReplicaGroup uplift - line 241: // TODO:Non-replica-group based selection - line 322: //TODO: Finish preprocessing for Downsizing using unSetInstance() - line 381: //TODO: Finish normalizing for numReplicaGroups>numFaultDomains pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotJoinToDynamicBroadcastRule.java (4 lines): - line 107: * TODO #1: Only support SEMI-JOIN, once JOIN operator is supported by leaf-stage we should allow it to match - line 109: * TODO #2: Only convert to dynamic broadcast from right-to-left, allow option to specify dynamic broadcast direction. - line 110: * TODO #3: Only convert to dynamic broadcast if left is leaf stage, allow the option for intermediate stage. - line 111: * TODO #4: currently adding a pass-through after join, support leaf-stage to chain arbitrary operator(s) next. pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/QueryRunner.java (4 lines): - line 149: // TODO: Consider using separate config for intermediate stage and leaf stage - line 277: // TODO: Here we are breaking the stats invariants, sending errors without including the stats of the - line 309: * TODO: This design is at odds with MSE because MSE runs even the leaf stage via OpChainSchedulerService. - line 466: // TODO: Support pipeline breakers before merging this feature. pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotQueryRuleSets.java (4 lines): - line 62: // TODO: Revisit and see if they can be replaced with - line 69: // TODO: evaluate the SORT_JOIN_TRANSPOSE and SORT_JOIN_COPY rules - line 89: // TODO: Consider not reduce at all. - line 132: // TODO: Merge the following 2 rules into a single rule pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/TableConfigUtils.java (4 lines): - line 101: * FIXME: Merge this TableConfigUtils with the TableConfigUtils from pinot-common when merging of modules is done - line 152: * TODO: Add more validations for each section (e.g. validate conditions are met for aggregateMetrics) - line 370: // TODO: for multiple stream configs, validate them - line 627: // TODO: Remove these limitations pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/BaseJoinOperator.java (4 lines): - line 56: // TODO: Support memory size based resource limit. - line 147: // TODO: Add separate StatKey for each child join operator. - line 180: // TODO: Consider batching the rows to improve performance. - line 214: // TODO: Optimize this to avoid unnecessary object copy. pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java (3 lines): - line 406: // TODO: Support populating clientIp for GrpcRequestIdentity. - line 544: // TODO: Apply this to SSE as well - line 901: // TODO: Merge this with "mse" pinot-plugins/pinot-file-system/pinot-adls/src/main/java/org/apache/pinot/plugin/filesystem/ADLSGen2PinotFS.java (3 lines): - line 129: // TODO: consider to add the encryption of the following config - line 548: // TODO: need to find the other ways to check the directory if it becomes available. listFiles API returns - line 637: // TODO: the newer client now has the API 'uploadFromFile' that directly takes the file as an input. We can replace pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotRelDistributionTraitRule.java (3 lines): - line 111: // TODO: derive from input first, only if the result is ANY we change it to current - line 153: // TODO: we only map a single RelTrait from the LEFT table, later we should support RIGHT table as well - line 162: // TODO: add the rest of the nodes. pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/creator/SegmentGeneratorConfig.java (3 lines): - line 78: private String _recordReaderPath = null; //TODO: this should be renamed to recordReaderClass or even better removed - line 203: // TODO: Clean up the table configs with the deprecated settings, and always use the one in the indexing config - line 204: // TODO 2: Decide what to do with this. Index-spi is based on the idea that TableConfig is the source of truth pinot-core/src/main/java/org/apache/pinot/core/common/DataBlockCache.java (3 lines): - line 246: // TODO: This is not cached - line 253: // TODO: This is not cached - line 260: // TODO: This is not cached pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java (3 lines): - line 215: // TODO: more and more filters can be added later on, like excludeErrorSegments, excludeConsumingSegments, etc. - line 666: // TODO: support to force download immutable segments from RealTime table. - line 858: // TODO: more and more filters can be added later on, like excludeErrorSegments, excludeConsumingSegments, etc. pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandler.java (3 lines): - line 91: * TODO: Add support for the following: - line 324: // TODO: Also check if raw index version needs to be changed - line 918: // TODO: Move this logic as a static function in each index creator. pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java (3 lines): - line 190: * TODO: Consider adding a way to look up the usage of a function for better error message when there is no matching - line 221: * TODO: Move all usages to {@link #lookupFunctionInfo(String, ColumnDataType[])}. - line 318: // TODO: Revisit this. pinot-query-planner/src/main/java/org/apache/pinot/query/planner/physical/v2/PinotDataDistribution.java (3 lines): - line 47: * TODO: An alternative is to store workers separately. One reason workers are needed is because - line 145: // TODO: Add support for partial check (i.e. if distributed by [1], then we can avoid re-dist for constraint [1, 2]. - line 172: // TODO: Preserve collation too. pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/text/LuceneTextIndexReader.java (3 lines): - line 98: // TODO: consider using a threshold of num docs per segment to decide between building - line 229: // TODO: see if we can prefetch the pages - line 270: // TODO: it'd be better to unmap without flushing the buffer. Only some buffer types support it, though. pinot-query-runtime/src/main/java/org/apache/pinot/query/service/server/QueryServer.java (3 lines): - line 73: // TODO: Inbound messages can get quite large because we send the entire stage metadata map in each call. - line 82: // TODO: with complex query submission logic we should allow asynchronous query submission return instead of - line 260: // TODO: populate the thread context with TSE information pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/upsertcompactmerge/UpsertCompactMergeTaskGenerator.java (3 lines): - line 212: // TODO this can be later removed if we want to just do single-segment compaction from this task - line 216: // TODO see if multiple groups of same partition can be added - line 369: // TODO this check can be later removed if we want single-segment compaction from this task itself pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/mutable/MutableForwardIndex.java (3 lines): - line 101: // TODO: The new int[], long[], etc objects are not actually used as arrays iterated again, so we could skip the - line 103: // TODO: Longer methods are not optimized by C2 JIT. We should try to move each loop to a specific method. - line 452: * TODO: Not supported yet pinot-core/src/main/java/org/apache/pinot/core/common/datatable/DataTableBuilder.java (3 lines): - line 45: * TODO: Consider skipping seeking for the column offsets and directly write to the byte buffer - line 87: // TODO: Support MV BYTES - line 100: * TODO: Revisit this pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/exchange/BlockExchange.java (3 lines): - line 45: // TODO: Deduct this value via grpc config maximum byte size; and make it configurable with override. - line 46: // TODO: Max block size is a soft limit. only counts fixedSize datatable byte buffer - line 176: // TODO: This is a no-op right now. pinot-query-planner/src/main/java/org/apache/pinot/calcite/sql/fun/PinotOperatorTable.java (3 lines): - line 104: * TODO: Add more operators as needed. - line 187: // TODO: Replace these with SqlStdOperatorTable.LEAD and SqlStdOperatorTable.LAG when the function implementations - line 289: // TODO: Consider unifying the return type to Timestamp pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/HashJoinOperator.java (3 lines): - line 45: // TODO: Support memory size based resource limit. - line 57: // TODO: Revisit whether we should use IntList or RoaringBitmap for smaller memory footprint. - line 58: // TODO: Optimize this pinot-core/src/main/java/org/apache/pinot/core/operator/docidsets/AndDocIdSet.java (3 lines): - line 75: // TODO: Consider deciding the order based on the stats of BlockDocIdIterators - line 115: // TODO: 1. remainingDocIdIterators currently doesn't report cardinality; therefore, it cannot be - line 144: // TODO: Optimize this pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/creator/ForwardIndexCreator.java (3 lines): - line 94: // TODO: The new int[], long[], etc objects are not actually used as arrays iterated again, so we could skip the - line 96: // TODO: Longer methods are not optimized by C2 JIT. We should try to move each loop to a specific method. - line 262: * TODO: Not supported yet pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/memory/PagedPinotOutputStream.java (3 lines): - line 86: * TODO: Add one option that let caller choose start and end offset. - line 249: * TODO: Add one option that let caller choose start and end offset. - line 256: // TODO: We can remove this check pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisStreamMetadataProvider.java (3 lines): - line 136: // TODO: Once we start supporting multiple shards in a PartitionGroup, - line 189: // TODO: Revisit this. Kinesis starts consuming AFTER the start sequence number, and we might miss the first - line 220: * FIXME: Although practically the shard values follow this format, the Kinesis docs don't guarantee it. pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/SortedMailboxReceiveOperator.java (3 lines): - line 41: * TODO: Once sorting on the {@code MailboxSendOperator} is available, modify this to use a k-way merge instead of - line 55: // TODO: Support merge sort when sender side sort is supported. - line 92: // TODO: This might not be efficient because we are sorting all the received rows. We should use a k-way merge pinot-core/src/main/java/org/apache/pinot/core/query/scheduler/QueryScheduler.java (2 lines): - line 171: // TODO: Perform this check sooner during the serialization of DataTable. - line 197: * TODO: come up with other criteria for forcing a log and come up with better numbers pinot-core/src/main/java/org/apache/pinot/core/operator/dociditerators/MVScanDocIdIterator.java (2 lines): - line 64: // TODO: The performance can be improved by batching the docID lookups similar to how it's done in - line 102: // TODO: The performance can be improved by batching the docID lookups similar to how it's done in pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotSortExchangeNodeInsertRule.java (2 lines): - line 58: // TODO: Assess whether sorting is needed on both sender and receiver side or only receiver side. Potentially add - line 61: // TODO: Revisit whether we should use hash distribution pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java (2 lines): - line 137: // TODO: Change it to BooleanSupplier - line 423: // TODO: if a table can enable both dedup and upsert in the future, we need to revisit the preloading logic here, pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/mergerollup/MergeRollupTaskGenerator.java (2 lines): - line 364: // TODO: If there are many small merged segments, we should merge them again - line 996: * TODO: Current code will remove all metrics in case we invoke the ad-hoc task scheduling on a single table. pinot-core/src/main/java/org/apache/pinot/core/geospatial/transform/function/ScalarFunctions.java (2 lines): - line 229: // TODO: to fully support Geography contains operation. - line 254: // TODO: to fully support Geography within operation. pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisConsumer.java (2 lines): - line 94: // TODO: Revisit the offset handling logic. Reading after the start sequence number can lose the first message - line 121: // TODO: Revisit whether Kinesis can return empty batch when there are available records. The consumer cna handle pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotJoinExchangeNodeInsertRule.java (2 lines): - line 84: // TODO: Validate if the configured distribution types are valid - line 99: // TODO: Consider creating different JOIN Rel for each join strategy pinot-common/src/main/java/org/apache/pinot/common/utils/request/RequestUtils.java (2 lines): - line 228: // TODO: this check doesn't protect from overflow during big decimal -> long conversion! - line 238: // TODO: Support exact decimal value pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/AvroSchemaToPinotSchema.java (2 lines): - line 50: // TODO: support forbids = {"-avroDataFile"}, - line 54: // TODO: support forbids = {"-avroSchemaFile"}, pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/SumPrecisionAggregationFunction.java (2 lines): - line 351: // TODO: Revisit if we should change this to BIG_DECIMAL - line 368: // TODO: Revisit if we should change this to BIG_DECIMAL pinot-connectors/pinot-spark-3-connector/src/main/scala/org/apache/pinot/connector/spark/v3/datasource/PinotBufferedRecordReader.scala (2 lines): - line 31: * TODO: To improve resilience, write records to disk when memory is full. - line 39: // TODO: Honor 'fieldsToRead' parameter to avoid ingesting unwanted fields. pinot-tools/src/main/java/org/apache/pinot/tools/anonymizer/PinotDataAndQueryAnonymizer.java (2 lines): - line 690: // TODO: Re-implement query generator using SQL. Keep the PQL query generator for reference - line 1035: // // TODO: handle parenthesised predicate pinot-server/src/main/java/org/apache/pinot/server/starter/helix/HelixInstanceDataManager.java (2 lines): - line 84: // TODO: Consider making segment locks per table instead of per instance - line 350: // TODO: Consider using table data manager to delete the segment. This will allow the table data manager to clean pinot-connectors/pinot-spark-common/src/main/scala/org/apache/pinot/connector/spark/common/reader/PinotServerDataFetcher.scala (2 lines): - line 53: // TODO add support for TLS-secured server - line 101: // TODO: support netty-sec pinot-query-planner/src/main/java/org/apache/pinot/query/planner/physical/MailboxAssignmentVisitor.java (2 lines): - line 74: // TODO: Support local exchange with parallelism but no key - line 137: // TODO: Add support for more exchange types pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/partition/metadata/ColumnPartitionMetadata.java (2 lines): - line 113: * TODO: remove range format once all segments use integer format - line 146: * TODO: remove custom deserializer once all segments use integer format pinot-core/src/main/java/org/apache/pinot/core/operator/docvalsets/FilteredDataBlockValSet.java (2 lines): - line 38: * TODO: Support MV - line 90: // TODO: Needs to be changed when we start supporting MV in multistage pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/immutable/ImmutableSegmentLoader.java (2 lines): - line 57: // TODO: Clean up this class to use the schema from the IndexLoadingConfig - line 261: // FIXME: text-index only works with local SegmentDirectory pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/utils/BlockingMultiStreamConsumer.java (2 lines): - line 387: // TODO: Add the sender stage id to the error message - line 400: // TODO: Add the sender stage id to the error message pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/SegmentCompletionManager.java (2 lines): - line 56: // TODO Can we log using the segment name in the log message? - line 76: // TODO keep some history of past committed segments so that we can avoid looking up PROPERTYSTORE if some server pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/impl/NoDictionaryOnHeapDictionaryJointRule.java (2 lines): - line 91: // Exclude MV cols TODO: currently no index column is only applicable for SV columns, change this after it's - line 140: //TODO: improve this estimation pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/AggregateOperator.java (2 lines): - line 120: // TODO: Keeping only 'LIMIT' groups can cause inaccurate result because the groups are randomly selected - line 137: // TODO: Allow leaf return final result for non-group-by queries pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/HistogramAggregationFunction.java (2 lines): - line 110: // TODO: Represent infinity as literal instead of identifier - line 349: //TODO: Add MV support for histogram pinot-broker/src/main/java/org/apache/pinot/broker/api/resources/PinotBrokerDebug.java (2 lines): - line 85: // TODO: Add APIs to return the RoutingTable (with unavailable segments) - line 249: // TODO: Handle nested queries pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/EmptyResponseUtils.java (2 lines): - line 45: // TODO: Consider extracting common code for these 2 classes. - line 51: // TODO: Use schema to fill the correct data types for empty results. pinot-server/src/main/java/org/apache/pinot/server/predownload/PredownloadStatusRecorder.java (2 lines): - line 70: // TODO: make those name configurable - line 74: // TODO: make the path configurable pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/BaseMailboxReceiveOperator.java (2 lines): - line 82: // TODO: Revisit if we should throw exception here. - line 171: // TODO: Check if we should also release mailbox on not successful EOS. pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/text/NativeTextIndexReader.java (2 lines): - line 58: //TODO: Pass the load mode in (Direct, MMap) - line 123: // TODO: this method doesn't release native buffers held by _fst pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/WindowAggregateOperator.java (2 lines): - line 73: * TODO: - line 232: // TODO: Revisit null direction handling for all query types pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/RealtimeSegmentAssignment.java (2 lines): - line 86: // TODO: remove this check after we also refactor consuming segments assignment strategy - line 137: // TODO: Refactor check replication this for segment assignment strategy in follow up PR pinot-query-planner/src/main/java/org/apache/pinot/query/planner/physical/colocated/GreedyShuffleRewriteVisitor.java (2 lines): - line 80: // TODO: If this assumption is wrong, we can compute the reverse topological ordering explicitly. - line 326: // TODO: Only equality joins can be colocated. We don't have join clause info available right now. pinot-core/src/main/java/org/apache/pinot/core/util/GroupByUtils.java (2 lines): - line 118: // TODO: Keeping only 'LIMIT' groups can cause inaccurate result because the groups are randomly selected - line 164: // TODO: Resolve the HAVING clause within the IndexedTable before returning the result pinot-compatibility-verifier/src/main/java/org/apache/pinot/compat/SegmentOp.java (2 lines): - line 59: * TODO: - line 261: // TODO: verify by getting the number of rows before adding the segment, and the number of rows after adding the pinot-query-planner/src/main/java/org/apache/pinot/query/planner/physical/v2/PRelToPlanNodeConverter.java (2 lines): - line 202: // TODO: The constants are already extracted in the PinotWindowExchangeNodeInsertRule, we can remove them from - line 338: * TODO: Revisit this method: pinot-core/src/main/java/org/apache/pinot/server/realtime/ServerSegmentCompletionProtocolHandler.java (2 lines): - line 67: // TODO: Use exception based code to handle different types of exceptions. - line 131: // TODO We need to make this work with trusted certificates if the VIP is using https. pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/IndexLoadingConfig.java (2 lines): - line 61: // TODO: Revisit them - line 91: * TODO: Revisit the init handling. Currently it doesn't apply tiered config override pinot-common/src/main/java/org/apache/pinot/common/utils/FileUploadDownloadClient.java (2 lines): - line 665: * TODO: fix the realtime segment commit protocol to add table name as a parameter. - line 959: * TODO: migrate this method to another class pinot-broker/src/main/java/org/apache/pinot/broker/api/resources/PinotClientRequest.java (2 lines): - line 314: // TODO: Not implemented yet. - line 604: * TODO: Add more comparison logic for different query types. This would require handling edge cases with group pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/refreshsegment/RefreshSegmentTaskExecutor.java (2 lines): - line 94: // TODO: Instead of relying on needPreprocess(), process segment metadata file to determine if refresh is needed. - line 125: // TODO: Maybe we can support singleValue to multi-value conversions are supproted and vice-versa. pinot-query-planner/src/main/java/org/apache/pinot/query/planner/physical/v2/opt/rules/AggregatePushdownRule.java (2 lines): - line 145: // TODO: Currently it only handles one WITHIN GROUP collation across all AggregateCalls. - line 239: // TODO: Revisit the following logic: pinot-minion/src/main/java/org/apache/pinot/minion/BaseMinionStarter.java (2 lines): - line 229: // TODO: put all the metrics related configs down to "pinot.server.metrics" - line 316: // TODO: add health check here pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/bloom/OnHeapGuavaBloomFilterCreator.java (2 lines): - line 39: *

TODO: Remove the dependency on {@link BloomFilter} and have our own implementation to prevent guava library - line 53: // TODO: This method is here for compatibility reasons, should be removed in future PRs pinot-query-planner/src/main/java/org/apache/pinot/query/planner/logical/PinotLogicalQueryPlanner.java (2 lines): - line 70: // TODO: Currently we don't support multiple sub-plans. Revisit the following logic when we add the support. - line 123: // TODO: Should be SINGLETON (currently SINGLETON has to be local, so use BROADCAST_DISTRIBUTED instead) pinot-core/src/main/java/org/apache/pinot/core/query/executor/ServerQueryExecutorV1Impl.java (2 lines): - line 216: // TODO: the freshness time should not be collected at query time because there is no guarantee that the consuming - line 270: // TODO: Change broker to watch both IdealState and ExternalView to not query the removed segments pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java (2 lines): - line 1408: // TODO: detect if an index changes from Dictionary to Variable Length Dictionary or vice versa. - line 1409: // TODO: RV TEST pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/BaseMultipleSegmentsConversionExecutor.java (2 lines): - line 77: * TODO: add test for SegmentZKMetadataCustomMapModifier - line 300: // TODO: This is not clean to put the override here, but let's think about it harder to see what is the proper pinot-common/src/main/java/org/apache/pinot/common/request/context/LiteralContext.java (2 lines): - line 41: // TODO: Support all of the types for sql. - line 310: // TODO: Remove single quotes except for string and bytes after broker reducer supports expression format change. pinot-timeseries/pinot-timeseries-spi/src/main/java/org/apache/pinot/tsdb/spi/operator/BaseTimeSeriesOperator.java (2 lines): - line 30: * TODO: Add common hierarchy with other operators like Multistage and Pinot core. This will likely require us to - line 48: // TODO: add stats pinot-query-planner/src/main/java/org/apache/pinot/query/planner/physical/v2/PlanFragmentAndMailboxAssignment.java (2 lines): - line 94: // TODO: This is quite a complex invariant. - line 127: false /* TODO: set sort on receiver */, false /* TODO: set sort on sender */, sendNode); pinot-core/src/main/java/org/apache/pinot/core/transport/TableRouteInfo.java (2 lines): - line 91: * TODO: Note that this method cannot be supported for Logical Tables as the classes cannot be used to maintain a - line 101: * TODO: Note that this method cannot be supported for Logical Tables as the classes cannot be used to maintain a pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/forward/FixedByteMVMutableForwardIndex.java (2 lines): - line 93: // TODO: Optimize it - line 195: // TODO Use powers of two for _rowCountPerChunk to optimize computation for the pinot-plugins/pinot-input-format/pinot-orc/src/main/java/org/apache/pinot/plugin/inputformat/orc/ORCRecordReader.java (2 lines): - line 58: *

  • BOOLEAN -> String
  • TODO: -> Boolean? - line 65: *
  • LIST -> Object[] of the supported types
  • TODO: -> List? pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotTaskManager.java (2 lines): - line 744: // TODO: find a better way to report task generation information - line 762: // TODO: find a better way to report task generation information pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/LookupJoinOperator.java (2 lines): - line 132: // TODO: Consider batching the rows to improve performance. - line 171: // TODO: Optimize this to avoid unnecessary object copy. pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java (2 lines): - line 437: // TODO Add access control similar to PinotSegmentUploadDownloadRestletResource for segment download. - line 462: // TODO Limit the number of concurrent downloads of segments because compression is an expensive operation. pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/ReceivingMailbox.java (2 lines): - line 68: // TODO: Make the queue size configurable - line 69: // TODO: Revisit if this is the correct way to apply back pressure pinot-spi/src/main/java/org/apache/pinot/spi/stream/PartitionLevelConsumer.java (2 lines): - line 36: * TODO Issue 5359 remove this API once external kafka consumers implements return of StreamPartitionMsgOffset - line 65: // TODO Issue 5359 remove this default implementation once all kafka consumers have migrated to use this API pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotUpsertRestletResource.java (2 lines): - line 102: // TODO: Switch to use TableConfigs - line 147: // Here we only calculate the map content size. TODO: Add the map entry size and the array size within the map. pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/converter/SegmentV1V2ToV3FormatConverter.java (2 lines): - line 221: // TODO: see if this can be done by reusing some existing methods - line 258: // TODO: see if this can be done by reusing some existing methods pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/hint/PinotHintOptions.java (2 lines): - line 58: // TODO: Apply this to leaf stage as well - line 126: // TODO: Consider adding a Join implementation with join strategy. pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MultiStageOperator.java (2 lines): - line 157: // TODO: Ideally close() call should finish within request deadline. - line 158: // TODO: Consider passing deadline as part of the API. pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/text/TextIndexType.java (2 lines): - line 69: // TODO: Should V1Constants.Indexes.LUCENE_TEXT_INDEX_DOCID_MAPPING_FILE_EXTENSION be added here? - line 164: // TODO: Support loading native text index from a PinotDataBuffer pinot-common/src/main/java/org/apache/pinot/common/datablock/ZeroCopyDataBlockSerde.java (2 lines): - line 128: // TODO: probably we can also do this when length is 0 - line 145: // TODO: This is actually allocating, we can improve it in the future if it is problematic pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/TransformFunctionFactory.java (2 lines): - line 378: // TODO: Move to a test util class - line 388: // TODO: Move to a test util class pinot-core/src/main/java/org/apache/pinot/core/common/DataFetcher.java (2 lines): - line 50: // TODO: Figure out a way to close the reader context within the ColumnValueReader - line 305: * TODO: Type conversion for BOOLEAN and TIMESTAMP is not handled pinot-spi/src/main/java/org/apache/pinot/spi/config/table/SegmentsValidationAndRetentionConfig.java (2 lines): - line 29: // TODO: Consider break this config into multiple configs - line 79: // TODO: Get field spec of _timeColumnName from Schema for the timeType pinot-controller/src/main/java/org/apache/pinot/controller/util/SegmentIntervalUtils.java (2 lines): - line 36: * TODO: Use TimeFieldSpec in Schema - line 45: * TODO: Use TimeFieldSpec in Schema pinot-core/src/main/java/org/apache/pinot/core/operator/ExecutionStatistics.java (2 lines): - line 30: // TODO: make the semantic of entry the same in _numEntriesScannedInFilter and _numEntriesScannedPostFilter. Currently - line 38: // TODO: Remove _numTotalDocs because it is not execution stats, and it is not set from the operators because they pinot-common/src/main/java/org/apache/pinot/common/datablock/BaseDataBlock.java (2 lines): - line 190: // TODO: Add a allocation free deserialization mechanism. - line 345: // TODO: This will allocate. Can we change code to use ImmutableRoaringBitmap? pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/NonEquiJoinOperator.java (2 lines): - line 42: // TODO: Revisit whether we should use IntList or RoaringBitmap for smaller memory footprint. - line 108: // TODO: Optimize this to avoid unnecessary object copy. pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/AggregationFunctionType.java (2 lines): - line 53: // TODO: min/max only supports NUMERIC in Pinot, where Calcite supports COMPARABLE_ORDERED - line 160: // TODO: revisit support for ExprMin/Max count in V2, particularly plug query rewriter in the right place pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java (2 lines): - line 209: // TODO introduce a table config ctor with json string. - line 247: // TODO: validate that table was created successfully pinot-query-planner/src/main/java/org/apache/pinot/query/planner/logical/RexExpressionUtils.java (2 lines): - line 178: // TODO: This needs to be improved. - line 225: // TODO: Handle SYMBOL in the planning phase. pinot-query-planner/src/main/java/org/apache/pinot/query/context/PhysicalPlannerContext.java (2 lines): - line 46: * TODO: We should not use this map and instead have a centralized place for instanceId to QueryServerInstance - line 62: * TODO: Controller should only rely on SQL parser to extract table names. pinot-spi/src/main/java/org/apache/pinot/spi/config/table/IndexingConfig.java (2 lines): - line 58: private List _noDictionaryColumns; // TODO: replace this with noDictionaryConfig. - line 95: // TODO: Add a new configuration related to the segment generation pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/SortOperator.java (2 lines): - line 69: // TODO: make this default behavior configurable. - line 166: // TODO: when push-down properly, we shouldn't get more than _numRowsToKeep pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/vector/HnswVectorIndexReader.java (2 lines): - line 65: // TODO: consider using a threshold of num docs per segment to decide between building - line 157: // TODO: see if we can prefetch the pages pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MailboxSendOperator.java (2 lines): - line 55: * TODO: Add support to sort the data prior to sending if sorting is enabled - line 69: // TODO: Support sort on sender pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/nativefst/NativeFSTIndexReader.java (2 lines): - line 51: // TODO: Implement an InputStream directly on PinotDataBuffer - line 75: //TODO: why does this class not close FST ? pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/DefaultRebalancePreChecker.java (2 lines): - line 97: // TODO: give this check a separate threshold other than the disk utilization threshold - line 114: * TODO: Add an API to check for whether segments in deep store are up to date with the table configs and schema pinot-spi/src/main/java/org/apache/pinot/spi/utils/IngestionConfigUtils.java (2 lines): - line 90: // TODO: Revisit the callers of this method. We should use the stream config for a given partition, instead of the - line 96: // TODO: Revisit the callers of this method. We should use the stream config for a given partition, instead of the pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/InvertedIndexAndDictionaryBasedForwardIndexCreator.java (2 lines): - line 63: * TODO: Currently for multi-value columns generating the forward index can lead to a data loss as frequency information - line 67: * TODO (index-spi): Rename this class, as it is not an implementation of pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/writer/StatelessRealtimeSegmentWriter.java (2 lines): - line 81: * TODO: - line 417: // TODO: currentPartitionGroupConsumptionStatus should be fetched from idealState + segmentZkMetadata, pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/segment/SizeBasedSegmentFlushThresholdComputer.java (2 lines): - line 147: // TODO: add feature to adjust time threshold as well - line 196: // TODO Picking Integer.MAX_VALUE for number of rows will most certainly make the segment unloadable pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/BinaryOperatorTransformFunction.java (2 lines): - line 49: * TODO: Support MV columns - line 154: * TODO: Extract the common logic. pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/SchemaUtils.java (2 lines): - line 45: * FIXME: Merge this SchemaUtils with the SchemaUtils from pinot-common when merging of modules happens - line 94: * TODO: Transform functions have moved to table config. Once we stop supporting them in schema, remove the pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/LeafStageTransferableBlockOperator.java (2 lines): - line 391: // TODO: Revisit if we should treat all exceptions as query failure. Currently MERGE_RESPONSE_ERROR and - line 409: // TODO: this latch mechanism is not the most elegant. We should change it to use a CompletionService. pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/utils/TypeUtils.java (2 lines): - line 38: * TODO: Revisit to see if we should use original type instead of stored type - line 98: // TODO: Add more conversions pinot-core/src/main/java/org/apache/pinot/core/operator/docvalsets/DataBlockValSet.java (2 lines): - line 37: * TODO: Support MV - line 67: // TODO: Needs to be changed when we start supporting MV in multistage pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/RealtimeProvisioningHelperCommand.java (2 lines): - line 260: // TODO: allow multiple segments. - line 291: // TODO: Make a recommendation of what config to choose by considering more inputs such as qps pinot-core/src/main/java/org/apache/pinot/core/common/datablock/DataBlockBuilder.java (2 lines): - line 76: // TODO: consolidate these null utils into data table utils. - line 218: // TODO: consolidate these null utils into data table utils. pinot-core/src/main/java/org/apache/pinot/core/operator/combine/BaseCombineOperator.java (2 lines): - line 194: // TODO: Do not use class name here but the operator name explain plan. To do so, that method must be moved from - line 208: // TODO: Only include exception message if it is a QueryException. Otherwise, it might expose sensitive information pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/LLCSegmentCompletionHandlers.java (2 lines): - line 207: // TODO: remove this API. Should not upload segment via controller - line 277: // TODO: memoryUsedInBytes = 0 if not present in params. Add validation when we start using it pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/MinAggregationFunction.java (1 line): - line 142: // TODO: even though the source data has BIG_DECIMAL type, we still only support double precision. pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/OfflineSegmentAssignment.java (1 line): - line 72: // TODO: Right now as per tier assignment, different instances will be picked up for different tiers which pinot-server/src/main/java/org/apache/pinot/server/starter/helix/SegmentMessageHandlerFactory.java (1 line): - line 136: // TODO: check _segmentName to be backward compatible. Moving forward, we just need to check the list to pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/readers/sort/PinotSegmentSorter.java (1 line): - line 43: * TODO: add the support for no-dictionary and multi-value columns. pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/JsonExtractIndexTransformFunction.java (1 line): - line 76: if (_jsonIndexReader == null) { //TODO: rework pinot-core/src/main/java/org/apache/pinot/core/query/request/context/QueryContext.java (1 line): - line 598: // TODO: Add validation logic here pinot-core/src/main/java/org/apache/pinot/core/query/request/ServerQueryRequest.java (1 line): - line 124: // TODO: support optional segments for GrpcQueryServer pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/impl/BloomFilterRule.java (1 line): - line 100: * TODO: The partitioned dimension should used in the “=” (IN, NOT IN, != are not using bloom filter in Pinot for pinot-spi/src/main/java/org/apache/pinot/spi/data/Schema.java (1 line): - line 915: // TODO: Switch to new format after releasing 0.11.0 pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/HeaderAdder.java (1 line): - line 65: // TODO HACK TO BE REMOVED ONCE CLIENTS HAVE UPGRADED TO NEW THIRD-EYE JAR pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/ConsistentDataPushUtils.java (1 line): - line 219: // TODO: Remove the check for REFRESH when we support consistent push for APPEND table pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/PercentileTDigestAggregationFunction.java (1 line): - line 40: * TODO: Decided not to support custom compression for version 0 queries as it seems to be the older syntax and requires pinot-core/src/main/java/org/apache/pinot/core/segment/processing/timehandler/TimeHandler.java (1 line): - line 34: // TODO: Support DATE_TIME handler which rounds and partitions time on calendar date boundary with timezone support pinot-common/src/main/java/org/apache/pinot/sql/parsers/SqlNodeAndOptions.java (1 line): - line 29: // TODO: support option literals other than STRING pinot-spi/src/main/java/org/apache/pinot/spi/executor/DecoratorExecutorService.java (1 line): - line 43: * TODO: Convert this class and its usages into an Executor instead of an ExecutorService pinot-core/src/main/java/org/apache/pinot/core/transport/grpc/GrpcQueryServer.java (1 line): - line 68: // TODO: Plug in QueryScheduler pinot-connectors/pinot-spark-3-connector/src/main/scala/org/apache/pinot/connector/spark/v3/datasource/PinotWriteBuilder.scala (1 line): - line 31: // TODO: utilize filters pinot-query-planner/src/main/java/org/apache/pinot/query/planner/logical/LiteralValueNode.java (1 line): - line 30: * TODO: A placeholder class for literal values coming after SubPlan execution. pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/util/PinotDataBitSetV2.java (1 line): - line 27: // TODO: This is copied from DocIdSetPlanNode. pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/CompositeTransformer.java (1 line): - line 43: // TODO: Integrate ComplexTypeTransformer into the CompositeTransformer. pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceSummaryResult.java (1 line): - line 415: // TODO: Add a metric to estimate the total time it will take to rebalance pinot-spi/src/main/java/org/apache/pinot/spi/stream/StreamConfigProperties.java (1 line): - line 33: // TODO: this can be removed, check all properties before doing so pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/server/PulsarDataProducer.java (1 line): - line 38: * TODO: Improve the current implementation that creates the producer object for each `produce()` call. pinot-core/src/main/java/org/apache/pinot/core/operator/query/GroupByOperator.java (1 line): - line 138: // TODO: Currently the groups are not trimmed if there is no ordering specified. Consider ordering on group-by pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceChecker.java (1 line): - line 186: // TODO: Just exponential backoff by factor 2 for now. Can add other retry polices as needed. pinot-tools/src/main/java/org/apache/pinot/tools/admin/PinotAdministrator.java (1 line): - line 163: // TODO: Use the natively supported version and usage by picocli pinot-core/src/main/java/org/apache/pinot/core/query/selection/SelectionOperatorService.java (1 line): - line 81: * TODO: Do merge sort after releasing 0.13.0 when server side results are sorted pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/groupby/GroupByExecutor.java (1 line): - line 54: * TODO: benchmark the performance of PQ vs. topK pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisStreamMessageMetadata.java (1 line): - line 27: // TODO: Make it a util class pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MultistageAggregationExecutor.java (1 line): - line 157: // TODO: Fix it pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/SingleConnectionBrokerRequestHandler.java (1 line): - line 122: // TODO Use scatterGatherStats as serverStats pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java (1 line): - line 630: // TODO: when ttl is enabled, we can allow pinot-broker/src/main/java/org/apache/pinot/broker/routing/instanceselector/MultiStageReplicaGroupSelector.java (1 line): - line 156: // TODO: Evaluate whether we need to provide support for COMPLETE partitions. pinot-core/src/main/java/org/apache/pinot/core/operator/filter/ScanBasedFilterOperator.java (1 line): - line 96: * TODO: Replace this with a priority method for all filter operators pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/io/params/NoDictionaryOnHeapDictionaryJointRuleParams.java (1 line): - line 52: // TODO: improve this estimation if possible pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/partition/PartitionFunctionFactory.java (1 line): - line 67: // TODO: introduce a way to inject custom partition function pinot-controller/src/main/resources/app/components/SearchBar.tsx (1 line): - line 25: // TODO: Remove unneccessary styles, This styles are from demo of material-ui doc pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/geospatial/MutableH3Index.java (1 line): - line 78: // TODO: support multiple resolutions pinot-segment-local/src/main/java/org/apache/pinot/segment/local/aggregator/PercentileEstValueAggregator.java (1 line): - line 30: // TODO: This is copied from PercentileEstAggregationFunction. pinot-common/src/main/java/org/apache/pinot/common/messages/SegmentRefreshMessage.java (1 line): - line 35: * TODO: "tableName" and "segmentName" are new added fields. Change SegmentMessageHandlerFactory to use these 2 fields pinot-core/src/main/java/org/apache/pinot/core/query/distinct/DistinctExecutor.java (1 line): - line 29: // TODO: Tune the initial capacity pinot-plugins/pinot-timeseries-lang/pinot-timeseries-m3ql/src/main/java/org/apache/pinot/tsdb/m3ql/parser/Tokenizer.java (1 line): - line 27: * TODO: Dummy implementation. Will be switched out with a proper implementation soon. pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotHelixTaskResourceManager.java (1 line): - line 830: // TODO: If a task consists of subtasks belonging to the current table and other tables at the same time, pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/creator/name/NormalizedDateSegmentNameGenerator.java (1 line): - line 37: // TODO: This we defined in CommonConstants in common module. SPI should depend on common, so copying here for now, pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/TransformPipeline.java (1 line): - line 115: // TODO: consolidate complex type transformer into composite type transformer pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java (1 line): - line 262: // TODO fail these queries going forward. Added this logic to take care of tautologies like BETWEEN 0 and -1. pinot-query-planner/src/main/java/org/apache/pinot/query/planner/serde/PlanNodeSerializer.java (1 line): - line 348: // TODO: Currently we only support ASCENDING and DESCENDING. pinot-core/src/main/java/org/apache/pinot/core/query/scheduler/BinaryWorkloadScheduler.java (1 line): - line 130: // TODO: Considering setting a lower priority to avoid busy loop when all threads are busy processing queries. pinot-core/src/main/java/org/apache/pinot/core/query/scheduler/resources/ResourceManager.java (1 line): - line 43: // TODO: This class supports hard and soft thread limits. Potentially, we can make pinot-core/src/main/java/org/apache/pinot/core/query/pruner/BloomFilterSegmentPruner.java (1 line): - line 55: // TODO: make this threshold configurable? threshold 10 is also used in CombinePlanNode, which accesses the pinot-broker/src/main/java/org/apache/pinot/broker/routing/timeboundary/TimeBoundaryManager.java (1 line): - line 54: *

    TODO: Support SDF (simple date format) time column pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/store/SegmentLocalFSDirectory.java (1 line): - line 405: // TODO: thread-safety. Single writer may be shared pinot-core/src/main/java/org/apache/pinot/core/query/logger/ServerQueryLogger.java (1 line): - line 207: * TODO: come up with other criteria for forcing a log and come up with better numbers. pinot-minion/src/main/java/org/apache/pinot/minion/MinionAdminApiApplication.java (1 line): - line 65: // TODO: Add bindings as needed in future. pinot-controller/src/main/resources/app/theme/index.ts (1 line): - line 56: // TODO: remove all StyledButton usages pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/forward/FixedByteSVMutableForwardIndex.java (1 line): - line 48: // TODO: Optimize it pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/forward/VarByteChunkForwardIndexReaderV4.java (1 line): - line 51: * TODO: Consider reading directly from sliced ByteBuffer instead of copying to byte[] first pinot-query-planner/src/main/java/org/apache/pinot/query/planner/physical/DispatchableSubPlan.java (1 line): - line 147: // TODO: this isn't entirely accurate and can be improved. One issue is that the maxExecutionThreads can be pinot-spi/src/main/java/org/apache/pinot/spi/executor/MdcExecutor.java (1 line): - line 34: * TODO: Convert this class and its usages into an Executor instead of an ExecutorService pinot-common/src/main/java/org/apache/pinot/common/cursors/AbstractResponseStore.java (1 line): - line 188: // TODO: Set a result table with schema and no rows pinot-controller/src/main/java/org/apache/pinot/controller/validation/OfflineSegmentIntervalChecker.java (1 line): - line 150: // TODO: we can further split the existing ValidationMetricName enum to OFFLINE and REALTIME, pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/logical/PinotLogicalExchange.java (1 line): - line 44: // TODO: Revisit this as we add more custom distribution types. pinot-server/src/main/java/org/apache/pinot/server/starter/helix/SegmentOnlineOfflineStateModelFactory.java (1 line): - line 48: // TODO: Check how Helix handle CONSUMING -> DROPPED transition and remove this cache if it's not needed. pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneTextIndex.java (1 line): - line 194: // TODO: Optimize this similar to how we have done for offline/completed segments. pinot-controller/src/main/java/org/apache/pinot/controller/util/CompletionServiceHelper.java (1 line): - line 83: // TODO: use some service other than completion service so that we know which server encounters the error pinot-core/src/main/java/org/apache/pinot/core/operator/blocks/ProjectionBlock.java (1 line): - line 67: // TODO: only support one level of path for now, e.g. `map.key` pinot-core/src/main/java/org/apache/pinot/core/plan/DistinctPlanNode.java (1 line): - line 64: // TODO: reserve special value in dictionary (e.g. -1) for null in the future so pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/QuickstartRunner.java (1 line): - line 222: // TODO: Stop Minion pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/StartBrokerCommand.java (1 line): - line 63: // TODO: support forbids = {"-brokerHost", "-brokerPort"}) pinot-broker/src/main/java/org/apache/pinot/broker/routing/adaptiveserverselector/LatencySelector.java (1 line): - line 49: // TODO: If two or more servers have same latency, break the tie intelligently. pinot-plugins/pinot-stream-ingestion/pinot-kafka-base/src/main/java/org/apache/pinot/plugin/stream/kafka/KafkaConfigBackwardCompatibleUtils.java (1 line): - line 47: //FIXME: This needs to be done because maven shade plugin also overwrites the constants in the classes pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/groupby/NoDictionaryMultiColumnGroupKeyGenerator.java (1 line): - line 47: * TODO: pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/SumValuesIntegerTupleSketchAggregationFunction.java (1 line): - line 37: // TODO if extra aggregation modes are supported, make this switch pinot-common/src/main/java/org/apache/pinot/common/response/broker/BrokerResponseNativeV2.java (1 line): - line 38: * TODO: Currently this class cannot be used to deserialize the JSON response. pinot-server/src/main/java/org/apache/pinot/server/starter/helix/IngestionBasedConsumptionStatusChecker.java (1 line): - line 145: // TODO: implement a way to have this work correctly for kafka consumers pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/StartServerCommand.java (1 line): - line 81: // TODO support forbids = {"-serverHost", "-serverPort", "-dataDir", "-segmentDir"} pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/CoalesceTransformFunction.java (1 line): - line 300: // TODO: Support BOOLEAN, TIMESTAMP, BYTES pinot-common/src/main/java/org/apache/pinot/common/request/context/RequestContextUtils.java (1 line): - line 426: // TODO: Pass the literal context into the Predicate so that we can read the value based on the data type. Currently pinot-controller/src/main/resources/app/router.tsx (1 line): - line 40: // TODO: make async pinot-spi/src/main/java/org/apache/pinot/spi/data/SchemaValidatorFactory.java (1 line): - line 38: //TODO: support schema validation for more data formats like ORC. pinot-broker/src/main/java/org/apache/pinot/broker/routing/instanceselector/BaseInstanceSelector.java (1 line): - line 80: * TODO: refresh new/old segment state where there is no update from helix for long time. pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/SumAggregationFunction.java (1 line): - line 138: // TODO: even though the source data has BIG_DECIMAL type, we still only support double precision. pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/timeseries/LeafTimeSeriesOperator.java (1 line): - line 60: // TODO: Return error in the TimeSeriesBlock instead? pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/FlinkSegmentWriter.java (1 line): - line 203: // TODO: Revisit whether we should transform the row pinot-core/src/main/java/org/apache/pinot/core/segment/processing/genericrow/GenericRowFileWriter.java (1 line): - line 39: * TODO: Consider using ByteBuffer instead of OutputStream. pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/enricher/function/CustomFunctionEnricher.java (1 line): - line 37: * TODO: Merge this and ExpressionTransformer pinot-timeseries/pinot-timeseries-spi/src/main/java/org/apache/pinot/tsdb/spi/RangeTimeSeriesRequest.java (1 line): - line 51: // TODO: It's not ideal to have another default, that plays with numGroupsLimit, etc. pinot-core/src/main/java/org/apache/pinot/core/query/optimizer/statement/JsonStatementOptimizer.java (1 line): - line 108: * us to implicitly convert the output of json path expression to DOUBLE. TODO: There are better ways of doing this pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/FilterOperator.java (1 line): - line 99: // TODO: Consider batching the rows to improve performance. pinot-core/src/main/java/org/apache/pinot/core/operator/query/FilteredGroupByOperator.java (1 line): - line 189: // TODO: Currently the groups are not trimmed if there is no ordering specified. Consider ordering on group-by pinot-plugins/pinot-segment-writer/pinot-segment-writer-file-based/src/main/java/org/apache/pinot/plugin/segmentwriter/filebased/FileBasedSegmentWriter.java (1 line): - line 144: // TODO: Revisit whether we should transform the row pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/datasource/DataSource.java (1 line): - line 81: *

    TODO: Have a separate interface for range index. pinot-core/src/main/java/org/apache/pinot/core/operator/combine/GroupByCombineOperator.java (1 line): - line 158: //TODO: change upsert api so that it accepts intermediateRecord directly pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/NativeMutableTextIndex.java (1 line): - line 43: //TODO: Move to mutable dictionary pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/forward/VarByteChunkMVForwardIndexReader.java (1 line): - line 35: * TODO: Consider reading directly from sliced ByteBuffer instead of copying to byte[] first pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/nativefst/automaton/Automaton.java (1 line): - line 312: //TODO: move to bitsets pinot-common/src/main/java/org/apache/pinot/common/utils/ServiceStartableUtils.java (1 line): - line 91: // TODO: Currently it puts all keys to the instance config. Consider standardizing instance config keys and pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/fwd/CLPForwardIndexCreatorV2.java (1 line): - line 322: * over serdes overhead. TODO: add the code in a separate PR to simplify review process pinot-core/src/main/java/org/apache/pinot/core/geospatial/transform/function/StWithinFunction.java (1 line): - line 54: // TODO: to fully support Geography within operation. pinot-core/src/main/java/org/apache/pinot/core/operator/combine/CombineOperatorUtils.java (1 line): - line 52: // TODO: Check all operators and properly implement the getIndexSegment. pinot-query-runtime/src/main/java/org/apache/pinot/query/service/dispatch/DispatchClient.java (1 line): - line 40: * TODO: It might be neater to implement pooling at the client level. Two options: (1) Pass a channel provider and pinot-broker/src/main/java/org/apache/pinot/broker/grpc/BrokerGrpcServer.java (1 line): - line 335: //TODO: move this method from OSS Pinot class into util, and then re-use this util pinot-core/src/main/java/org/apache/pinot/core/operator/docvalsets/FilteredRowBasedBlockValSet.java (1 line): - line 40: * TODO: Support MV pinot-query-planner/src/main/java/org/apache/pinot/query/context/PlannerContext.java (1 line): - line 43: * TODO: currently we don't support option or query rewrite. pinot-core/src/main/java/org/apache/pinot/core/query/optimizer/filter/FilterOptimizer.java (1 line): - line 28: * TODO: Support AlwaysTrueFilter and AlwaysFalseFilter pinot-query-planner/src/main/java/org/apache/pinot/query/planner/physical/DispatchablePlanVisitor.java (1 line): - line 64: // TODO: Figure out a way to parallelize Empty OVER() and OVER(ORDER BY) so the computation can be done across pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/lineage/DefaultLineageManager.java (1 line): - line 121: // TODO: Currently, we preserve the replaced segments for 1 day for REFRESH tables only. Once we support pinot-timeseries/pinot-timeseries-spi/src/main/java/org/apache/pinot/tsdb/spi/series/TimeSeries.java (1 line): - line 146: // TODO: This can be cleaned up pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/inv/geospatial/BaseH3IndexCreator.java (1 line): - line 116: // TODO: support multiple resolutions pinot-common/src/main/java/org/apache/pinot/common/segment/generation/SegmentGenerationUtils.java (1 line): - line 263: //TODO: sort input files based on creation time pinot-spi/src/main/java/org/apache/pinot/spi/config/table/TierConfig.java (1 line): - line 58: // TODO: only "serverTag" is supported currently. In next iteration, "InstanceAssignmentConfig pinot-plugins/pinot-stream-ingestion/pinot-kafka-base/src/main/java/org/apache/pinot/plugin/stream/kafka/KafkaStreamMessageMetadata.java (1 line): - line 26: // TODO: Make it a util class pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/window/aggregate/WindowValueAggregatorFactory.java (1 line): - line 41: // TODO: Add type specific aggregator implementations pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/operands/FunctionOperand.java (1 line): - line 109: // TODO: Optimize per record conversion pinot-broker/src/main/java/org/apache/pinot/broker/routing/adaptiveserverselector/NumInFlightReqSelector.java (1 line): - line 47: // TODO: If two or more servers have same number of in flight requests, break the tie intelligently. pinot-plugins/pinot-file-system/pinot-s3/src/main/java/org/apache/pinot/plugin/filesystem/S3PinotFS.java (1 line): - line 728: // TODO: parts can be uploaded in parallel for higher throughput, given a thread pool. pinot-spi/src/main/java/org/apache/pinot/spi/config/table/assignment/InstanceReplicaGroupPartitionConfig.java (1 line): - line 59: // TODO: remove this config in the next official release pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/channel/MailboxStatusObserver.java (1 line): - line 51: // TODO: this feedback info is not used to throttle the send speed. it is currently being discarded. pinot-spi/src/main/java/org/apache/pinot/spi/config/table/DedupConfig.java (1 line): - line 30: // TODO: Consider removing this field. It should always be true when DedupConfig is present. pinot-tools/src/main/java/org/apache/pinot/tools/PinotToolLauncher.java (1 line): - line 58: // TODO: Use the natively supported version and usage by picocli pinot-compatibility-verifier/src/main/java/org/apache/pinot/compat/StreamOp.java (1 line): - line 67: * TODO: Consider using a file-based stream, where "pushing" events is simply adding new files to pinot-common/src/main/java/org/apache/pinot/common/utils/tls/JvmDefaultSslContext.java (1 line): - line 52: * TODO: need to support "javax.net.ssl.keyStoreProvider", "javax.net.ssl.trustStoreProvider", "https.protocols" and pinot-query-planner/src/main/java/org/apache/pinot/query/routing/WorkerMetadata.java (1 line): - line 41: * TODO: WorkerMetadata now doesn't have info directly about how to construct the mailboxes. instead it rely on pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/instance/InstanceAssignmentDriver.java (1 line): - line 130: // TODO: support more constraints pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/MultiStageBrokerRequestHandler.java (1 line): - line 498: // TODO: Currently we don't emit metric for QUERY_TOTAL_TIME_MS pinot-spi/src/main/java/org/apache/pinot/spi/stream/StreamMetadataProvider.java (1 line): - line 109: * TODO - Remove the flag and fix the clients calling computePartitionGroupMetadata() pinot-controller/src/main/java/org/apache/pinot/controller/util/ConsumingSegmentInfoReader.java (1 line): - line 240: // TODO: Invert response to be a map of partition to a vector of [currentOffset, recordsLag, latestUpstreamOffset, pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/forward/CLPMutableForwardIndex.java (1 line): - line 37: // TODO: We can get better dynamic estimates using segment stats pinot-spi/src/main/java/org/apache/pinot/spi/ingestion/batch/spec/SegmentGenerationJobSpec.java (1 line): - line 51: // TODO: set the default value to false after all clients are aware of this. pinot-core/src/main/java/org/apache/pinot/core/operator/timeseries/TimeSeriesOperatorUtils.java (1 line): - line 47: // TODO: Check isNumGroupsLimitReached, and propagate it somehow to the caller. pinot-tools/src/main/java/org/apache/pinot/tools/config/validator/TableConfigValidator.java (1 line): - line 29: // TODO: ADD MORE VALIDATIONS. pinot-core/src/main/java/org/apache/pinot/core/util/NumberUtils.java (1 line): - line 204: try { // TODO: remove toString() pinot-core/src/main/java/org/apache/pinot/core/query/utils/rewriter/ParentAggregationResultRewriter.java (1 line): - line 42: * TODO: Add support for `AS` clauses. pinot-core/src/main/java/org/apache/pinot/core/query/pruner/ColumnValueSegmentPruner.java (1 line): - line 187: // TODO: This check should be performed on the broker pinot-core/src/main/java/org/apache/pinot/core/query/scheduler/resources/PolicyBasedResourceManager.java (1 line): - line 72: // TODO: For 1 thread we should have the query run in the same queryRunner thread pinot-spi/src/main/java/org/apache/pinot/spi/stream/StreamPartitionMsgOffsetFactory.java (1 line): - line 42: * TODO: Revisit this API since offset should be immutable. pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessor.java (1 line): - line 68: // TODO: Use Schema from IndexLoadingConfig pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java (1 line): - line 546: * TODO: Revisit if index should be one-based (both Presto and Postgres use one-based index, which starts with 1) pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/readers/MultiplePinotSegmentRecordReader.java (1 line): - line 172: * TODO: add the support for multi value columns pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/BrokerCache.java (1 line): - line 55: * TODO can we introduce a SSE based controller endpoint to make the update reactive in the client? pinot-broker/src/main/java/org/apache/pinot/broker/querylog/QueryLogger.java (1 line): - line 227: // TODO: Consider adding the number of unavailable segments to the response pinot-server/src/main/java/org/apache/pinot/server/api/resources/ReingestionResource.java (1 line): - line 83: // TODO: Make them configurable pinot-spi/src/main/java/org/apache/pinot/spi/config/table/SegmentZKPropsConfig.java (1 line): - line 21: // TODO Possibly redundant class. Use SegmentZKMetadata throughout? pinot-plugins/pinot-input-format/pinot-protobuf/src/main/java/org/apache/pinot/plugin/inputformat/protobuf/ProtoBufSchemaUtils.java (1 line): - line 169: // TODO: Switch to new format after releasing 0.11.0 pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/IntegerTupleSketchAggregationFunction.java (1 line): - line 140: // TODO if extra aggregation modes are supported, make this switch pinot-core/src/main/java/org/apache/pinot/core/query/optimizer/filter/TimePredicateFilterOptimizer.java (1 line): - line 290: // TODO: Support SDF output format pinot-compatibility-verifier/src/main/java/org/apache/pinot/compat/ClusterDescriptor.java (1 line): - line 21: // TODO Support https, perhaps based on configuration pinot-controller/src/main/java/org/apache/pinot/controller/api/upload/ZKOperator.java (1 line): - line 57: * TODO: Merge it into PinotHelixResourceManager pinot-controller/src/main/java/org/apache/pinot/controller/recommender/realtime/provisioning/MemoryEstimator.java (1 line): - line 382: // TODO: better way to estimate inverted indexes memory utilization pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/ConsumerCoordinator.java (1 line): - line 77: * TODO: Revisit if we want to handle the following corner case: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/H3Utils.java (1 line): - line 78: // TODO: this can be further optimized to use native H3 implementation. They have plan to support natively. pinot-spi/src/main/java/org/apache/pinot/spi/data/readers/RecordReaderFactory.java (1 line): - line 39: // TODO: This could be removed once we have dynamic loading plugins supports. pinot-core/src/main/java/org/apache/pinot/core/query/scheduler/SchedulerPriorityQueue.java (1 line): - line 36: // TODO: throw OutOfCapacity or drop from the front of the queue ? pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotProjectJoinTransposeRule.java (1 line): - line 30: * TODO: Allow transposing project into left side of lookup join. pinot-spi/src/main/java/org/apache/pinot/spi/data/readers/RecordReader.java (1 line): - line 69: * TODO: Consider clearing the row within the record reader to simplify the caller pinot-common/src/main/java/org/apache/pinot/common/assignment/InstancePartitions.java (1 line): - line 47: *

    TODO: Support explicit partition configuration for CONSUMING instance partitions pinot-controller/src/main/java/org/apache/pinot/controller/util/ServerSegmentMetadataReader.java (1 line): - line 87: * TODO Some performance improvement ideas to explore: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ComplexTypeTransformer.java (1 line): - line 85: * TODO: support multi-dimensional array handling pinot-core/src/main/java/org/apache/pinot/core/startree/plan/StarTreeProjectPlanNode.java (1 line): - line 85: // TODO: figure out a way to close this operator, as it may hold reader context pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/channel/ChannelManager.java (1 line): - line 47: // TODO: Revisit parameters pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/src/main/java/org/apache/pinot/plugin/stream/kafka20/KafkaPartitionLevelConsumer.java (1 line): - line 93: // TODO: A better solution would be to fetch earliest offset from topic and see if it is greater than startOffset. pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/converter/stats/MutableColumnStatistics.java (1 line): - line 39: * TODO: Gather more info on the fly to avoid scanning the segment pinot-segment-local/src/main/java/org/apache/pinot/segment/local/dedup/PartitionDedupMetadataManager.java (1 line): - line 47: * TODO: As commented in PartitionUpsertMetadataManager, revisit this method and see if we can use the same pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/realtimetoofflinesegments/RealtimeToOfflineSegmentsTaskExecutor.java (1 line): - line 200: * TODO: Making the minion task update the ZK metadata is an anti-pattern, however cannot see another way to do it pinot-broker/src/main/java/org/apache/pinot/broker/broker/helix/BrokerUserDefinedMessageHandlerFactory.java (1 line): - line 124: // TODO: Fetch the table config here and pass it into the managers, or consider merging these 2 managers pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/join/LookupTable.java (1 line): - line 30: // TODO: Make it configurable pinot-connectors/pinot-spark-3-connector/src/main/scala/org/apache/pinot/connector/spark/v3/datasource/PinotDataWriter.scala (1 line): - line 198: // TODO Support file systems other than local and HDFS pinot-controller/src/main/java/org/apache/pinot/controller/util/TableSizeReader.java (1 line): - line 257: // TODO: here we assume server contains all segments in ideal state, which might not be the case pinot-core/src/main/java/org/apache/pinot/core/transport/server/routing/stats/ServerRoutingStatsManager.java (1 line): - line 249: * TODO: Explore if reads to the _serverQueryStatsMap can be done without locking. pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ExpressionTransformer.java (1 line): - line 46: * TODO: Merge this and CustomFunctionEnricher pinot-broker/src/main/java/org/apache/pinot/broker/queryquota/HelixExternalViewBasedQueryQuotaManager.java (1 line): - line 454: // TODO consider computing only the online brokers which serve the tables under the database pinot-query-runtime/src/main/java/org/apache/pinot/query/service/dispatch/timeseries/TimeSeriesDispatchObserver.java (1 line): - line 34: * TODO: This shouldn't exist and we should re-use DispatchObserver. TBD as part of multi-stage pinot-common/src/main/java/org/apache/pinot/common/config/provider/TableCache.java (1 line): - line 715: // TODO : Add expression override map for logical table, issue #15607 pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotEvaluateLiteralRule.java (1 line): - line 294: // TODO: Add more type handling pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/RebalanceTableCommand.java (1 line): - line 111: // TODO: Add pre-checks option to this command. This needs the PinotHelixResourceManager to be wired in to use pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java (1 line): - line 123: // TODO: should throw Not found exception, but need to fix how UI interpret the error pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/stats/MapColumnPreIndexStatsCollector.java (1 line): - line 188: // TODO: I've been told that we already have a function to do this, so find that function and replace this pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarConfig.java (1 line): - line 66: //TODO: find a good way to support pushing this secret to all servers. pinot-spi/src/main/java/org/apache/pinot/spi/trace/Tracer.java (1 line): - line 30: * TODO: Consider using string id or random id. Currently different broker might send query with the same request id. pinot-common/src/main/java/org/apache/pinot/common/datatable/StatMap.java (1 line): - line 60: // TODO: Study whether this is fine or we should impose a single thread policy in StatMaps pinot-core/src/main/java/org/apache/pinot/core/query/reduce/SelectionOnlyStreamingReducer.java (1 line): - line 54: // TODO: For data table map with more than one data tables, remove conflicting data tables pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/json/MutableJsonIndexImpl.java (1 line): - line 114: // TODO: Consider storing tuples as the key of the posting list so that the strings can be reused, and the hashcode pinot-controller/src/main/java/org/apache/pinot/controller/helix/SegmentStatusChecker.java (1 line): - line 210: * TODO: revisit the logic and reduce the ZK access pinot-core/src/main/java/org/apache/pinot/core/query/scheduler/resources/BinaryWorkloadResourceManager.java (1 line): - line 101: // TODO: For 1 thread we should have the query run in the same queryRunner thread pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/DistinctCountIntegerTupleSketchAggregationFunction.java (1 line): - line 36: // TODO if extra aggregation modes are supported, make this switch pinot-controller/src/main/java/org/apache/pinot/controller/api/ControllerAdminApiApplication.java (1 line): - line 61: // TODO See ControllerResponseFilter pinot-common/src/main/java/org/apache/pinot/common/datablock/DataBlockUtils.java (1 line): - line 113: // TODO: Pass in meaningful error code. pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/strategy/SegmentAssignmentStrategyFactory.java (1 line): - line 51: // TODO: Handle segment assignment strategy in future for CONSUMING segments in follow up PR pinot-core/src/main/java/org/apache/pinot/core/operator/filter/predicate/RangePredicateEvaluatorFactory.java (1 line): - line 252: // TODO: Tune this threshold pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/metadata/ColumnMetadataImpl.java (1 line): - line 237: // TODO: Use getProperty() for other properties as well to avoid the overhead of variable substitution pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/readerwriter/impl/FixedByteSingleValueMultiColumnReaderWriter.java (1 line): - line 36: * TODO: Clean this up pinot-common/src/main/java/org/apache/pinot/common/metrics/ControllerGauge.java (1 line): - line 71: // TODO: Unify below subtask metrics into a single metric with status label pinot-core/src/main/java/org/apache/pinot/core/operator/combine/BaseSingleBlockCombineOperator.java (1 line): - line 79: * TODO: Get the actual number of query worker threads instead of using the default value. pinot-clients/pinot-jdbc-client/src/main/java/org/apache/pinot/client/base/AbstractBaseResultSetMetadata.java (1 line): - line 97: //TODO: Implement API to provide tableName with BrokerResponse. pinot-core/src/main/java/org/apache/pinot/core/transport/InstanceRequestHandler.java (1 line): - line 73: // TODO: make it configurable pinot-broker/src/main/java/org/apache/pinot/broker/broker/helix/BaseBrokerStarter.java (1 line): - line 376: // TODO: decouple protocol and engine selection. pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/SegmentDeletionManager.java (1 line): - line 197: // TODO: If removing segments from deep store fails (e.g. controller crashes, deep store unavailable), these pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/retention/RetentionManager.java (1 line): - line 158: // TODO: handle the orphan segment deletion for hybrid table pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArrayFunctions.java (1 line): - line 113: // TODO: if values1.length << values2.length. Use binary search can speed up the query pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/StartControllerCommand.java (1 line): - line 66: // TODO support: pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/StartMinionCommand.java (1 line): - line 54: // TODO: support forbids = {"-minionHost", "-minionPort"} pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/AnonymizeDataCommand.java (1 line): - line 136: // TODO: Uncomment after re-implementing the SQL query generator pinot-core/src/main/java/org/apache/pinot/core/query/optimizer/filter/IdenticalPredicateFilterOptimizer.java (1 line): - line 73: * TODO: The rewrite is already happening in PredicateComparisonRewriter.updateFunctionExpression(), pinot-common/src/main/java/org/apache/pinot/common/response/PinotBrokerTimeSeriesResponse.java (1 line): - line 40: * TODO: We will evolve this until Pinot 1.3. At present we are mimicking Prometheus HTTP API. pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalanceProgressStats.java (1 line): - line 267: // TODO: Clean this up once new stats are verified pinot-common/src/main/java/org/apache/pinot/common/utils/config/TableConfigUtils.java (1 line): - line 372: * TODO: Other tier specific configs like segment assignment policy may be handled in this helper method too, to pinot-segment-local/src/main/java/org/apache/pinot/segment/local/function/FunctionEvaluatorFactory.java (1 line): - line 53: // TODO: once we have published a release w/ IngestionConfig#TransformConfigs, stop reading transform function pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/readers/PinotSegmentRecordReader.java (1 line): - line 270: // TODO: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/H3IndexFilterOperator.java (1 line): - line 68: // TODO: handle nested geography/geometry conversion functions pinot-query-planner/src/main/java/org/apache/pinot/query/planner/physical/PinotDispatchPlanner.java (1 line): - line 105: // TODO: Switch to Worker SPI to avoid multiple-places where workers are assigned. pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/RecordReaderSegmentCreationDataSource.java (1 line): - line 36: // TODO: make it Closeable so that resource in record reader can be released pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/segmentgenerationandpush/SegmentGenerationAndPushTaskExecutor.java (1 line): - line 154: // TODO: Make this use SegmentUploader pinot-core/src/main/java/org/apache/pinot/core/operator/docvalsets/RowBasedBlockValSet.java (1 line): - line 39: * TODO: Support MV pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/store/FilePerIndexDirectory.java (1 line): - line 102: // TODO: this leaks the removed data buffer (it's not going to be freed in close() method) pinot-common/src/main/java/org/apache/pinot/sql/parsers/rewriter/PredicateComparisonRewriter.java (1 line): - line 67: // TODO: Convert literals to boolean expressions pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ControllerResponseFilter.java (1 line): - line 32: // TODO Remove this class and find out how to use LoggingFeature as described in pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/executor/OpChainSchedulerService.java (1 line): - line 86: // TODO: Change the code so we ownership is expressed in the code in a better way pinot-core/src/main/java/org/apache/pinot/core/operator/blocks/results/ResultsBlockUtils.java (1 line): - line 43: // TODO: Use schema to fill the correct data types for empty results. pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java (1 line): - line 175: // TODO: The function should ideally be named 'round' pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/vector/VectorIndexType.java (1 line): - line 93: // TODO: Support more vector index types. pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/plan/MultiStageQueryStats.java (1 line): - line 521: // TODO: we can serialize with short or variable size pinot-core/src/main/java/org/apache/pinot/server/realtime/ControllerLeaderLocator.java (1 line): - line 209: // TODO: improve the exception handling. pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotFilterJoinRule.java (1 line): - line 87: // TODO - add logic to derive additional filters. E.g., from pinot-query-runtime/src/main/java/org/apache/pinot/query/service/dispatch/timeseries/AsyncQueryTimeSeriesDispatchResponse.java (1 line): - line 28: * TODO: This shouldn't exist and we should re-use AsyncQueryDispatchResponse. TBD as part of multi-stage pinot-spi/src/main/java/org/apache/pinot/spi/data/readers/GenericRow.java (1 line): - line 50: * FIXME: Based on the current behavior, we support the following data types: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java (1 line): - line 292: // TODO: Add partial result flag to RequestContext pinot-common/src/main/java/org/apache/pinot/common/minion/MinionTaskMetadataUtils.java (1 line): - line 160: // TODO: another way of finding old minion task metadata path is: (1) use reflection to find all task types, pinot-clients/pinot-jdbc-client/src/main/java/org/apache/pinot/client/base/AbstractBaseConnectionMetaData.java (1 line): - line 224: //TODO: Implement conversion mapping pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/bloom/BloomIndexType.java (1 line): - line 93: // TODO (saurabh) Check if we can do a better estimate pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/SetOperator.java (1 line): - line 129: // TODO: Consider batching the rows to improve performance. pinot-core/src/main/java/org/apache/pinot/core/query/distinct/table/DistinctTable.java (1 line): - line 32: // TODO: Tune the initial capacity pinot-common/src/main/java/org/apache/pinot/common/messages/SegmentReloadMessage.java (1 line): - line 67: // TODO: use the new List field and deprecate the partition name in next release. pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarStreamMessageMetadata.java (1 line): - line 34: // TODO: Make it a util class pinot-common/src/main/java/org/apache/pinot/common/datablock/ColumnarDataBlock.java (1 line): - line 77: // TODO: add whole-column access methods. pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/utils/BrokerSelectorUtils.java (1 line): - line 109: // TODO: This is probably unnecessary and even wrong. `brokerData` should include the fully qualified name. pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/utils/StatisticalAggregationFunctionUtils.java (1 line): - line 40: //TODO: Add MV support for covariance pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/executor/OpChainScheduler.java (1 line): - line 83: * TODO: Figure out shutdown flow in context of graceful shutdown. pinot-controller/src/main/java/org/apache/pinot/controller/recommender/data/generator/DataGenerator.java (1 line): - line 47: // TODO: add DATE_TIME to the data generator pinot-broker/src/main/java/org/apache/pinot/broker/routing/adaptiveserverselector/HybridSelector.java (1 line): - line 55: // TODO: If two or more servers have the same score, break the tie intelligently. pinot-core/src/main/java/org/apache/pinot/core/data/table/IndexedTable.java (1 line): - line 167: // TODO: Directly return final result in _tableResizer.getTopRecords to avoid extracting final result multiple times pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/RealtimeSegmentConfig.java (1 line): - line 72: // TODO: Clean up this constructor. Most of these things can be extracted from tableConfig. pinot-core/src/main/java/org/apache/pinot/core/plan/ProjectPlanNode.java (1 line): - line 84: // TODO: figure out a way to close this operator, as it may hold reader context pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java (1 line): - line 544: * TODO: Consider changing the return type to Timestamp pinot-core/src/main/java/org/apache/pinot/core/common/BlockDocIdSet.java (1 line): - line 58: // TODO: This scan is not counted in the execution stats pinot-tools/src/main/java/org/apache/pinot/tools/anonymizer/MapBasedGlobalDictionaries.java (1 line): - line 160: * TODO: May be if we can do entry.setValue() on Map.Entry, then we don't need this wrapper pinot-spi/src/main/java/org/apache/pinot/spi/config/table/ReplicaGroupStrategyConfig.java (1 line): - line 46: * TODO: use partition info from SegmentPartitionConfig pinot-tools/src/main/java/org/apache/pinot/tools/perf/PerfBenchmarkDriver.java (1 line): - line 94: // TODO: read from configuration. pinot-plugins/pinot-stream-ingestion/pinot-kafka-3.0/src/main/java/org/apache/pinot/plugin/stream/kafka30/KafkaPartitionLevelConsumer.java (1 line): - line 93: // TODO: A better solution would be to fetch earliest offset from topic and see if it is greater than startOffset. pinot-broker/src/main/java/org/apache/pinot/broker/routing/segmentpruner/TimeSegmentPruner.java (1 line): - line 170: * TODO: 1. Merge adjacent intervals pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/column/PhysicalColumnIndexContainer.java (1 line): - line 81: // TODO (index-spi): Verify that readers can be closed in any order pinot-query-planner/src/main/java/org/apache/pinot/query/type/TypeFactory.java (1 line): - line 104: // TODO: support LIST, MV column should go fall into this category. pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DataTypeConversionFunctions.java (1 line): - line 43: // TODO: Support cast for MV pinot-common/src/main/java/org/apache/pinot/common/datablock/RowDataBlock.java (1 line): - line 80: // TODO: add whole-row access methods. pinot-core/src/main/java/org/apache/pinot/core/geospatial/transform/function/StContainsFunction.java (1 line): - line 55: // TODO: to fully support Geography contains operation. pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/BaseTransformFunction.java (1 line): - line 69: // TODO: Support MV BIG_DECIMAL pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/forward/CLPMutableForwardIndexV2.java (1 line): - line 137: // TODO: Provide more optimized default values in the future pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/memory/PinotDataBuffer.java (1 line): - line 687: // TODO: Most calls to this method are just used to then read the content of the buffer. pinot-tools/src/main/java/org/apache/pinot/tools/BootstrapTableTool.java (1 line): - line 172: // TODO: it's unclear why this broke as it's been working, but using the PUT API would solve this pinot-query-planner/src/main/java/org/apache/pinot/query/planner/physical/DispatchablePlanMetadata.java (1 line): - line 60: // TODO: Change the following maps to lists pinot-tools/src/main/java/org/apache/pinot/tools/anonymizer/ArrayBasedGlobalDictionaries.java (1 line): - line 254: // TODO: make this iterative pinot-plugins/pinot-input-format/pinot-csv/src/main/java/org/apache/pinot/plugin/inputformat/csv/CSVRecordReader.java (1 line): - line 198: // TODO: This is very expensive. Consider marking the reader then reset it. The challenge here is that the reader pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroUtils.java (1 line): - line 371: // TODO: Switch to new format after releasing 0.11.0 pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/refreshsegment/RefreshSegmentTaskGenerator.java (1 line): - line 175: // TODO: See comment above - add this later if required. pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/plan/PlanNodeToOpChain.java (1 line): - line 185: // TODO: Consider adding non-equi as a separate join strategy. pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/ConnectionFactory.java (1 line): - line 265: // TODO: This code incorrectly assumes that connection properties are always the same pinot-core/src/main/java/org/apache/pinot/core/data/table/Key.java (1 line): - line 37: * TODO: Consider replacing Key with Record as the concept is very close and the implementation is the same pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/BaseBooleanAggregationFunction.java (1 line): - line 35: // TODO: change this to implement BaseSingleInputAggregationFunction when we get proper pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/funnel/FunnelCountAggregationFunction.java (1 line): - line 167: // TODO: Improve the ser/de logic pinot-core/src/main/java/org/apache/pinot/core/query/pruner/SegmentPruner.java (1 line): - line 47: * TODO: Revisit this because the caller doesn't require not changing the input segments pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/util/HelixSetupUtils.java (1 line): - line 230: // TODO: remove the logic of handling this config in both controller and server in the next official release. pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/TransformOperator.java (1 line): - line 53: // TODO: Check type matching between resultSchema and the actual result. pinot-query-planner/src/main/java/org/apache/pinot/query/planner/logical/SubPlanFragmenter.java (1 line): - line 47: * TODO: Currently it is not hooked up because we don't support multiple sub-plans yet. pinot-plugins/pinot-input-format/pinot-protobuf/src/main/java/org/apache/pinot/plugin/inputformat/protobuf/ProtoBufMessageDecoder.java (1 line): - line 38: //TODO: Add support for Schema Registry pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/plan/pipeline/PipelineBreakerExecutor.java (1 line): - line 77: // TODO: This PlanRequestContext needs to indicate it is a pre-stage opChain and only listens to pre-stage pinot-core/src/main/java/org/apache/pinot/core/query/prefetch/FetchPlanner.java (1 line): - line 31: * TODO: this interface and the dependent classes like QueryContext should be moved to query-spi pkg, yet to be added. pinot-core/src/main/java/org/apache/pinot/core/query/reduce/GapfillProcessor.java (1 line): - line 319: // TODO: may fill the default value from sql in the future. pinot-spi/src/main/java/org/apache/pinot/spi/utils/JsonUtils.java (1 line): - line 723: // TODO: Switch to new format after releasing 0.11.0 pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/PartitionUpsertMetadataManager.java (1 line): - line 72: * TODO: Revisit this and see if we can use the same IndexLoadingConfig for all segments. Tier info might be different pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotAccessControlUserRestletResource.java (1 line): - line 137: // TODO introduce a table config ctor with json string. pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/SegmentCommitterFactory.java (1 line): - line 71: // TODO: peer scheme non-null check exists for backwards compatibility. remove check once users have migrated pinot-spi/src/main/java/org/apache/pinot/spi/utils/builder/TableConfigBuilder.java (1 line): - line 78: // TODO: Remove 'DEFAULT_SEGMENT_PUSH_TYPE' in the future major release. pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/AggregationFunctionUtils.java (1 line): - line 287: // TODO: Create a short-circuit ProjectOperator when filter result is empty pinot-timeseries/pinot-timeseries-spi/src/main/java/org/apache/pinot/tsdb/spi/plan/serde/TimeSeriesPlanSerde.java (1 line): - line 36: * TODO: There are limitations to this and we will change this soon. Issues: pinot-core/src/main/java/org/apache/pinot/core/operator/BitmapDocIdSetOperator.java (1 line): - line 39: // TODO: Consider using BatchIterator to fill the document ids. Currently BatchIterator only reads bits for one pinot-segment-local/src/main/java/org/apache/pinot/segment/local/aggregator/PercentileTDigestValueAggregator.java (1 line): - line 32: // TODO: This is copied from PercentileTDigestAggregationFunction. pinot-compatibility-verifier/src/main/java/org/apache/pinot/compat/QueryOp.java (1 line): - line 38: * TODO: pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/AvgValueIntegerTupleSketchAggregationFunction.java (1 line): - line 38: // TODO if extra aggregation modes are supported, make this switch pinot-query-planner/src/main/java/org/apache/pinot/query/planner/partitioning/MultiColumnKeySelector.java (1 line): - line 58: // TODO: consider better hashing algorithms than hashCode sum, such as XOR'ing pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/plan/server/ServerPlanRequestVisitor.java (1 line): - line 167: // TODO: we should keep query stats here as well pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/groupby/DictionaryBasedGroupKeyGenerator.java (1 line): - line 150: // TODO: Clear the holder after processing the query instead of before pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/RoundDecimalTransformFunction.java (1 line): - line 32: //TODO: The function should ideally be named 'round' pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/MaxAggregationFunction.java (1 line): - line 142: // TODO: even though the source data has BIG_DECIMAL type, we still only support double precision.