integration/spark/src/main/scala/org/apache/spark/sql/optimizer/MVMatcher.scala (6 lines): - line 1132: // TODO: find out if we really need to check needRegrouping or just use myOutputList - line 1181: // TODO: implement regrouping with 1:N rejoin (rejoin tables being the "1" side) - line 1212: // TODO: implement me - line 1243: // TODO: implement me - line 1272: // TODO: refine this - line 1383: case _ => None // TODO: implement this mv/plan/src/main/scala/org/apache/carbondata/mv/plans/util/SQLBuildDSL.scala (5 lines): - line 40: // TODO: add library needed for SQL building - line 50: select: Seq[NamedExpression] = Nil, // TODO: more general, change to Seq[Either[Fragment, - line 66: // TODO: find a scheme to break up fragmentExtract() using unapply - line 240: // TODO: how to handle alias of attribute in MV - line 275: // TODO: how to handle alias of attribute in MV integration/spark/src/main/scala/org/apache/carbondata/spark/rdd/CarbonScanRDD.scala (4 lines): - line 643: // TODO: rewrite this logic to call free memory in FailureListener on failures and - line 645: // TODO: no memory leak should be there, resources should be freed on - line 880: // TODO find the better way set it. - line 885: // TODO find the better way set it. python/pycarbon/core/carbon_reader.py (4 lines): - line 84: TODO: process support - line 98: TODO: blocklet_selector - line 105: TODO: cur_shard & shard_count - line 118: TODO: experimental_reader_v2 for carbon core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java (4 lines): - line 794: * @TODO for row group only last value is incremented problem in this cases - line 1484: // TODO : should remove the unnecessary fields. - line 1942: // TODO: move this to carbon store API as it is related to TableInfo creation - line 3249: * TODO Read segment file and corresponding index file to get the correct carbondata file instead integration/spark/src/main/scala/org/apache/spark/sql/execution/command/view/CarbonCreateMVCommand.scala (3 lines): - line 267: // TODO mv table support partition - line 559: // TODO check with white list will be better. - line 606: // TODO:- Remove this case when incremental data loading is supported for multiple tables python/pycarbon/core/carbon.py (3 lines): - line 149: # TODO add mechanism to get the file path based on file filter - line 184: # TODO get record count from carbonapp based on file - line 215: # TODO Currently when projection is not added in carbon reader core/src/main/java/org/apache/carbondata/core/localdictionary/PageLevelDictionary.java (3 lines): - line 98: * @TODO Support for numeric data type dictionary exclude columns - line 105: // TODO support for actual data type dictionary ColumnSPEC - line 119: // TODO support data type specific stats collector for numeric data types python/pycarbon/core/carbon_dataset_metadata.py (3 lines): - line 117: # TODO: Simply pickling unischema will break if the UnischemaField class is changed, - line 142: # TODO add number of blocklets for each carbonfile - line 165: # TODO add pycarbon-generate-metadata.py core/src/main/java/org/apache/carbondata/core/scan/filter/resolver/ConditionalFilterResolverImpl.java (3 lines): - line 168: // TODO Auto-generated method stub - line 179: // TODO Auto-generated method stub - line 248: // TODO Auto-generated method stub integration/spark/src/main/java/org/apache/carbondata/spark/vectorreader/ColumnarVectorWrapperDirect.java (2 lines): - line 178: //TODO handle complex types - line 183: //TODO handle complex types core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/DefaultEncodingFactory.java (2 lines): - line 54: // TODO: make it configurable after added new encodingFactory - line 60: // TODO: add log integration/presto/src/main/prestodb/org/apache/carbondata/presto/impl/CarbonTableReader.java (2 lines): - line 360: // TODO : try to optimize the below loic as it may slowdown for huge splits - line 418: // TODO: Support configurable core/src/main/java/org/apache/carbondata/core/scan/expression/ColumnExpression.java (2 lines): - line 116: // TODO Auto-generated method stub - line 122: // TODO Auto-generated method stub integration/spark/src/main/scala/org/apache/carbondata/api/CarbonStore.scala (2 lines): - line 295: // TODO: move dbName and tableName to caller, caller should handle the log and error - line 331: // TODO: move dbName and tableName to caller, caller should handle the log and error integration/spark/src/main/scala/org/apache/spark/sql/catalyst/CarbonDDLSqlParser.scala (2 lines): - line 29: * TODO remove the duplicate code and add the common methods to common class. - line 68: // TODO : Add Support for CODEGEN, COST AND FORMATTED below in case of Explain query integration/flink-proxy/src/main/java/org/apache/carbon/flink/ProxyFileWriterFactory.java (2 lines): - line 60: // TODO 检查参数 - line 61: // TODO 检查是否已被注册,重复注册,直接忽略 integration/spark/src/main/scala/org/apache/spark/sql/parser/CarbonSpark2SqlParser.scala (2 lines): - line 54: * TODO remove the duplicate code and add the common methods to common class. - line 799: // TODO: Spark has started supporting CharType/VarChar types in Spark 3.1 but both are core/src/main/java/org/apache/carbondata/hadoop/CarbonInputSplit.java (2 lines): - line 143: * TODO remove this code after Index server count(*) optimization - line 406: //TODO remove this code once count(*) optimization is added in case of index server integration/presto/src/main/prestosql/org/apache/carbondata/presto/impl/CarbonTableReader.java (2 lines): - line 355: // TODO : try to optimize the below loic as it may slowdown for huge splits - line 390: // TODO: Support configurable core/src/main/java/org/apache/carbondata/core/scan/expression/logical/AndExpression.java (2 lines): - line 52: // TODO Auto-generated method stub - line 58: // TODO Auto-generated method stub integration/spark/src/main/scala/org/apache/spark/sql/execution/datasources/SparkCarbonTableFormat.scala (2 lines): - line 536: // TODO Implement write support interface to support writing Row directly to record writer - line 633: // TODO: why not use CarbonScalaUtil.convertToDateAndTimeFormats ? integration/presto/src/main/prestosql/org/apache/carbondata/presto/CarbonDataFileWriter.java (2 lines): - line 146: // TODO: need to support this, Jira CARBONDATA-4038 is created - line 196: // TODO: need to support this, Jira CARBONDATA-4038 is created core/src/main/java/org/apache/carbondata/core/scan/expression/LiteralExpression.java (2 lines): - line 52: // TODO Auto-generated method stub - line 58: // TODO Auto-generated method stub streaming/src/main/java/org/apache/carbondata/streaming/segment/StreamSegment.java (2 lines): - line 247: // TODO: handle the min max writing for string type based on character limit for streaming - line 311: // TODO better to check backup index at first integration/spark/src/main/scala/org/apache/spark/sql/index/CarbonIndexUtil.scala (2 lines): - line 182: // TODO: make it breakable - line 189: // TODO: make it breakable mv/plan/src/main/scala/org/apache/carbondata/mv/plans/util/SQLBuilder.scala (2 lines): - line 33: // TODO: generalize this using SQLBuild to handle nested sub-query expression, using the - line 171: // TODO: how to use alias to avoid duplicate names with distinct core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/ColumnPageEncoderMeta.java (2 lines): - line 163: // TODO: support stats for complex type - line 211: // TODO: support stats for complex type integration/spark/src/main/scala/org/apache/spark/sql/execution/strategy/CarbonPlanHelper.scala (2 lines): - line 65: // TODO: remove this else if check once the 2.1 version is unsupported by carbon - line 84: // TODO: remove this else check once the 2.1 version is unsupported by carbon integration/presto/src/main/java/org/apache/carbondata/presto/PrestoCarbonVectorizedRecordReader.java (2 lines): - line 172: // TODO : Implement it based on total number of rows it is going to retrieve. - line 228: // TODO: Prepare ColumnarVectorWrapper if row level filtering is needed integration/spark/src/main/scala/org/apache/spark/sql/CarbonEnv.scala (2 lines): - line 97: // TODO: move it to proper place, it should be registered by indexSchema implementation - line 209: // TODO: get create relation event integration/spark/src/main/scala/org/apache/spark/sql/hive/CarbonRelation.scala (2 lines): - line 148: // TODO: Use data from the footers. - line 149: // TODO For 2.1 integration/spark/src/main/java/org/apache/carbondata/spark/vectorreader/ColumnarVectorWrapper.java (2 lines): - line 264: //TODO handle complex types - line 269: //TODO handle complex types core/src/main/java/org/apache/carbondata/core/metadata/datatype/DecimalConverterFactory.java (2 lines): - line 119: // TODO we need to find way to directly set to vector with out conversion. This way is very - line 346: //TODO handle complex child integration/spark/src/main/scala/org/apache/spark/sql/execution/strategy/DDLHelper.scala (2 lines): - line 358: // TODO remove this limitation later - line 388: // TODO remove this limitation later integration/presto/src/main/prestosql/org/apache/carbondata/presto/CarbonDataLocationService.java (2 lines): - line 43: // TODO: test in cloud scenario in S3/OBS and make it compatible for cloud scenario - line 56: // TODO: test in cloud scenario in S3/OBS and make it compatible for cloud scenario integration/spark/src/main/scala/org/apache/spark/sql/optimizer/MVRewrite.scala (2 lines): - line 132: // TODO: support rollUp for join queries - line 814: // TODO Remove the unnecessary columns from selection. python/pycarbon/core/carbon_tf_utils.py (2 lines): - line 226: # TODO: implement a mechanism for signaling that we have no more data - line 258: # TODO: implement a mechanism for signaling that we have no more data integration/spark/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableDropColumnCommand.scala (2 lines): - line 112: // TODO: if deleted column list includes bucketed column throw an error - line 182: // TODO: 1. add check for deletion of index tables python/pycarbon/integration/tensorflow.py (2 lines): - line 222: # TODO: implement a mechanism for signaling that we have no more data - line 253: # TODO: implement a mechanism for signaling that we have no more data dev/javastyle-config.xml (2 lines): - line 80: - line 143: processing/src/main/java/org/apache/carbondata/processing/store/TablePage.java (2 lines): - line 66: // TODO: we should have separate class for key columns so that keys are stored together in - line 273: // TODO: this function should be refactoried, ColumnPage should support complex type encoding core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java (1 line): - line 457: * TODO: remove it after index metadata refactory core/src/main/java/org/apache/carbondata/core/datastore/chunk/impl/AbstractDimensionColumnPage.java (1 line): - line 97: // TODO Auto-generated method stub core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeGreaterThanFilterExecutorImpl.java (1 line): - line 262: // TODO handle min and max for null values. integration/flink-proxy/src/main/java/org/apache/carbon/flink/ProxyRecoverableOutputStream.java (1 line): - line 70: // TODO streaming结束的时候和出异常的时候都会调用该方法 core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeLessThanEqualFilterExecutorImpl.java (1 line): - line 204: // TODO handle min and max for null values. integration/spark/src/main/scala/org/apache/spark/sql/parser/CarbonSparkSqlParserUtil.scala (1 line): - line 390: // TODO: implement temporary tables integration/spark/src/main/scala/org/apache/carbondata/streamer/Source.scala (1 line): - line 197: // TODO: check without alias and remove alias integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/rdd/SecondaryIndexCreator.scala (1 line): - line 237: // TODO: use new insert into flow, instead of DataFrame prepare RDD[InternalRow] hadoop/src/main/java/org/apache/carbondata/hadoop/CarbonRecordReader.java (1 line): - line 165: // TODO : Implement it based on total number of rows it is going to retrieve. core/src/main/java/org/apache/carbondata/core/index/dev/Index.java (1 line): - line 72: // TODO Move this method to Abstract class core/src/main/java/org/apache/carbondata/core/indexstore/blockletindex/IndexWrapper.java (1 line): - line 30: * TODO it could be removed after refactor core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/ColumnPageEncoder.java (1 line): - line 186: * TODO: remove this interface after complex column page is unified with column page processing/src/main/java/org/apache/carbondata/processing/merger/CompactionResultSortProcessor.java (1 line): - line 463: // TODO: Now it is only supported onheap merge, but we can have unsafe merge processing/src/main/java/org/apache/carbondata/processing/store/writer/AbstractFactDataWriter.java (1 line): - line 317: // TODO: currently there is no implementation for notifyBlockEnd(), python/pycarbon/core/carbon_utils.py (1 line): - line 41: # TODO currenlty usinf carbon to read and write _common_metadat, need to handle in carbon core/src/main/java/org/apache/carbondata/core/scan/collector/impl/DictionaryBasedResultCollector.java (1 line): - line 216: // TODO have to fill out for dictionary columns. Once the support for push down in processing/src/main/java/org/apache/carbondata/processing/sort/sortdata/InMemorySortTempChunkHolder.java (1 line): - line 79: //TODO add code to get directly Object[] Instead Of CarbonRow Object mv/plan/src/main/scala/org/apache/carbondata/mv/plans/modular/ModularPatterns.scala (1 line): - line 45: // TODO: find out why the following stmt not working core/src/main/java/org/apache/carbondata/core/memory/IntPointerBuffer.java (1 line): - line 43: // TODO can be configurable, it is initial size and it can grow automatically. integration/spark/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonInsertIntoCommand.scala (1 line): - line 383: // TODO: For partition case, remaining projections need to validate ? processing/src/main/java/org/apache/carbondata/processing/util/CarbonDataProcessorUtil.java (1 line): - line 272: // TODO: need to simplify it. Not required create string first. core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeLessThanFilterExecutorImpl.java (1 line): - line 201: // TODO handle min and max for null values. mv/plan/src/main/scala/org/apache/carbondata/mv/plans/util/Printers.scala (1 line): - line 198: if (pos == Sort) { // TODO: how to verify type of order: && order core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentStatusManager.java (1 line): - line 141: // @TODO: move reading LoadStatus file to separate class core/src/main/java/org/apache/carbondata/core/metadata/schema/SchemaReader.java (1 line): - line 36: * TODO: It should be removed after store manager implementation. processing/src/main/java/org/apache/carbondata/processing/loading/sort/unsafe/UnsafeCarbonRowPage.java (1 line): - line 67: // TODO Only using 98% of space for safe side.May be we can have different logic. integration/spark/src/main/scala/org/apache/carbondata/spark/rdd/CarbonMergerRDD.scala (1 line): - line 185: // TODO: only in case of add and drop this variable should be true sdk/sdk/src/main/java/org/apache/carbondata/sdk/file/PaginationCarbonReader.java (1 line): - line 174: // TODO: read blocklets in multi-thread if there is a performance requirement. core/src/main/java/org/apache/carbondata/core/scan/expression/conditional/LessThanEqualToExpression.java (1 line): - line 81: // TODO Auto-generated method stub processing/src/main/java/org/apache/carbondata/processing/loading/sort/unsafe/sort/SortDataFormat.java (1 line): - line 34: // TODO: Making Buffer a real trait would be a better abstraction, but adds some complexity. hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonTableOutputFormat.java (1 line): - line 75: // TODO Move dictionary generator which is coded in spark to MR framework. integration/spark/src/main/java/org/apache/spark/sql/secondaryindex/query/SecondaryIndexQueryResultProcessor.java (1 line): - line 346: // TODO Handle for nested array and other complex types integration/spark/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala (1 line): - line 566: // TODO: move this to carbon store API core/src/main/java/org/apache/carbondata/core/util/CarbonProperties.java (1 line): - line 217: // TODO : Validation for carbon.lock.type should be handled for addProperty flow index/lucene/src/main/java/org/apache/carbondata/index/lucene/LuceneIndexFactoryBase.java (1 line): - line 141: // TODO: how to get analyzer ? python/pycarbon/reader.py (1 line): - line 54: TODO: process support index/lucene/src/main/java/org/apache/carbondata/index/lucene/LuceneIndexInputSplit.java (1 line): - line 26: // TODO: seems no one use this? format/src/main/thrift/schema.thrift (1 line): - line 80: //TODO:where to put the CSV column name and carbon table column name mapping? should not keep in schema sdk/sdk/src/main/java/org/apache/carbondata/store/LocalCarbonStore.java (1 line): - line 75: // TODO: use InputFormat to prune data and read data dev/carbon_pr.py (1 line): - line 68: # TODO Introduce a convention as this is too brittle core/src/main/java/org/apache/carbondata/core/scan/filter/FilterUtil.java (1 line): - line 578: // TODO Auto-generated method stub core/src/main/java/org/apache/carbondata/core/datastore/page/DecoderBasedFallbackEncoder.java (1 line): - line 118: //TODO It's a quick fix, we can fallback to old logic adding Length outside of integration/spark/src/main/scala/org/apache/carbondata/spark/util/CarbonScalaUtil.scala (1 line): - line 189: // TODO: decode required ? currently it is working integration/spark/src/main/scala/org/apache/spark/sql/execution/command/mutation/merge/CarbonMergeDataSetCommand.scala (1 line): - line 188: // TODO: handle the case for partial updates core/src/main/java/org/apache/carbondata/core/util/ObjectSizeCalculator.java (1 line): - line 28: * TODO: Need to have carbon implementation if carbon needs to be used without spark integration/presto/src/main/prestosql/org/apache/carbondata/presto/CarbondataPageSourceProvider.java (1 line): - line 87: // TODO: check and use dynamicFilter in CarbondataPageSource processing/src/main/java/org/apache/carbondata/processing/merger/CarbonCompactionUtil.java (1 line): - line 397: * TODO: only in case of add and drop this variable should be true mv/plan/src/main/scala/org/apache/carbondata/mv/plans/modular/Harmonizer.scala (1 line): - line 139: // TODO: the following condition is too pessimistic, more work needed using methods mv/plan/src/main/scala/org/apache/carbondata/mv/plans/util/LogicalPlanSignatureGenerator.scala (1 line): - line 60: // TODO: implement this (link to BaseRelation) integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/joins/BroadCastSIFilterPushJoin.scala (1 line): - line 515: // TODO Values in the IN filter is duplicate. replace the list with set integration/spark/src/main/scala/org/apache/spark/util/PartitionCacheManager.scala (1 line): - line 91: // TODO: Need to fix!! processing/src/main/java/org/apache/carbondata/processing/store/CarbonFactDataHandlerModel.java (1 line): - line 56: // TODO: we should try to minimize this class as refactorying loading process core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/adaptive/AdaptiveCodec.java (1 line): - line 45: // TODO: cache and reuse the same encoder since snappy is thread-safe integration/spark/src/main/java/org/apache/carbondata/spark/vectorreader/VectorizedCarbonRecordReader.java (1 line): - line 207: // TODO : Implement it based on total number of rows it is going to retrieve. mv/plan/src/main/scala/org/apache/carbondata/mv/plans/modular/Flags.scala (1 line): - line 40: // TODO: add EXPAND core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeGreaterThanEqualFilterExecutorImpl.java (1 line): - line 204: // TODO handle min and max for null values. integration/spark/src/main/scala/org/apache/spark/sql/execution/strategy/DMLStrategy.scala (1 line): - line 254: // TODO: Spark is using hints now, carbon also should use join hints core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/rle/RLECodec.java (1 line): - line 285: // TODO: add a on-the-fly decoder for filter query with high selectivity core/src/main/java/org/apache/carbondata/core/stream/StreamPruner.java (1 line): - line 103: // TODO optimize and move the code to StreamSegment , but it's in the streaming module. hadoop/src/main/java/org/apache/carbondata/hadoop/util/CarbonVectorizedRecordReader.java (1 line): - line 249: // TODO : Implement it based on total number of rows it is going to retrieve. core/src/main/java/org/apache/carbondata/core/indexstore/blockletindex/BlockletIndexFactory.java (1 line): - line 622: // TODO: pass SORT_COLUMNS into this class integration/spark/src/main/scala/org/apache/spark/sql/carbondata/execution/datasources/SparkCarbonFileFormat.scala (1 line): - line 110: // TODO find better way to know its a child integration/flink-proxy/src/main/java/org/apache/carbon/flink/ProxyRecoverableSerializer.java (1 line): - line 39: // TODO: make it configurable core/src/main/java/org/apache/carbondata/core/metadata/AbsoluteTableIdentifier.java (1 line): - line 47: //TODO this should be moved to common place where path handling will be handled processing/src/main/java/org/apache/carbondata/processing/loading/converter/impl/ComplexFieldConverterImpl.java (1 line): - line 54: // TODO Its temporary, needs refactor here. integration/spark/src/main/java/org/apache/spark/sql/secondaryindex/jobs/BlockletIndexDetailsWithSchema.java (1 line): - line 31: * TODO: When this code is moved to open source, this class can be removed and the required code processing/src/main/java/org/apache/carbondata/processing/store/CarbonDataWriterFactory.java (1 line): - line 39: // TODO Auto-generated constructor stub core/src/main/java/org/apache/carbondata/core/scan/expression/logical/BinaryLogicalExpression.java (1 line): - line 40: // TODO Auto-generated constructor stub core/src/main/java/org/apache/carbondata/core/datastore/filesystem/S3CarbonFile.java (1 line): - line 53: TODO: The current implementation of renameForce is not correct as it deletes the destination core/src/main/java/org/apache/carbondata/core/metadata/schema/table/CarbonTable.java (1 line): - line 1270: // TODO: need support get the visible status of Index without sessionInfo in the future core/src/main/java/org/apache/carbondata/core/scan/model/QueryModel.java (1 line): - line 71: * TODO need to remove this ad pass only the path mv/plan/src/main/scala/org/apache/carbondata/mv/plans/modular/AggregatePushDown.scala (1 line): - line 34: // TODO: find out if the first two case as follows really needed. Comment out for now. core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelFilterExecutorImpl.java (1 line): - line 117: // TODO: right now this is used only for Array_contains() filter, core/src/main/java/org/apache/carbondata/core/datastore/columnar/DummyBlockIndexerStorage.java (1 line): - line 22: * TODO: refactor to use 'encode by meta' without going through BlockIndexerStorage core/src/main/java/org/apache/carbondata/core/index/dev/IndexFactory.java (1 line): - line 84: * TODO: refactor to unify with IndexWriter integration/spark/src/main/scala/org/apache/carbondata/view/MVEvents.scala (1 line): - line 70: // TODO refreshed segment ids, auto or manual integration/spark/src/main/scala/org/apache/spark/sql/execution/command/management/CommonLoadUtils.scala (1 line): - line 786: // TODO: Even though copying internalRow is faster, we should avoid it core/src/main/java/org/apache/carbondata/core/scan/filter/executer/IncludeFilterExecutorImpl.java (1 line): - line 582: // TODO handle min and max for null values. integration/spark/src/main/spark3.1/org/apache/spark/sql/execution/CarbonCodegenSupport.scala (1 line): - line 25: // TODO: Spark has started supporting Codegen for Join, Carbon needs to implement the same. integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/rdd/CarbonSIRebuildRDD.scala (1 line): - line 205: // TODO: only in case of add and drop this variable should be true hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonTableInputFormat.java (1 line): - line 464: // TODO: currently only batch segment is supported, add support for streaming table integration/spark/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableRenameCommand.scala (1 line): - line 113: // TODO: Pass new Table Path in pre-event. core/src/main/java/org/apache/carbondata/core/indexstore/row/IndexRow.java (1 line): - line 26: * TODO move this class a global row and use across loading after DataType is changed class integration/spark/src/main/scala/org/apache/carbondata/view/MVRefresher.scala (1 line): - line 275: // TODO: handle in case of multiple tables load to mv table