samza-core/src/main/java/org/apache/samza/clustermanager/ContainerManager.java (5 lines): - line 119: * TODO: SAMZA-2399: Investigate & configure a timeout for container stop if needed - line 258: * TODO: SAMZA-2512 Add integ test for handleContainerStopFail - line 324: // TODO: SAMZA-2330: Handle expired request for host affinity disabled case by retying request for getting ANY_HOST - line 437: // TODO: SAMZA-2457: Allow host affinity disabled jobs to move containers to specific host - line 529: * TODO SAMZA-2480: Move logic related to onResourcesCompleted from ContainerProcessManager to ContainerManager samza-core/src/main/java/org/apache/samza/zk/ZkJobCoordinator.java (5 lines): - line 78: // TODO: MetadataCache timeout has to be 0 for the leader so that it can always have the latest information associated - line 137: // TODO: When we consolidate metadata stores for standalone, this check can be removed. For now, we expect this type. - line 353: // TODO: SAMZA-2273 - publish configs async - line 551: * TODO: SAMZA-2635: Rebalances in standalone doesn't handle DAG changes for restarted processor - line 563: * TODO: SAMZA-2645: Allow onNewJobModel as a valid state transition. Due to this limitation, we are forced samza-sql/src/main/java/org/apache/samza/sql/translator/QueryTranslator.java (3 lines): - line 94: * TODO: [SAMZA-2031]: the time-based metrics here for insert and query are - line 165: /* TODO: remove scanTime validation once code to assign it is stable */ - line 168: /** TODO: change if hasArrivalTime to validation once arrivalTime is assigned, samza-core/src/main/java/org/apache/samza/storage/blobstore/util/BlobStoreUtil.java (3 lines): - line 118: //TODO MED shesharma document error handling (checkpoint ver, blob not found, getBlob) - line 461: // TODO HIGH shesharm add integration tests to ensure empty files and directories are handled correctly E2E. - line 535: // TODO HIGH shesharm maybe use the more efficient CRC32C / PureJavaCRC32 impl samza-yarn3/src/main/java/org/apache/samza/job/yarn/YarnClusterResourceManager.java (3 lines): - line 310: * TODO: Support non-builder methods to launch resources. Maybe, refactor into a ContainerLaunchStrategy interface - line 367: //TODO: Get rid of the YarnContainer object and just use Container in state.runningProcessors hashmap. - line 415: * TODO: fix this to make stop idempotent?. samza-sql/src/main/java/org/apache/samza/sql/runner/SamzaSqlApplicationConfig.java (3 lines): - line 220: // TODO: SAMZA-2355: Make the UDFResolver pluggable. - line 247: // TODO: Get the converter factory based on the file type. Create abstraction around this. - line 255: // FIXME: the snippet below does not work for Samza SQL dsl but is required for other dsls. Future fix could be samza-core/src/main/java/org/apache/samza/clustermanager/ResourceManagerFactory.java (3 lines): - line 25: * //TODO: move the class to Samza-API? - line 32: * //TODO: Remove the SamzaAppState param and refactor into a smaller focussed class. - line 33: * //TODO: Investigate the possibility a common Samza UI for all cluster managers - Yarn,Mesos,Standalone samza-core/src/main/java/org/apache/samza/runtime/LocalApplicationRunner.java (3 lines): - line 150: // TODO: Remove restriction to only ZkJobCoordinator after next phase of metadata store abstraction. - line 310: // TODO: Add missing metadata store abstraction for creating the underlying store to address SAMZA-2182 - line 332: // TODO: This work around method is necessary due to SAMZA-2182 - Metadata store: disconnect between creation and usage of the underlying storage samza-sql/src/main/java/org/apache/samza/sql/planner/SamzaSqlValidator.java (3 lines): - line 108: // TODO: Remove this API. This API is introduced to take care of cases where RelSchemaProviders have a complex - line 123: // TODO: - line 264: // TODO: Support UDF argument validation. Currently, only return types are validated and argument types are samza-yarn/src/main/java/org/apache/samza/job/yarn/YarnClusterResourceManager.java (3 lines): - line 311: * TODO: Support non-builder methods to launch resources. Maybe, refactor into a ContainerLaunchStrategy interface - line 368: //TODO: Get rid of the YarnContainer object and just use Container in state.runningProcessors hashmap. - line 416: * TODO: fix this to make stop idempotent?. samza-sql-shell/src/main/java/org/apache/samza/sql/client/impl/SamzaExecutor.java (3 lines): - line 174: // TODO: after fixing the TODO in generateResultSchema function, we can uncomment the following piece of code. - line 282: * TODO: currently the Shell only shows some UDFs supported by Samza internally. We may need to require UDFs - line 417: // TODO: Need to find a way to convert the relational to SQL Schema. After fixing this TODO, please resolve the TODOs samza-core/src/main/java/org/apache/samza/coordinator/MetadataResourceUtil.java (2 lines): - line 34: // TODO: Replace with a metadata admin interface when the {@link MetadataStore} is fully augmented to handle all metadata sources. - line 38: private final JobModel jobModel; // TODO: Should be loaded by metadata store in the future samza-core/src/main/java/org/apache/samza/storage/blobstore/metrics/BlobStoreBackupManagerMetrics.java (2 lines): - line 36: // TODO LOW shesharm per-task throughput - line 68: // TODO shesharm LOW move to SamzaHistogram samza-yarn/src/main/java/org/apache/samza/job/yarn/YarnAppState.java (2 lines): - line 35: * TODO: make these variables private, provide thread-safe accessors. - line 124: //TODO: Make the below 3 variables immutable. Tracked as a part of SAMZA-902. Save for later. samza-sql/src/main/java/org/apache/samza/sql/translator/JoinTranslator.java (2 lines): - line 237: //TODO Not sure why we can not allow literal as part of the join condition will revisit this in another scope - line 282: //TODO We can relax this by allowing another filter to be evaluated post lookup see SAMZA-2554 samza-sql/src/main/java/org/apache/samza/sql/planner/RelSchemaConverter.java (2 lines): - line 73: // TODO: SAMZA-2345 - Change RelSchemaConverter code to apply nullability based on Sql schema. - line 98: // TODO Calcite execution engine doesn't support record type yet. samza-core/src/main/java/org/apache/samza/execution/LocalJobPlanner.java (2 lines): - line 49: * TODO: we need to consolidate this with {@link ExecutionPlanner} after SAMZA-1811. - line 100: // TODO: this works for single-job applications. For multi-job applications, ExecutionPlan should return an AppConfig samza-core/src/main/java/org/apache/samza/execution/JobPlanner.java (2 lines): - line 55: * TODO: Fix SAMZA-1811 to consolidate this class with {@link ExecutionPlanner} - line 85: // TODO: This should all be consolidated with ExecutionPlanner after fixing SAMZA-1811 samza-core/src/main/java/org/apache/samza/execution/JobNodeConfigurationGenerator.java (2 lines): - line 174: // TODO: Fix this in SAMZA-1893. For now, returning all tables for single-job execution plan - line 179: // TODO: SAMZA-1841: remove this once we support defining broadcast input stream in high-level samza-sql/src/main/java/org/apache/samza/sql/runner/SamzaSqlApplication.java (2 lines): - line 54: // TODO: Introduce an API to return a dsl string containing one or more sql statements. - line 86: * TODO When serialization of ApplicationDescriptor is actually needed, then something will need to be updated here, samza-sql/src/main/java/org/apache/samza/sql/data/SamzaSqlRelMsgMetadata.java (2 lines): - line 55: * TODO: copy eventTime through from source to RelMessage - line 62: * TODO: set arrivalTime during conversion from IME to SamzaMessage samza-core/src/main/java/org/apache/samza/runtime/RemoteApplicationRunner.java (2 lines): - line 75: // TODO SAMZA-2432: Clean this up once SAMZA-2405 is completed when legacy flow is removed. - line 164: // TODO: SAMZA-2738: Return real status for local jobs after avoiding recreating the Job in runner.status() samza-core/src/main/java/org/apache/samza/container/grouper/task/GroupByContainerCount.java (2 lines): - line 44: * TODO: SAMZA-1197 - need to modify balance to work with processorId strings - line 207: // TODO: Change logic from using int arrays to a Map (id -> taskCount) samza-core/src/main/java/org/apache/samza/clustermanager/ClusterBasedJobCoordinatorRunner.java (2 lines): - line 96: // TODO: Clean this up once SAMZA-2405 is completed when legacy flow is removed. - line 122: // TODO SAMZA-2432: Clean this up once SAMZA-2405 is completed when legacy flow is removed. samza-test/src/main/python/samza_job_yarn_deployer.py (2 lines): - line 242: # TODO we should implement the below helper methods over time, as we need them. - line 245: # TODO raise NotImplementedError samza-yarn3/src/main/java/org/apache/samza/job/yarn/YarnAppState.java (2 lines): - line 35: * TODO: make these variables private, provide thread-safe accessors. - line 124: //TODO: Make the below 3 variables immutable. Tracked as a part of SAMZA-902. Save for later. samza-core/src/main/java/org/apache/samza/execution/RemoteJobPlanner.java (2 lines): - line 39: * TODO: we need to consolidate this class with {@link ExecutionPlanner} after SAMZA-1811. - line 69: // TODO: this works for single-job applications. For multi-job applications, ExecutionPlan should return an AppConfig samza-core/src/main/java/org/apache/samza/clustermanager/SamzaApplicationState.java (2 lines): - line 34: * TODO: - line 115: * TODO: SAMZA-2607 : remove this map and all its usages. samza-core/src/main/java/org/apache/samza/zk/ZkJobCoordinatorFactory.java (2 lines): - line 44: // TODO: Separate JC related configs into a "ZkJobCoordinatorConfig" - line 49: // TODO: This should be merged with coordinatorStreamStore - SAMZA-2272 samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala (2 lines): - line 128: // TODO SAMZA-2671: there is further room for improvement for metrics wiring in general - line 771: // TODO HIGH pmaheshw SAMZA-2338: since store restore needs to trim changelog messages, samza-core/src/main/java/org/apache/samza/config/TaskConfig.java (2 lines): - line 419: // TODO remove check of standby enabled when SAMZA-2353 is completed - line 420: // TODO remove check of async commit when SAMZA-2505 is completed samza-core/src/main/java/org/apache/samza/storage/blobstore/metrics/BlobStoreRestoreManagerMetrics.java (2 lines): - line 36: // TODO LOW shesharma per-task throughput - line 57: // TODO LOW shesharma move to SamzaHistogram samza-core/src/main/java/org/apache/samza/storage/blobstore/util/DirDiffUtil.java (1 line): - line 77: // TODO HIGH shesharm add unit tests samza-core/src/main/scala/org/apache/samza/util/CoordinatorStreamUtil.scala (1 line): - line 74: // TODO: This logic should be part of the final coordinator stream metadata store abstraction. See SAMZA-2182 samza-core/src/main/scala/org/apache/samza/container/TaskInstance.scala (1 line): - line 264: // TODO BLOCKER pmaheshw reenable after demo samza-core/src/main/java/org/apache/samza/config/StorageConfig.java (1 line): - line 57: // TODO: setting replication.factor seems not working as in KafkaConfig. samza-yarn3/src/main/scala/org/apache/samza/job/yarn/YarnJob.scala (1 line): - line 178: // TODO SAMZA-2432: Clean this up once SAMZA-2405 is completed when legacy flow is removed. samza-core/src/main/java/org/apache/samza/application/descriptors/TaskApplicationDescriptorImpl.java (1 line): - line 54: // TODO: SAMZA-1841: need to add to the broadcast streams if inputDescriptor is for a broadcast stream samza-core/src/main/java/org/apache/samza/storage/StorageManagerUtil.java (1 line): - line 377: // TODO dchen use checkpoint v2 file before migrating off of dual checkpoints samza-yarn3/src/main/scala/org/apache/samza/job/yarn/YarnJobFactory.scala (1 line): - line 33: // TODO fix this. needed to support http package locations. samza-sql/src/main/java/org/apache/samza/sql/planner/QueryPlanner.java (1 line): - line 145: // TODO: Introduce a pluggable rule factory. samza-kafka/src/main/java/org/apache/samza/system/kafka/KafkaSystemAdmin.java (1 line): - line 99: // used in TestRepartitionJoinWindowApp TODO - remove SAMZA-1945 samza-core/src/main/scala/org/apache/samza/coordinator/server/HttpServer.scala (1 line): - line 98: // TODO This is where you'd add Hadoop's Kerberos security filters. samza-kv-couchbase/src/main/java/org/apache/samza/table/remote/couchbase/CouchbaseTableWriteFunction.java (1 line): - line 86: // TODO Add support for partial updates LISAMZA-21874 samza-core/src/main/java/org/apache/samza/context/InternalTaskContext.java (1 line): - line 62: * TODO: The public {@link JobContext} exposes {@link JobModel} now, so can this internal method be replaced by the samza-core/src/main/java/org/apache/samza/coordinator/staticresource/StaticResourceJobCoordinator.java (1 line): - line 217: * TODO This wouldn't be a problem if the partition count monitor used the job model to calculate initial metadata samza-core/src/main/java/org/apache/samza/task/StreamOperatorTask.java (1 line): - line 97: * TODO: Consolidate the thread pool used by OperatorImpl and StreamOperatorTask. For now, we need to keep the samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamWriter.java (1 line): - line 82: //TODO: validate keys and values samza-core/src/main/java/org/apache/samza/execution/JobNode.java (1 line): - line 168: //TODO: SAMZA-1893: should test whether the table is used in the current JobNode samza-core/src/main/java/org/apache/samza/coordinator/JobCoordinatorListener.java (1 line): - line 45: // TODO: Can change interface to ContainerModel if maxChangelogStreamPartitions can be made a part of ContainerModel samza-kv-rocksdb/src/main/java/org/apache/samza/storage/kv/RocksDbOptionsHelper.java (1 line): - line 43: // TODO HIGH pmaheshw Add these to RockdDBTableDescriptor samza-core/src/main/java/org/apache/samza/coordinator/StreamPartitionCountMonitor.java (1 line): - line 56: * TODO get the initial metadata from the job model instead of querying it, so that there isn't a chance that the samza-core/src/main/java/org/apache/samza/storage/KafkaChangelogStateBackendFactory.java (1 line): - line 159: //TODO HIGH snjain implement this samza-core/src/main/java/org/apache/samza/coordinator/JobModelCalculator.java (1 line): - line 91: // TODO: It is non-ideal to have config as a medium to transmit the locality information; especially, if the locality information evolves. Evaluate options on using context objects to pass dependent components. samza-core/src/main/java/org/apache/samza/application/descriptors/StreamApplicationDescriptorImpl.java (1 line): - line 92: // TODO: SAMZA-1841: need to add to the broadcast streams if inputDescriptor is for a broadcast stream samza-core/src/main/java/org/apache/samza/runtime/ContainerLaunchUtil.java (1 line): - line 234: // TODO: this is going away when SAMZA-1168 is done and the initialization of metrics reporters are done via samza-core/src/main/java/org/apache/samza/operators/util/InternalInMemoryStore.java (1 line): - line 41: * TODO: This class is a stop-gap until we implement persistent store creation from TaskContext. samza-core/src/main/java/org/apache/samza/clustermanager/ClusterBasedJobCoordinator.java (1 line): - line 86: * TODO: samza-sql-shell/src/main/java/org/apache/samza/sql/client/interfaces/QueryResult.java (1 line): - line 33: // TODO: after fixing the TODO in SamzaExecutor.executeQuery(), we can uncomment the following piece of code. samza-sql/src/main/java/org/apache/samza/sql/avro/AvroRelConverter.java (1 line): - line 118: // than the cached schema. [TODO: SAMZA-1679] samza-api/src/main/java/org/apache/samza/startpoint/Startpoint.java (1 line): - line 33: // TODO: Remove the @JsonTypeInfo annotation and use the ObjectMapper#enableDefaultTyping method in samza-core/src/main/java/org/apache/samza/config/ApplicationConfig.java (1 line): - line 39: * TODO: Deprecated in 0.13. After 0.13+, this id is generated using {@link org.apache.samza.runtime.ProcessorIdGenerator} samza-core/src/main/java/org/apache/samza/zk/ZkCoordinationUtils.java (1 line): - line 70: // TODO - SAMZA-1128 CoordinationService should directly depend on ZkUtils and DebounceTimer samza-tools/src/main/java/org/apache/samza/tools/json/JsonRelConverterFactory.java (1 line): - line 76: // TODO limited support right now. samza-core/src/main/java/org/apache/samza/execution/ExecutionPlanner.java (1 line): - line 65: // TODO: ExecutionPlanner needs to be able to generate single node JobGraph for low-level TaskApplication as well (SAMZA-1811) samza-core/src/main/java/org/apache/samza/storage/blobstore/BlobStoreRestoreManager.java (1 line): - line 239: // TODO HIGH shesharm what should we do with the local state already present on disk, if any? samza-sql/src/main/java/org/apache/samza/sql/dsl/SamzaSqlDslConverter.java (1 line): - line 53: // TODO: Introduce an API to parse a dsl string and return one or more sql statements samza-core/src/main/java/org/apache/samza/clustermanager/ContainerAllocator.java (1 line): - line 188: * TODO: SAMZA-2330 Handle expired request for host affinity disabled case samza-kafka/src/main/scala/org/apache/samza/checkpoint/kafka/KafkaCheckpointManager.scala (1 line): - line 184: // TODO: Remove this producer recreation logic after SAMZA-1393. samza-core/src/main/java/org/apache/samza/zk/ZkKeyBuilder.java (1 line): - line 53: * TODO: For now, it looks like $jobName-$jobId. We need to add a unique deployment/attempt identifier as well. samza-yarn3/src/main/scala/org/apache/samza/job/yarn/ClientHelper.scala (1 line): - line 153: // TODO: remove the customized approach for package resource and use the common one. samza-azure/src/main/java/org/apache/samza/coordinator/AzureJobCoordinator.java (1 line): - line 102: //TODO: Cleanup previous values in the table when barrier times out. samza-yarn/src/main/scala/org/apache/samza/job/yarn/YarnJob.scala (1 line): - line 178: // TODO SAMZA-2432: Clean this up once SAMZA-2405 is completed when legacy flow is removed. samza-yarn/src/main/scala/org/apache/samza/job/yarn/YarnJobFactory.scala (1 line): - line 33: // TODO fix this. needed to support http package locations. samza-kv-inmemory/src/main/java/org/apache/samza/storage/kv/inmemory/InMemoryKeyValueStore.java (1 line): - line 103: // TODO: Bug: SAMZA-2564: does not satisfy immutability constraint, since entrySet is backed by the underlying map. samza-core/src/main/java/org/apache/samza/storage/blobstore/BlobStoreBackupManager.java (1 line): - line 141: // TODO LOW shesharma exclude stores that are no longer configured during init samza-core/src/main/java/org/apache/samza/config/JobConfig.java (1 line): - line 248: // TODO get rid of this in a future release. samza-core/src/main/java/org/apache/samza/storage/TaskSideInputHandler.java (1 line): - line 169: // TODO: SAMZA-2255: optimize writes to side input stores samza-core/src/main/java/org/apache/samza/zk/ZkLeaderElector.java (1 line): - line 80: // TODO: This should go away once we integrate with Zk based Job Coordinator samza-sql/src/main/java/org/apache/samza/sql/runner/SamzaSqlApplicationRunner.java (1 line): - line 79: // TODO: Introduce an API to return a dsl string containing one or more sql statements samza-log4j/src/main/java/org/apache/samza/logging/log4j/StreamAppender.java (1 line): - line 280: // TODO we need the ACTUAL metrics registry, or the metrics won't get reported by the metric reporters! samza-core/src/main/java/org/apache/samza/container/RunLoopTask.java (1 line): - line 35: * TODO SAMZA-2531: isAsyncCommitEnabled is either an incomplete feature or misnamed samza-core/src/main/java/org/apache/samza/zk/ZkProcessorLatch.java (1 line): - line 49: // TODO: Verify that validatePaths doesn't fail with exceptions samza-sql/src/main/java/org/apache/samza/sql/translator/ScanTranslator.java (1 line): - line 210: * TODO This should ideally be populated by the InputTransformer in future. samza-core/src/main/java/org/apache/samza/clustermanager/SamzaResource.java (1 line): - line 36: //TODO: Investigate adding disk space. Mesos supports disk based reservations. samza-yarn/src/main/scala/org/apache/samza/job/yarn/ClientHelper.scala (1 line): - line 153: // TODO: remove the customized approach for package resource and use the common one. samza-api/src/main/java/org/apache/samza/runtime/ProcessorContext.java (1 line): - line 27: * TODO: pending change with SAMZA-1714 samza-core/src/main/java/org/apache/samza/zk/ZkBarrierForVersionUpgrade.java (1 line): - line 148: // TODO: Handle ZkNodeExistsException - SAMZA-1304 samza-core/src/main/java/org/apache/samza/startpoint/StartpointFanOutPerTask.java (1 line): - line 37: // TODO: Remove the @JsonSerialize and @JsonDeserialize annotations and use the SimpleModule#addKeySerializer and samza-core/src/main/java/org/apache/samza/clustermanager/SamzaResourceRequest.java (1 line): - line 39: * //TODO: Define a SamzaResourceRequestBuilder API as specified in SAMZA-881 samza-core/src/main/java/org/apache/samza/clustermanager/ClusterResourceManager.java (1 line): - line 68: * TODO: samza-sql/src/main/java/org/apache/samza/sql/planner/SamzaSqlFilterRemoteJoinRule.java (1 line): - line 125: // TODO - add logic to derive additional filters. E.g., from samza-core/src/main/java/org/apache/samza/standalone/PassthroughJobCoordinator.java (1 line): - line 89: // TODO metrics registry has been null here for a while; is it safe? samza-kv/src/main/scala/org/apache/samza/storage/kv/KeyValueStorageEngine.scala (1 line): - line 212: flush() // TODO HIGH pmaheshw SAMZA-2338: Need a way to flush changelog producers. This only flushes the stores.