ste/mgr-JobPartTransferMgr.go (23 lines): - line 60: // TODO: added for debugging purpose. remove later - line 62: // TODO: added for debugging purpose. remove later - line 75: // TODO: Cleanup FailActiveUpload/FailActiveUploadWithStatus & FailActiveS2SCopy/FailActiveS2SCopyWithStatus - line 523: ft := jptm.FromTo() // TODO: consider changing isDownload (and co) to have struct receiver instead of pointer receiver, so don't need variable like this - line 539: // TODO refactor into something like jptm.IsLastModifiedTimeEqual() so that there is NO LastModifiedTime method and people therefore CAN'T do it wrong due to time zone - line 615: // TODO: review this guesstimated threshold - line 696: // TODO: Can we kill this method? - line 707: // TODO: one day it might be cleaner to simply transition the TransferStatus - line 760: // TODO: added for debugging purpose. remove later - line 766: // TODO: added for debugging purpose. remove later - line 795: // TODO: FailActive* need be further refactored with a separate workitem. - line 835: // TODO here we only act if the transfer is not yet canceled - line 855: jptm.SetErrorCode(int32(status)) // TODO: what are the rules about when this needs to be set, and doesn't need to be (e.g. for earlier failures)? - line 874: // TODO: this results in the final job output line being: Final Job Status: Cancelled - line 883: // TODO: right now the convention re cancellation seems to be that if you cancel, you MUST both call cancel AND - line 884: // TODO: ... call ReportChunkDone (with the latter being done for ALL the expected chunks). Is that maintainable? - line 885: // TODO: ... Is that really ideal, having to call ReportChunkDone for all the chunks AFTER cancellation? - line 886: // TODO: ... but it is currently necessary,because of the way the transfer is only considered done (and automatic epilogue only triggers) - line 887: // TODO: ... if all expected chunks report as done - line 928: info := jptm.Info() // TODO we are getting a lot of Info calls and its (presumably) not well-optimized. Profile that? - line 946: // TODO: Log*Error need be further refactored with a separate workitem. - line 985: // TODO: I feel like this should take the status & we kill SetStatus - line 993: // TODO: it would be nice if this protection was actually in jobPartMgr.ReportTransferDone, ste/mgr-JobMgr.go (19 lines): - line 70: // TODO: added for debugging purpose. remove later - line 72: // TODO: added for debugging purpose. remove later - line 74: // TODO: added for debugging purpose. remove later - line 118: // TODO : PartsChannelSize Needs to be discussed and can change. - line 232: // TODO: label max file buffer ram with how we obtained it (env var or default) - line 281: // TODO: added for debugging purpose. remove later - line 310: // throughput common.CountPerSecond // TODO: Set LastCheckedTime to now - line 358: // TODO: added for debugging purpose. remove later - line 364: // TODO: added for debugging purpose. remove later - line 370: // TODO: added for debugging purpose. remove later - line 402: // TODO: can we find a better way to get this info into the log? The caller is at app level, - line 538: // TODO: do we really need to to Resurrect the job twice? - line 551: // TODO: Optimize this when it's necessary for delete. - line 734: jm.chunkStatusLogger.FlushLog() // TODO: remove once we sort out what will be calling CloseLog (currently nothing) - line 780: // TODO: Add JobsAdmin reference to each JobMgr so that in any circumstances JobsAdmin should not freed, - line 947: go jm.chunkProcessor(nextWorkerId) // TODO: make sure this numbering is OK, even if we grow and shrink the pool (the id values don't matter right?) - line 970: // TODO: confirm we don't need this: expandedMonitoringInterval *= 2 - line 1052: // TODO: Question: In order to safely support high goroutine counts, - line 1073: // TODO fix preceding space sddl/sddlHelper_linux.go (13 lines): - line 45: * TODO: Do we need to support ACL_REVISION_DS (4) with support for Object ACEs? - line 511: // TODO: Validate completeness/correctness. - line 569: // TODO: Validate completeness/correctness. - line 822: // TODO: Add domain RID support. We need to prefix the domain SID. - line 895: // TODO: Allow SACL ACE type, conditional ACE Types. - line 980: // TODO: Empty object_guid;inherit_object_guid. - line 1242: // TODO: Add support for SACL. - line 1435: // TODO: Support object ACEs? - line 1511: // TODO: Add and audit SACL support. - line 1555: // TODO: Is it safer to skip/ignore the ACEs? - line 1603: // TODO: Is it safer to skip/ignore the ACEs? - line 1659: // TODO: Add support for "DACL + SACL + Owner + Group". - line 1700: // TODO: Add support for "DACL + SACL + Owner + Group". ste/xfer-remoteToLocal-file.go (12 lines): - line 71: // TODO Question: the above comment had this following text too: "and increasing the bytestransferred by the size of the source." what does it mean? - line 122: // TODO: consider cases where we might set it to true. It might give more predictable and understandable disk throughput. - line 252: // TODO: Question: do we need to Stat the file, to check its size, after explicitly making it with the desired size? - line 300: // TODO: currently, the epilogue will only run if the number of completed chunks = numChunks. - line 317: id := common.NewChunkID(info.Destination, startIndex, adjustedChunkSize) // TODO: stop using adjustedChunkSize, below, and use the size that's in the ID - line 321: // TODO: as per comment above, currently, if there's an error here we must continue because we must schedule all chunks - line 322: // TODO: ... Can we refactor/improve that? - line 438: // TODO: should we refactor to force this to accept jptm isLive as a parameter, to encourage it to be checked? - line 445: // TODO: the old version of this code did NOT consider it an error to be unable to set the modification date/time - line 446: // TODO: ...So I have preserved that behavior here. - line 447: // TODO: question: But is that correct? - line 521: if jptm.ShouldLog(common.LogInfo) { // TODO: question: can we remove these ShouldLogs? Aren't they inside Log? cmd/copy.go (12 lines): - line 127: output string // TODO: Is this unused now? replaced with param at root level? - line 232: fromTo, err := ValidateFromTo(raw.src, raw.dst, raw.fromTo) // TODO: src/dst - line 571: // TODO: Remove these warnings once service starts supporting it - line 790: // TODO: Remove the above checks where they can't be done. - line 1155: // TODO: consider find a way to enforce that, or else to allow them to be preserved. Initially, they are just for benchmark jobs, so not a problem immediately because those jobs can't be resumed, by design. - line 1217: // TODO discuss with Jeff what features should be supported by redirection, such as metadata, content-type, etc. - line 1449: // TODO: Get the entirety of remove into the new copyEnumeratorInit script so we can remove this - line 1584: // TODO: Move these into the same place for maintainability. - line 1593: // TODO: Remove this check when FileBlob w/ File OAuth works. - line 1622: // TODO merge with BlobTrash case - line 1964: // TODO check file size, max is 4.75TB - line 1972: SuggestFor: []string{"cpy", "cy", "mv"}, // TODO why does message appear twice on the console e2etest/declarativeScenario.go (12 lines): - line 58: stripTopDir bool // TODO: figure out how we'll control and use this - line 62: state scenarioState // TODO: does this really need to be a separate struct? - line 262: // TODO: handle account to account (multi-container) scenarios - line 269: // TODO: handle the multi-container (whole account) scenario - line 270: // TODO: handle wider variety of account types - line 444: // TODO: Benchmark validation will occur in new e2e test framework. For now the goal is to test that AzCopy doesn't crash. - line 448: isSrcEncoded := s.fromTo.From().IsRemote() // TODO: is this right, reviewers? - line 449: isDstEncoded := s.fromTo.To().IsRemote() // TODO: is this right, reviewers? - line 456: // TODO: testing of skipped is implicit, in that they are created at the source, but don't exist in Success or Failed lists - line 464: // TODO: how are we going to validate folder transfers???? - line 467: // TODO: for failures, consider validating the failure messages (for which we have expected values, in s.fs; but don't currently have a good way to get - line 571: // TODO: JohnR: need to remove the strip top dir prefix from the map (and normalize the delimiters) e2etest/scenario_helpers.go (11 lines): - line 21: // TODO this file was forked from the cmd package, it needs to cleaned to keep only the necessary part - line 78: // "saywut.pdf?yo=bla&WUWUWU=foo&sig=yyy", // TODO this breaks on windows, figure out a way to add it only for tests on Unix - line 142: // TODO: You'll need to set up things like attributes, and other relevant things from - line 175: // TODO: You'll need to set up things like attributes, and other relevant things from - line 637: // TODO: can we make it so that this sleeping only happens when we really need it to? - line 687: // TODO : Return ACL in list - line 885: // TODO: do we need a SetProperties method on dir...? Discuss with zezha-msft - line 888: // TODO: nakulkar-msft the attributes stuff will need to be implemented here before attributes can be tested on Azure Files - line 891: // TODO: I'm pretty sure we don't prserve lastWritetime or contentProperties (headers) for folders, so the above if statement doesn't test those - line 967: // TODO: do we want to put some random content into it? - line 1218: // TODO: Possibly add virtual-hosted-style and dual stack support. Currently use path style for testing. jobsAdmin/JobsAdmin.go (10 lines): - line 45: // TODO: can use the same struct to sort job part plan files on the basis of job number and part number - line 115: // let our CPU monitor self-calibrate BEFORE we start doing any real work TODO: remove if we switch to gopsutil - line 184: // TODO maybe one day measure actual RAM available - line 425: // TODO : sort the file. - line 458: // TODO: I think something is wrong here: I think delete and cleanup should be merged together. - line 460: // TODO: add the clean up logic for all Jobparts. - line 475: // TODO: take care of this. - line 485: // TODO: Fix jpm.planMMF.Unmap() // unmapping the memory map JobPart file - line 490: //TODO: jobHandler.shutDownHandler(ji.logger) - line 523: // TODO: review or replace (or confirm to leave as is?) Originally, JobAdmin couldn't use individual job logs because there could ste/mgr-JobPartMgr.go (9 lines): - line 45: // TODO: added for debugging purpose. remove later - line 47: // TODO: added for debugging purpose. remove later - line 105: // TODO : Default logging policy is not equivalent to old one. tracing HTTP request - line 351: // TODO: insert the factory func interface in jptm. - line 481: // TODO do we want these charset=utf-8? - line 632: // TODO: Delete file? - line 638: // TODO: added for debugging purpose. remove later - line 645: // TODO: added for debugging purpose. remove later - line 678: // TODO: Can we delete this method? ste/performanceAdvisor.go (8 lines): - line 51: "The service reported that is was busy"} // TODO: reviewers, any suggestion on better wording for this? - line 151: // TODO: mbps = stats.Mbps Is this how we'll get this? Want it to have same start time as rest of the stats... - line 217: // TODO: reviewers: any improvement on "other reasons" in this text? - line 277: // TODO: can we detect if we're in the same region? And can we do any better than that, because in many - line 314: // TODO: review whether we still need the environment variable, and adjust this message if we remove it - line 332: // TODO: consider how to factor in CPU load - will it be reflected in concurrency tuner results, or separate? - line 334: // TODO: should we also output aka.ms links to the relevant doc pages? Hard to maintain? - line 336: // TODO: do we need to add a case for files that are too small for HTBB (need to tell difference between premium block block common/chunkStatusLogger.go (8 lines): - line 53: // TODO: it's a bit odd having two pointers in a struct like this. Review, maybe we should always work - line 142: // TODO: consider change the above so that they don't create new struct on every call? Is that necessary/useful? - line 243: FlushLog() // not close, because we had issues with writes coming in after this // TODO: see if that issue still exists - line 354: continue // TODO can become break (or be moved to later if we close unsaved entries, once we figure out how we got stuff written to us after CloseLog was called) - line 359: // TODO: remove when we figure out how we got stuff written to us after CloseLog was called. For now, this should handle those cases (if they still exist) - line 458: nearZeroQueueSize = 10 // TODO: is there any intelligent way to set this threshold? It's just an arbitrary guestimate of "small" at the moment - line 494: const bigDifference = 5 // TODO: review/tune the arbitrary constant here - line 499: // TODO: find a better place for this code jobsAdmin/init.go (7 lines): - line 62: // TODO: We may want to list listen first and terminate if there is already an instance listening - line 81: // TODO: Check the HTTP verb here? - line 121: serialize(ListJobTransfers(payload), writer) // TODO: make struct - line 156: return nil // TODO: don't return (like normal main) - line 462: // TODO: added for debugging purpose. remove later (is covered by GetPerfInfo now anyway) - line 564: ErrorCode: jppt.ErrorCode()}) // TODO: Optimize - line 607: // TODO: added for debugging purpose. remove later (is covered by GetPerfInfo now anyway) cmd/sync.go (7 lines): - line 137: // TODO: if expand the set of source/dest combos supported by sync, update this method the declarative test framework: - line 276: // TODO: the check on raw.preservePermissions on the next line can be removed once we have full support for these properties in sync - line 738: // TODO: Remove this check when FileBlob w/ File OAuth works. - line 818: // TODO: enable for copy with IfSourceNewer - line 828: // TODO: enable when we support local <-> File - line 874: // TODO follow sym link is not implemented, clarify behavior first - line 877: // TODO sync does not support all BlobAttributes on the command line, this functionality should be added ste/xfer-anyToRemote-file.go (6 lines): - line 92: // // TODO: Infer availability based upon blob size as well, for premium page blobs. - line 357: // TODO: are we comfortable with this approach? - line 417: id := common.NewChunkID(srcPath, startIndex, adjustedChunkSize) // TODO: stop using adjustedChunkSize, below, and use the size that's in the ID - line 547: // TODO: should we refactor to force this to accept jptm isLive as a parameter, to encourage it to be checked? - line 580: if jptm.HoldsDestinationLock() { // TODO consider add test of jptm.IsDeadInflight here, so we can remove that from inside all the cleanup methods - line 610: if jptm.ShouldLog(common.LogInfo) { // TODO: question: can we remove these ShouldLogs? Aren't they inside Log? ste/downloader-azureFiles_linux.go (6 lines): - line 95: // TODO: Cifs client may cause the ctime to be updated. Need to think in details. - line 111: // TODO: be better at handling these errors. - line 179: // TODO: Is setting SE_DACL_PROTECTED control bit equivalent to passing - line 198: // TODO: review and potentially remove the use of the global mutex here, once we finish drilling into issues - line 206: * TODO: Why does Windows open the filehandle with InheritHandle set to 1? - line 222: // TODO: this method may become obsolete if/when we are able to get permissions from the share root cmd/removeEnumerator.go (5 lines): - line 44: // TODO: Make this merge into the other copy refactor code - line 45: // TODO: initEnumerator is significantly more verbose at this point, evaluate the impact of switching over - line 131: // TODO: this appears to be obsolete due to the above err == NothingScheduledError. Review/discuss. - line 146: // TODO move after ADLS/Blob interop goes public - line 147: // TODO this simple remove command is only here to support the scenario temporarily cmd/jobsResume.go (5 lines): - line 39: // TODO the behavior of the resume command should be double-checked - line 40: // TODO figure out how to merge resume job with copy - line 41: // TODO the progress reporting code is almost the same as the copy command, the copy-paste should be avoided - line 91: // TODO: can we combine this with the copy one (and the sync one?) - line 420: // TODO: Replace context with root context ste/downloader-azureFiles_windows.go (4 lines): - line 94: // TODO: return here on error? or ignore - line 125: // TODO: be better at handling these errors. - line 209: // TODO: review and potentially remove the use of the global mutex here, once we finish drilling into issues - line 250: // TODO: this method may become obsolete if/when we are able to get permissions from the share root ste/JobPartPlanFileName.go (4 lines): - line 35: // TODO: This needs testing - line 45: // TODO: confirm the alternative approach. fmt.Sscanf not working for reading back string into struct JobId. - line 157: // if blockSize == 0 { // TODO: Fix below - line 283: // TODO: discuss the performance drop of marshaling metadata twice cmd/zc_enumerator.go (4 lines): - line 438: // TODO (last service migration) : Remove dependency on URLs. - line 487: // TODO (last service migration) : Remove dependency on URLs. - line 602: // TODO convert to path style URL - line 667: // TODO: consider making objectMorpher an interface, not a func, and having newStoredObject take an array of them, instead of just one cmd/copyEnumeratorInit.go (4 lines): - line 104: // TODO: Fix the bug. - line 368: filters = append(filters, &IncludeFilter{patterns: cca.IncludePatterns}) // TODO should this call buildIncludeFilters? - line 468: // TODO: Reduce code dupe somehow - line 497: // TODO: Create a flag for the quota e2etest/declarativeHelpers.go (4 lines): - line 92: // TODO: if obtained or expected is a pointer, do we want to dereference it before comparing? Do we even need that in our codebase? - line 209: // TODO : Make this *actually* check with azcopy code instead of assuming azcopy's black magic. - line 268: suppressAutoFileToFile bool // TODO: invert this // if true, we won't automatically replace File -> Blob with File -> File. We do that replacement by default because File -> File is the more common scenario (and, for sync, File -> Blob is not even supported currently). - line 481: // TODO: remove this temp block e2etest/declarativeResourceManagers.go (4 lines): - line 66: // TODO: any better names for this? - line 445: shareClient *share.Client // // TODO: Either eliminate SDK URLs from ResourceManager or provide means to edit it (File SDK) for which pipeline is required - line 605: // TODO implement me - line 618: // TODO implement me common/fe-ste-models.go (4 lines): - line 558: // TODO: ELocation.GCP - line 735: // TODO: deletes are not covered by the above Is* routines - line 807: // TODO confirm whether this is actually needed - line 1256: // TODO: consider to use "[A-Za-z_]\w*" to replace this implementation, after ensuring the complexity is O(N). e2etest/newe2e_task_runazcopy_parameters.go (4 lines): - line 200: // TODO : Flags default seems to be broken; WI#26954065 - line 205: // TODO: reconsider/reengineer this flag; WI#26475473 - line 208: // TODO: handle prompting and input; WI#26475441 - line 213: // TODO: Ongoing performance profiling work cmd/cancel.go (3 lines): - line 30: // TODO should this command be removed? Previously AzCopy was supposed to have an independent backend (out of proc) - line 31: // TODO but that's not the plan anymore - line 106: // TODO remove after preview release. cmd/syncEnumerator.go (3 lines): - line 64: // TODO: enable symlink support in a future release after evaluating the implications - line 65: // TODO: Consider passing an errorChannel so that enumeration errors during sync can be conveyed to the caller. - line 86: // TODO: enable symlink support in a future release after evaluating the implications e2etest/declarativeRunner.go (3 lines): - line 111: validate Validate, // TODO: do we really want the test author to have to nominate which validation should happen? Pros: better perf of tests. Cons: they have to tell us, and if they tell us wrong test may not test what they think it tests - line 112: // _ interface{}, // TODO if we want it??, blockBlobsOnly or specific/all blob types - line 123: // TODO: do we need something here to explicitly say that we expect success or failure? For now, we are just inferring that from the elements of sourceFiles ste/sender.go (3 lines): - line 188: jptm.OccupyAConnection() // TODO: added the two operations for debugging purpose. remove later - line 219: // TODO: Log these? @JohnRusk and @zezha-msft this creates quite a bit of spam in the logs but is important info. - line 220: // TODO: Perhaps we should log it only if it isn't a block blob? ste/sender-blockBlob.go (3 lines): - line 247: // TODO: finalize and wrap in functions whether 0 is included or excluded in status comparisons - line 264: // TODO: Remove this snippet once service starts supporting CPK with blob tier - line 338: // TODO: review (one last time) should we really do this? Or should we just give better error messages on "too many uncommitted blocks" errors ste/sender-blobFolders.go (3 lines): - line 90: // TODO : Here should we undo delete "Hdi_isfolder" too? - line 190: // TODO (gapra): figure out better way to deal with hdi_isfolder metadata key capitalization - line 313: // TODO implement me ste/xferLogPolicy.go (3 lines): - line 23: // TODO: Further discuss whether to add callback into RequestLogOptions for Azure Storage Blob Go SDK. - line 24: // TODO: (new) consider also the relationship between the above comment and todos, and the new LogSanitizer - line 115: // TODO : Unravel LogOptions and RequestLogOptions e2etest/newe2e_resource_managers_blob.go (3 lines): - line 532: // TODO : Investigate bug in multistep uploader for PageBlob. (WI 28334208) - line 578: // TODO : Investigate bug in multistep uploader for AppendBlob. (WI 28334208) - line 738: //TODO implement me e2etest/declarativeTestFiles.go (3 lines): - line 44: // TODO: do we really need/want the fields here to be individually nillable? For Blob, at least, setting these is an all-or-nothing proposition anyway, so maybe there's little need for individual nillablity - line 167: adlsPermissionsACL *string // TODO: Test owner and group; needs a good target though. - line 605: // TODO: do we want to continue supporting plain strings in the expectation file lists (for convenience of test coders) cmd/pause.go (3 lines): - line 30: // TODO should this command be removed? Previously AzCopy was supposed to have an independent backend (out of proc) - line 31: // TODO but that's not the plan anymore - line 57: // TODO remove after preview release e2etest/zz_tests_to_add.go (3 lines): - line 3: // TODO soon: - line 9: // isn't there. See commented code marked TODO: nakulkar-msft - line 80: // TODO: document the following re test frameworks, and support for suites specifically: ste/JobPartPlan.go (3 lines): - line 187: // TODO: Refactor return type to an object - line 358: // ChunkCount uint16 // TODO: Remove this, we need to determine it at runtime - line 370: // TODO: ensure the length is enough cmd/credentialUtil.go (2 lines): - line 515: if name != "" && key != "" { // TODO: To remove, use for internal testing, SharedKey should not be supported from commandline - line 549: // TODO: consider replace with calls to getCredentialInfoForLocation cmd/copyUtil.go (2 lines): - line 106: // TODO: Need be replaced with anonymous embedded field technique. - line 141: // TODO: remove this, redactSigQueryParam could be added in SDK ste/pacer-autoPacer.go (2 lines): - line 53: deadBandDuration = 20 * time.Second // TODO: review this rather generous value. Might not be needed if we can pace the internal retry efforts inside the retryPolices, because we (presumably) won't get such big flurries of 503s if we do that - line 93: // TODO support an additive increase approach, if/when we use this pacer for account throughput as a whole? ste/sender-azureFile.go (2 lines): - line 93: // TODO: Should we get rid of this one? - line 488: // TODO: Remove this block once the bug is fixed. ste/sourceInfoProvider-File.go (2 lines): - line 194: //TODO: Should we do that? - line 281: // TODO: is it OK that this does not get set if s2sGetPropertiesInBackend is false? Probably yes, because it's only a cached value, and getPropertiesInBackend is always false of AzFiles anyway at present (early 2020) ste/pageRangeOptimizer.go (2 lines): - line 63: // TODO follow up with the service folks to confirm the scale at which the timeouts occur - line 64: // TODO perhaps we need to add more logic here to optimize for more cases common/rpc-models.go (2 lines): - line 224: OfStatus string // TODO: OfStatus with string type sounds not good, change it to enum - line 280: // TODO: added for debugging purpose. remove later cmd/pathUtils.go (2 lines): - line 236: // TODO: make GetAccountRoot and GetContainerName use their own specific APIs as well. It's _unlikely_ at best that the URL format will change drastically. - line 240: // TODO: Find a clever way to reduce code duplication in here. Especially the URL parsing. e2etest/declarativeWithPropertyProviders.go (2 lines): - line 89: common.PanicIfErr(err) // TODO: any better option? - line 244: // TODO: as at 1 July 2020, we are not actually validating these. Should we? It could be nice. If we don't, ste/xfer-setProperties.go (2 lines): - line 94: //TODO the canonical thing in this is changing key value to upper case. How to go around it? - line 225: // TODO : Migrate on azfile common/writeThroughFile_darwin.go (2 lines): - line 37: // TODO: currently writeThrough is ignored here on darwin. Review, and use it if we have an implementation we are comfortable with - line 49: // TODO: Need to appropriate fallocate api for darwin common/ProxyLookupCache.go (2 lines): - line 75: // TODO: should we one day find a better solution, so it can be contributed to mattn.go-ieproxy instead of done here? - line 79: // TODO: consider that one consequence of the current lack of integration with mattn.go-ieproxy is that pipelines created by common/chunkedFileWriter.go (2 lines): - line 132: const maxDesirableActiveChunks = 20 // TODO: can we find a sensible way to remove the hard-coded count threshold here? - line 363: // in some cases, e.g. Storage Spaces in Azure VMs, chopping up the writes helps perf. TODO: look into the reasons why it helps cmd/zc_traverser_blob.go (2 lines): - line 548: // TODO optimize for the case where recursive is off - line 630: // TODO log to frontend log that parallel listing was disabled, once the frontend log PR is merged common/writeThoughFile_linux.go (2 lines): - line 147: // TODO: Do we need this? - line 171: // TODO: conduct further testing of this code path, on Linux e2etest/scenario_os_helpers_for_windows.go (2 lines): - line 21: // TODO this file was forked from the cmd package, it needs to cleaned to keep only the necessary part - line 78: hi := ste.HandleInfo{ByHandleFileInformation: h} // TODO: do we want to rely on AzCopy code in tests like this? It does save a little time in test framework dev ste/downloader-azureFiles.go (2 lines): - line 70: // TODO: only day, do we really want this object to be able to exist in an uninitialized state? - line 140: // TODO : Why no enriched context here? enrichedContext := withRetryNotification(jptm.Context(), bd.filePacer) cmd/benchmark.go (2 lines): - line 260: // TODO: consider replace with regex? - line 307: // TODO: if/when we support benchmarking for S2S, note that the current code to set userAgent string in common/oauthTokenManager.go (2 lines): - line 471: // TODO: I think this method is only necessary until datalake is migrated. - line 820: // TODO: Add pipeline policy for token refreshing validating. cmd/syncIndexer.go (2 lines): - line 48: // TODO we might buffer too much data in memory, figure out whether we should limit the max number of files - line 49: // TODO previously we used 10M as the max, but it was proven to be too small for some users ste/md5Comparer.go (2 lines): - line 40: // TODO: let's add an aka.ms link to the message, that gives more info - line 46: // TODO: let's add an aka.ms link to the message, that gives more info cmd/zc_traverser_local.go (2 lines): - line 285: * TODO: Need to handle this case. - line 793: entityType, // TODO: add code path for folders ste/sender-pageBlob.go (2 lines): - line 182: // so we know that the pacer will be closed. // TODO: consider re-factor xfer-anyToRemote so that epilogue is always called if uploader is constructed, and move this to constructor - line 241: // TODO: Remove this snippet once service starts supporting CPK with blob tier e2etest/validator.go (2 lines): - line 89: // TODO: Think of how to validate files in case of remove - line 187: // TODO: do we also want to output specific filenames for ones that were expected to have that status, but did not get it? e2etest/runner.go (2 lines): - line 98: // TODO: TODO: nakulkar-msft there will be many more to add here - line 229: _, _ = stdin.Write([]byte(msgToApp + "\n")) // TODO: maybe change this to use chToStdIn ste/concurrency.go (2 lines): - line 123: // TODO: consider whether we should also use this (renamed to( MaxOpenFiles) for uploads, somehow (see command above). Is there any actual value in that? Maybe only highly handle-constrained Linux environments? - line 217: maxValue = 3000 // TODO: what should this be? Testing indicates that this value is all we're ever likely to need, even in small-files cases common/cpuMonitor.go (1 line): - line 33: // TODO: consider replacing this file with %age CPU usage measurement from gopsutil (bearing in mind that it would be a little harder to test, e2etest/newe2e_resource_manager_getter.go (1 line): - line 36: a.Error(fmt.Sprintf("TODO: Location %s is not yet supported", location)) common/singleChunkReader.go (1 line): - line 422: * TODO: We should not need to explicitly set this to nil but today we have a yet-unknown ref on cr which sddl/parseSddl.go (1 line): - line 30: var translateSID = OSTranslateSID // this layer of indirection is to support unit testing. TODO: it's ugly to set a global to test. Do something better one day ste/ErrorExt.go (1 line): - line 13: // TODO: consider rolling MSRequestID into this, so that all places that use this can pick up, and log, the request ID too ste/pacer-tokenBucketPacer.go (1 line): - line 64: maxSecondsToOverpopulateBucket = 2.5 // had 5, when doing coarse-grained pacing. TODO: find best all-round value, or parameterize e2etest/scenario_os_helpers.go (1 line): - line 23: // TODO this file was forked from the cmd package, it needs to cleaned to keep only the necessary part common/genericResourceURLParts.go (1 line): - line 12: // TODO: Use this to reduce code dupe in the cca.Source and jobPartOrder.Source setups common/iff.go (1 line): - line 23: // TODO : Remove this? ste/sender-blockBlobFromURL.go (1 line): - line 146: // TODO: Remove this snippet once service starts supporting CPK with blob tier ste/concurrencyTuner.go (1 line): - line 199: // TODO: review this, and look for root cause/better solution common/randomDataGenerator.go (1 line): - line 131: // TODO: add unit tests to assert the lack of compressibility (since for now we are just going ste/sourceInfoProvider-Local_linux.go (1 line): - line 29: // AT_SYMLINK_NOFOLLOW is not used, because we automagically resolve symlinks. TODO: Add option to not follow symlinks, and use AT_SYMLINK_NOFOLLOW when resolving is disabled. ste/putListNeed.go (1 line): - line 31: // TODO: do we want to keep using atomic, just to be sure this is safe no matter how or where its used? testSuite/cmd/testfile.go (1 line): - line 94: // TODO: parameter name doesn't match file scenario, discuss and refactor. common/folderDeletionManager.go (1 line): - line 63: // TODO: do we want this to report, so that we can log, any folders at the very end which still are not deleted? ste/sender-blockBlobFromLocal.go (1 line): - line 137: // TODO: Remove this snippet once service starts supporting CPK with blob tier cmd/helpMessages.go (1 line): - line 504: // TODO: document whether we delete the uploaded data cmd/copyEnumeratorHelper.go (1 line): - line 59: // TODO this should probably be removed after the high throughput block blob feature is implemented on the service side common/credCache_linux.go (1 line): - line 117: // TODO: better logging what's cause for token caching failure cmd/removeProcessor.go (1 line): - line 35: SourceRoot: cca.Source.CloneWithConsolidatedSeparators(), // TODO: why do we consolidate here, but not in "copy"? Is it needed in both places or neither? Or is copy just covering the same need differently? ste/xfer-deleteFile.go (1 line): - line 81: // TODO: doing this only on success raises the possibility of the ste/downloader-blobFS_linux.go (1 line): - line 18: // TODO: Consolidate and reduce duplication later e2etest/scenario_os_helpers_for_linux.go (1 line): - line 80: // AT_SYMLINK_NOFOLLOW is not used, because we automagically resolve symlinks. TODO: Add option to not follow symlinks, and use AT_SYMLINK_NOFOLLOW when resolving is disabled. ste/xfer.go (1 line): - line 44: // TODO: consider to unify the retry options. common/output.go (1 line): - line 24: // TODO: if/when we review the STE structure, with regard to the old out-of-process design vs the current in-process design, we should cmd/root.go (1 line): - line 200: preferToAutoTuneGRs := cmd == benchCmd // TODO: do we have a better way to do this than making benchCmd global? ste/sender-azureFileFromURL.go (1 line): - line 55: // TODO consider optimizations for sparse files cmd/zc_processor.go (1 line): - line 279: // TODO: If we ever do launch errors outside of the final "no transfers" error, make them output nicer things here. common/writeThoughFile_windows.go (1 line): - line 103: // TODO: is that acceptable? Seems overkill to re-instate the attribute if the open fails.... cmd/make.go (1 line): - line 134: // TODO : Ensure it is a container URL here and fail early? common/s3URLParts.go (1 line): - line 55: // TODO: Other S3 compatible service which might be with IP endpoint style e2etest/helpers.go (1 line): - line 21: // TODO this file was forked from the cmd package, it needs to cleaned to keep only the necessary part common/logger.go (1 line): - line 145: const TryEquals string = "Try=" // TODO: refactor so that this can be used by the retry policies too? So that when you search the logs for Try= you are guaranteed to find both types of retry (i.e. request send retries, and body read retries) cmd/zc_filter.go (1 line): - line 245: // TODO: consider whether we need to support some way to separately invoke prefix-based optimization common/osOpen_windows.go (1 line): - line 29: // TODO: os.Stat(name string) already uses backup semantics. Can/should we just use that? ste/sender-appendBlob.go (1 line): - line 191: // TODO: particularly, given that this is an APPEND blob, do we really need to delete it? But if we don't delete it, common/cacheLimiter.go (1 line): - line 39: // TODO: now that cacheLimiter is used for multiple purposes, the hard-coding of the distinction between common/exclusiveStringMap.go (1 line): - line 86: // TODO: double check, is it really? Should we replace by going through rune-by-rune converting to unicode "fold case"? common/mmf_darwin.go (1 line): - line 57: //TODO: Prefetch api for darwin x64 is different than the one for linux. main_windows.go (1 line): - line 36: // TODO: this api is implemented for windows as well but not required because Windows common/parallel/FileSystemCrawler.go (1 line): - line 80: // TODO: add at a test that CrawlLocalDirectory does NOT include the root (i.e. add test to define that behaviour) ste/xferStatsPolicy.go (1 line): - line 188: // TODO should we also count status 500? It is mentioned here as timeout:https://docs.microsoft.com/en-us/azure/storage/common/storage-scalability-targets ste/downloader-blob.go (1 line): - line 162: // TODO (gapra) : This can be removed after Access Conditions fix is released. common/parallel/dirReader.go (1 line): - line 108: return nil, err // TODO: is it correct to assume that if err has a value, then names is empty? cmd/syncProcessor.go (1 line): - line 296: // TODO: use b.targetLocation.String() in the next line, instead of "object", if we can make it come out as string common/parallel/Transformer.go (1 line): - line 32: input <-chan CrawlResult // TODO: would have liked this to be of InputObject, but it made our usage messy. Not sure of right solution to that yet. For now, using CrawlResult ties us to transforming result of crawl common/LongPathHandler.go (1 line): - line 45: PanicIfErr(err) //TODO: Handle errors better? common/lifecyleMgr.go (1 line): - line 352: // TODO minor: consider merging with Exit