EVA-3227 Match metadata files info with uploaded files #15
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
apriltuesday
approved these changes
Jul 15, 2024
tcezard
approved these changes
Jul 15, 2024
| System.out.printf("Directory %s listed successfully%n", submissionDirPath); | ||
| return response.getBody(); | ||
| } else { | ||
| System.out.printf("Failed to retrieve directory '%s': %s", submissionDirPath, response.getStatusCode()); |
There was a problem hiding this comment.
Should we not log rather than print out ?
There was a problem hiding this comment.
apologies copy paste mistake.. updated
Comment on lines
+114
to
+131
| String uploadedFilesInfo = globusDirectoryProvisioner.listSubmittedFiles(directoryToList); | ||
| if (uploadedFilesInfo.isEmpty()) { | ||
| throw new MetadataFileInfoMismatchException("Failed to retrieve any file info from submission directory."); | ||
| } else { | ||
| Map<String, Long> globusFileInfo = new HashMap<>(); | ||
| try { | ||
| ObjectMapper mapper = new ObjectMapper(); | ||
| ObjectNode globusFileInfoJson = (ObjectNode) mapper.readTree(uploadedFilesInfo); | ||
| if (globusFileInfoJson.get(GLOBUS_FILES_TAG) != null) { | ||
| globusFileInfo = StreamSupport.stream(globusFileInfoJson.get(GLOBUS_FILES_TAG).spliterator(), false) | ||
| .collect(Collectors.toMap( | ||
| dataNode -> dataNode.get(GLOBUS_FILE_NAME).asText(), | ||
| dataNode -> dataNode.get(GLOBUS_FILE_SIZE).asLong() | ||
| )); | ||
| } | ||
| } catch (JsonProcessingException ex) { | ||
| throw new MetadataFileInfoMismatchException("Error parsing fileInfo from Submission Directory"); | ||
| } |
There was a problem hiding this comment.
Opinion: I feel like this code should be in GlobusDirectoryProvisioner.
There was a problem hiding this comment.
IMHO this is not the core responsibility of GlobusDirectoryProvisioner. It should just be responsible for retrieving the dir/file details. The checking of missing files or file size mismatches should be done outside.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.