Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Complete the source url in the metadata entity #87

Open
waterflow80 opened this issue May 17, 2024 · 0 comments
Open

Complete the source url in the metadata entity #87

waterflow80 opened this issue May 17, 2024 · 0 comments

Comments

@waterflow80
Copy link
Collaborator

PR#83 is still missing the sourceUrl attribute to be filled in.

Seems like it is not straightforward, because this will require some communication between different classes to pass the source url from the Datasource class (in which we download the assembly report) all the way to the metadata class (or to the service that will populate it).

For example, we might need to change the return type of this method, in order to let it pass the sourceUrl along with the Path:

// NCBIAssemblyDataSource
@Retryable(value = Exception.class, maxAttempts = 5, backoff = @Backoff(delay = 2000, multiplier = 2))
    public Optional<Path> downloadAssemblyReport(String accession, NCBIBrowser ncbiBrowser) throws IOException {
        Optional<String> directory = ncbiBrowser.getGenomeReportDirectory(accession);
        if (!directory.isPresent()) {
            return Optional.empty();
        }
        logger.info("NCBI directory for assembly report download: " + directory.get());

        FTPFile ftpFile = ncbiBrowser.getNCBIAssemblyReportFile(directory.get());
        String ftpFilePath = directory.get() + ftpFile.getName();
        Path downloadFilePath = Paths.get(asmFileDownloadDir, ftpFile.getName());
        boolean success = ncbiBrowser.downloadFTPFile(ftpFilePath, downloadFilePath, ftpFile.getSize());
        if (success) {
            logger.info("NCBI assembly report downloaded successfully (" + ftpFile.getName() + ")");
            return Optional.of(downloadFilePath);
        } else {
            logger.error("NCBI assembly report could not be downloaded successfully(" + ftpFile.getName() + ")");
            return Optional.empty();

        }
    }

Note: we might find a better way of passing this information without the need for a refactor, maybe writing into a shared synchronized stack, queue, file, etc.. Still not sure about that.

I've made a separate issue because it might be better to fix and discuss it in a different PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant