feat(maven): Remove unnecessary HTML page fetches #32662
Conversation
zharinov
requested review from
rarkins and
viceice
and removed request for
rarkins
|
Currently, for every package we're fetching:
If first 3 are done correctly, seems like we don't need the latter one. |
There was a problem hiding this comment.
Currently, for every package we're fetching:
* `maven-metadata.xml` * GET the latest POM file (for `homepage`/`sourceUrl`) * HEAD the POM file of the selected/filtered release (often matches with GET of the latest one) * [Maven Central only] HTML index pageIf first 3 are done correctly, seems like we don't need the latter one.
If I'm not mistaken "GET the latest POM file (for homepage/sourceUrl)" is done after populating the list of releases, i.e.., any release that is unknown at this point won't be pulled. Hence, the "[Maven Central only] HTML index page" step is actually second, not last. Do you agree?
| let releaseMap = await this.fetchReleasesFromMetadata(dependency, repoUrl); | ||
| releaseMap = await this.addReleasesFromIndexPage( | ||
| releaseMap, | ||
| const releaseMap = await this.fetchReleasesFromMetadata( |
There was a problem hiding this comment.
I assume this could have an impact for those packages on Maven Central which have no maven-metadata.xml? See https://maven.apache.org/repository/central-metadata.html
To alleviate such cases, would it work to addReleasesFromIndexPage only if releaseMap is still empty after fetchReleasesFromMetadata instead of doing it always? It would also mean one index.html query for packages that don't exist on Maven Central.
There was a problem hiding this comment.
Yeah, but we recently discovered it didn't work for a while, and no one suffered anyways except the fact it always produced 404s
There was a problem hiding this comment.
And order is different, just like you described, yes
Changes
It doesn't seem necessary now when we're able to obtain all necessary information by querying the corresponding POM file of a particular package version.
Context
Documentation (please check one with an [x])
How I've tested my work (please select one)
I have verified these changes via: