-
Notifications
You must be signed in to change notification settings - Fork 69
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
arxiv: use arxiv.org urls #4353
Conversation
inspirehep/config.py
Outdated
ARXIV_PDF_URL_ALTERNATIVE = "http://arxiv.org/pdf/{arxiv_id}" | ||
ARXIV_TARBALL_URL = "http://export.arxiv.org/e-print/{arxiv_id}" | ||
ARXIV_PDF_URL = "http://arxiv.org/pdf/{arxiv_id}" | ||
ARXIV_PDF_URL_ALTERNATIVE = "http://export.arxiv.org/pdf/{arxiv_id}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is it http
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
both work.
just did some minor tests found out that https may actually be faster in their services.
Changing to https.
https ~ 5.3K speed
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 562k 100 562k 0 0 5256k 0 --:--:-- --:--:-- --:--:-- 5307k
❯ curl -X GET "https://arxiv.org/pdf/2410.20917" --output pdf
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 562k 100 562k 0 0 5251k 0 --:--:-- --:--:-- --:--:-- 5209k
❯ curl -X GET "https://arxiv.org/pdf/2410.20917" --output pdf
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 562k 100 562k 0 0 5231k 0 --:--:-- --:--:-- --:--:-- 5258k
❯ curl -X GET "https://arxiv.org/pdf/2410.20917" --output pdf
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 562k 100 562k 0 0 5035k 0 --:--:-- --:--:-- --:--:-- 5068k
❯ curl -X GET "https://arxiv.org/pdf/2410.20917" --output pdf
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 562k 100 562k 0 0 5224k 0 --:--:-- --:--:-- --:--:-- 5258k
❯ curl -X GET "https://arxiv.org/pdf/2410.20917" --output pdf
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 562k 100 562k 0 0 5277k 0 --:--:-- --:--:-- --:--:-- 5258k
❯ curl -X GET "http://arxiv.org/pdf/2410.20917" --output pdf
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
http ~ 2.7K speed
100 562k 100 562k 0 0 2758k 0 --:--:-- --:--:-- --:--:-- 2771k
❯ curl -X GET "http://arxiv.org/pdf/2410.20917" --output pdf
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 562k 100 562k 0 0 2775k 0 --:--:-- --:--:-- --:--:-- 2785k
❯ curl -X GET "http://arxiv.org/pdf/2410.20917" --output pdf
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 562k 100 562k 0 0 1787k 0 --:--:-- --:--:-- --:--:-- 1791k
❯ curl -X GET "http://arxiv.org/pdf/2410.20917" --output pdf
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 562k 100 562k 0 0 2615k 0 --:--:-- --:--:-- --:--:-- 2616k
❯ curl -X GET "http://arxiv.org/pdf/2410.20917" --output pdf
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 562k 100 562k 0 0 1114k 0 --:--:-- --:--:-- --:--:-- 1116k
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For reference, in the export route, the url without http
which we were using was also about 5x slower than if we would be using with https
:
❯ curl -X GET "https://export.arxiv.org/pdf/2410.21089" --output pdf
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1851k 100 1851k 0 0 862k 0 0:00:02 0:00:02 --:--:-- 863k
❯ curl -X GET "http://export.arxiv.org/pdf/2410.21089" --output pdf
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1851k 100 1851k 0 0 149k 0 0:00:12 0:00:12 --:--:-- 146k
cad8649
to
2b3c153
Compare
No description provided.