s2 appears to slow down after 20k calls

nathimel · Jul 31, 2023 · d3cf8ee · d3cf8ee
1 parent 01baedc
commit d3cf8ee
Show file tree

Hide file tree

Showing 5 changed files with 523 additions and 342 deletions.
diff --git a/src/examples/scratch/outputs/atlas_ads-7-25-23_hafenetal/README.md b/src/examples/scratch/outputs/atlas_ads-7-25-23_hafenetal/README.md
@@ -8,6 +8,6 @@ Got locked out of ADS for too many requests after
 
 Which means
 
-- 3908 publications, 
+- 3908 publications,
 - call_size 100
 - num_pubs_max at 500
diff --git a/src/examples/scratch/outputs/atlas_s2-7-29-23_centered_imeletal/README.md b/src/examples/scratch/outputs/atlas_s2-7-29-23_centered_imeletal/README.md
@@ -0,0 +1,10 @@
+## README
+
+Started a few days after being throttled by semantic scholar.
+
+"python main.py --bibtex_fp data/Imeletal2022a.bib --atlas_dir outputs/atlas_s2-7-29-23_centered_imeletal  --centered True --max_pubs_per_expand 500 --api S2 --call_size 10"
+
+First 41 pubs are ridiculously slow.
+Finished overnight with no issues, though. 
+And second run to 20k does not seem to be too slow either.
+Trying to expand to 30k, things seem to be slowing down. Maybe getting throttled.
diff --git a/src/examples/scratch/run_topography.py b/src/examples/scratch/run_topography.py
diff --git a/src/examples/scratch/temporal.ipynb b/src/examples/scratch/temporal.ipynb
diff --git a/src/examples/scratch/topography.ipynb b/src/examples/scratch/topography.ipynb