✨ generate csv and zip file server side

[danyx23]

This PR adds functionality to generate a CSV file on the server in a CF worker for the data of any chart. It also allows downloading a metadata.json file, a readme.md and a zip file of all three of these things.

This is in preparation of surfacing these things in the download UI of grapher in the upcoming cycle 2024.6 (#4015). This PR does not make any use of the new CF functions endpoints yet and the download UI is not yet changed.

This PR implements #3648

Testing

To test this, try http get requests against /grapher/SLUG.zip, /grapher/SLUG.csv, /grapher/SLUG.metadata.json at localhost:8788, or at the staging server linked below.

There is also an observable notebook that lets you browse the generated readme files in an easier way: https://observablehq.com/d/d410e9b2d2b7c330

[owidbot]

Quick links (staging server):

Site	Admin	Wizard

Login: ssh owid@staging-site-server-side-csv

• Site-screenshots: https://github.com/owid/site-screenshots/compare/server-side-csv
• SVG tester: https://github.com/owid/owid-grapher-svgs/compare/server-side-csv

SVG tester:

Number of differences (default views): 0 ✅
Number of differences (all views): 0 ✅

Edited: 2024-07-31 07:59:25 UTC
Execution time: 1.22 seconds

[github-actions]

This PR has had no activity within the last two weeks. It is considered stale and will be closed in 3 days if no further activity is detected.

[danyx23]

• 🧹 Restructure CF functions code into smaller files #4054 : 2 dependent PRs (#3603 , #4014 )
• Marcel's enhancements to server-side csv generation #4051
• ✨ generate csv and zip file server side #3613 👈
• ✨ Use grapher config from API instead of from HTML in multiembedder #3884 : 1 other dependent PR (#3922 )
• ✨ Render thumbnails for graphers by uuid #3880
• ✨ add fetching grapher configs by UUID #3879
• ✨ Adds consistent redirects handling for cf workers. #3895
• ✨ add CF function for getting SLUG.config.json #3870 : 1 other dependent PR (#3871 )
• 🔨 change CF thumbnail function to use chart configs from R2 #3867 : 1 other dependent PR (#3865 )
• 🔨 unify r2 config dirs #3945
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @danyx23 and the rest of your teammates on Graphite

[marcelgerber]

There are some UTF-8 issues with the generated README.
Looking into those, nothing to do there for you for now.

EDIT: Fixed in #4051.

[marcelgerber]

Nice!

I proposed some enhancements in #4051, so please have a look at that one.
Otherwise, this looks really good!

One thing I'm wondering is if in the metadata and README files, we want to note down the date of the download? I think that would be a nice idea.

[marcelgerber]

Another note for future-us:
We should extract some of this functionality out of grapherRenderer, and split it up into some separate files for

• parsing (image) options
• initializing grapher
• generating svg/png
• readme/metadata file generation
• csv file generation

[danyx23]

Merge activity

• Oct 10, 12:53 PM EDT: A user started a stack merge that includes this pull request via Graphite.
• Oct 10, 12:54 PM EDT: Graphite rebased this pull request as part of a merge.
• Oct 10, 12:55 PM EDT: A user merged this pull request with Graphite.