add_license_url DAG is inefficient and fails due to timeout

[obulat]

Problem

The DAG runs a SELECT query for each (license, license version) pair. Each such query lasts at least 30 minutes, while the update query runs for about 2-3 minutes.

Description

Instead of running one SELECT query per license pair, it's better to select all items (identifier, license and license_version) with NULL in meta_data, and then group them by license in Python and run the corresponding UPDATE queries.

Alternatives

We could also in the first step select all items with meta_data is NULL (identifier, license and license_version). Then, add the license_url to each row using Python, and then in the next step run the update queries for each row individually:

UPDATE image
SET meta_data=Json({license_url: get_license_info_from_license_pair(license, license_version)})
WHERE identifier=identifier::UUID

I'm not sure if running so many individual queries would be faster than the current batch approach.

Additional context

Details on the run of the previous version of the DAG are here: WordPress/openverse-catalog#1005 (comment)