Script to generate URLs for Ultracool Sheet. Closes #389

[kelle]

Here is a script to generate the SIMPLE URL for sources in the Ultracool Sheet.

This draft tries to do URL checking but unfortunately, any invalid URLs give status code 200. We need to figure out how to do proper URL checking.

[Will-Cooper]

can use urllib to handle strings

[kelle]

Thanks for the suggestion!

Any thoughts on how to properly test the links? Right now, even bad links return 200 because you're doing good error handling on the website.

[Will-Cooper]

Hmm, how about

results: dict = db.inventory(simple_source)
assert('Sources' in results.keys())

or that sort of logic?
Because anything that passes that will have a website page. Or are we trying to specifically ascertain that the created url will be correct?

[Will-Cooper]

Alternatively, I could edit the logic on the website to fail on a bad result and return a 404.

[kelle]

Yeah, I really want to test the URL.

Maybe we give a 404 error but include all that useful text on the 404 page. Is that possible?

[Will-Cooper]

Okie dokie I'll see what I can do

[Will-Cooper]

Returning the exact text as would appear on the 404 page on the website isn't quite as simple and not really worthwhile (would have to make custom headers etc), because of how WSGI handles things server side. It's not really the done thing to override the default status code message.
But since that PR on the website, it will return a 404 code to an API request instead of 200.

[kelle]

Even though this doesn't modify the database, I think this script is worth having. I think we will end up building more scripts related to the sheet.

[Will-Cooper]

LGTM, and I see from the email thread that you got a working csv of all matches!