[Bug] improve HTML Canonical Link behavior

[TimidRobot]

Description

Simplistic HTML Canonical Link support was added in #237. However it wrongly always uses the legal tool's Legacy Canonical URL.

Examples

Label	URL
Legacy Canonical URL:	https://creativecommons.org/licenses/by/4.0/
HTML Canonical Link:	https://creativecommons.org/licenses/by/4.0/deed.en
Legacy Canonical URL:	https://creativecommons.org/publicdomain/zero/1.0/
HTML Canonical Link:	https://creativecommons.org/publicdomain/zero/1.0/deed.en

Expectation

The HTML Canonical Link should point to appropriate language-specific Deed or Legal Code page without the .html extension.

• We are dropping the legacy canonical URL as it was language specific (English) for unported legal tools
• We can not maintain both a Legacy Canonical URL and a HTML Canonical Link for a given tool as the former will degrade the latter
  Consolidate Duplicate URLs with Canonicals | Google Search Central | Google Developers:

  Link to the canonical URL rather than a duplicate URL, when linking within your site. Linking consistently to the URL that you consider to be canonical helps Google understand your preference

• Having each published page in creativecommons/cc-legal-tools-data contain a HTML Canonical Link will help ensure development and staging environments do not come up in search results
• The HTML Canonical Link must be a Full URL (related: Ensure links do not downgrade security or break continuity (use path-only or https://) · Issue #65 · creativecommons/cc-legal-tools-app)

Page Examples

• HTML Canonical Link: https://creativecommons.org/licenses/by/4.0/deed.en

  • /licenses/by/4.0/ (Legacy Canonical URL, HTTP server configured to use default index page: index.html)
  • /licenses/by/4.0/deed (HTTP server configured to serve content without .html file extension)
  • /licenses/by/4.0/deed.html (symlink to default language: deed.en.html)
  • /licenses/by/4.0/deed.en (HTTP server configured to serve content without .html file extension)
  • /licenses/by/4.0/deed.en.html
  • /licenses/by/4.0/index.html (symlink to default language: deed.en.html)

• HTML Canonical Link: https://creativecommons.org/licenses/by/4.0/deed.nl

  • /licenses/by/4.0/deed.nl (HTTP server configured to serve content without .html file extension)
  • /licenses/by/4.0/deed.nl.html

• HTML Canonical Link: https://creativecommons.org/licenses/by/4.0/legalcode.en

  • /licenses/by/4.0/legalcode (HTTP server configured to serve content without .html file extension)
  • */licenses/by/4.0/legalcode.html (symlink to default language: legalcode.en.html)
  • /licenses/by/4.0/legalcode.en (HTTP server configured to serve content without .html file extension)
  • /licenses/by/4.0/legalcode.en.html

• HTML Canonical Link: https://creativecommons.org/licenses/by/4.0/legalcode.nl

  • /licenses/by/4.0/legalcode.nl (HTTP server configured to serve content without .html file extension)
  • /licenses/by/4.0/legalcode.nl.html

HTML Example

• HTML Canonical Link: https://creativecommons.org/licenses/by/4.0/deed.en

  <link href="https://creativecommons.org/licenses/by/4.0/deed.en" rel="canonical"/>

Change Approval

Dropping the shorter English-favoring Legacy Canonical URL in favor of the HTML Canonical Link was discussed with and approved by the CC Legal Team.

Additional context

• HTML metadata clean-up (app) by TimidRobot · Pull Request #237 · creativecommons/cc-legal-tools-app
• Canonical link element - Wikipedia
• Consolidate Duplicate URLs with Canonicals | Google Search Central | Google Developers
• <link>: The External Resource Link element - HTML: HyperText Markup Language | MDN