👌 Warn on unknown need keys in external/import sources

[chrisjsewell]

This PR centralises the declaration of what keys to omit from external/import sources, onto the NeedsCoreFields spec.
It then also uses NeedsCoreFields to identify unknown keys, and emits a warning if any are found.

Note, this could be breaking for existing builds, in that a warning may now be emitted,
but I decided not to put this under a "configuration flag", since users can simply choose to ignore it with suppress_warnings

---

One issue that was identified during this is the distinction of title vs full_title, whereby title is meant to be the potentially trimmed version of full_title.
I think storing both fields is unnecessary and confusing; trimming of the title is a presentational issue that should be handled when necessary, not stored in the data.
Also, it is of note that the need directive pre-trims the title, leading to both title and full_title being trimmed

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.88%. Comparing base (4e10030) to head (ec6784c).
Report is 93 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1316      +/-   ##
==========================================
+ Coverage   86.87%   87.88%   +1.00%     
==========================================
  Files          56       60       +4     
  Lines        6532     7066     +534     
==========================================
+ Hits         5675     6210     +535     
+ Misses        857      856       -1     

Flag	Coverage Δ
pytests	87.88% <100.00%> (+1.00%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ubmarco]

LGTM

[kreuzberger]

Hi! After upgrading from Sphinx to 4.1.0 from 3.0.0 i cannot build any more (Have Warnings treated as error on).
I ran into your case that i cannot import the needs.json files from my project (newly generated with 4.1.0) due to the warning with "Unknown keys".

The question is here for me why the keys should be unknown?

Unknown keys in import need source: ['testet', 'testet_back'] [needs.unknown_import_keys]

but these are valid "links" from

needs_extra_links = [
   {
      "option": "implements",
      "outgoing": "implementiert",
      "incoming": "ist umgesetzt durch",
   },
   {
      "option": "tests",
      "incoming": "testet",
      "outgoing": "ist getestet durch",
   }
]

[chrisjsewell]

Hey @kreuzberger, the "name" of the link you mention above comes from the option key, i.e. tests not testet; the incoming key for a link is just for adding titles (https://sphinx-needs.readthedocs.io/en/latest/configuration.html#needs-extra-links)

[kreuzberger]

Ok. than i understand that the needs.json should not export those links "incoming" and "outgoing". Can i suppress this / avoid this? Or has this to be fixed? The json file is recreated newly on the build.

[kreuzberger]

ok, just saw the configuration flags for needs export and filtering.
i will adjust them to get the warnings removed. Thanks for patience 👍