Warn about URL scope when editing Targets

[anjackson]

In general, all URLs like http://example.com/subsection should raise a warning, as the lack of a trailing slash means the whole site is in scope: http://example.com/

Similarly, http://example.com/subsection/subsub scopes in http://example.com/subsection.

This is particularly important for large platform sites, and where open access is being inherited.

NOTE that this scoping issue applies to the URL path, and any query parameters should be dropped before determining the scope.

[min2ha]

Isn't it the case where we'll be forced to double ask every time what curator meant?
I.e. on every new target pop up dialog with 2 options about the scope: 1. whole site (ending with /) or 2. subsection (ending without /) ? And the question Are you sure?

[anjackson]

I was thinking it should accept the URL, but flash up a warning (like it does for crawl-date-in-the-past). With a message like this for each affected:

The URL http://example.com/section has no trailing slash, so the whole of https://example.com/ will be considered part of this Target. If that's not what you intended, you should edit this URL accordingly.

But maybe best to run it past @nicolabingham and see what she wants?

[nicolabingham]

@anjackson, yes, I'd like this situation to flag up a warning with the text as you suggest. Thanks.