ActionController::UrlGenerationError exceptions in Whitehall

[floehopper]

I suspect these exceptions in the Whitehall app might be related to alphagov/whitehall#3627.

[floehopper]

Prior to alphagov/whitehall#3627, a request for this URL would've generated a 404 Not Found response from PublicUploadsController#fail, because no file with the specified path/filename exists in either the clean or incoming directories.

The URL itself is somewhat mysterious, because it contains Unicode non-breaking spaces (%C2A0) rather than underscore characters and a newline character (%0A) in the filename. This makes me suspicious that the URL was manually generated, perhaps by being copied and pasted from a document which resulted in the unhelpful characters. Unfortunately the HTTP_REFERER request header is not set and so it's hard to work out the origin of the request.

The ActionController::UrlGenerationError exception is the result of the following:

• The URL was almost certainly intended to be handled by AttachmentsController#preview, but because the attachment_data path segment contains a non-breaking space instead of an underscore, the request falls through to the more generic PublicUploadsController#show handler. Note that this would've been occurring prior to Redirect asset requests to asset host whitehall#3627.

• The redirect in PublicUploadsController#redirect_to_asset_host relies on an implicit extraction of the path in order to construct the redirect URL via ActionDispatch::Routing#url_for. However, this (somewhat understandably) can't cope with the newline character in the path and raises the slightly misleading exception. Prior to Redirect asset requests to asset host whitehall#3627, the redirect_to_asset_host before action did not exist and PublicUploadsController#show would've responded with a 404 Not Found as described above.

I'm not entirely sure what to do about this problem, but I am confident that it's not a serious issue because of the following:

• The URL was responding with a 404 Not Found previously. While responding with a 500 Server Error is obviously not ideal, it's not really going to make the experience of the end user any worse.
• The AttachmentData with the ID included within the URL relates to 3 Publications all of which are in the superseded state and the page with the slug matching these publications now seems to point at a different AttachmentData for which the CSV preview works.
• The content of the CSV file stored under the AttachmentData ID in the URL seems to match that of this new CSV file.

Thus I really don't think this is a serious problem and am happy to leave it until I can discuss it with @chrisroos & @chrislo on their return from holiday.

[floehopper]

This exception has just happened a bunch more times...

Staging

• https://sentry.io/govuk/app-whitehall/issues/427475207/
• https://sentry.io/govuk/app-whitehall/issues/428402938/

Production

• https://sentry.io/govuk/app-whitehall/issues/427475207/

They all look as if they relate to requests with a newline in the asset path. I wonder if there's some systematic problem - could Govspeak be accidentally generating URL paths including newlines?

@chrislo, @chrisroos: Do you have any thoughts on this?

[floehopper]

I'm no longer actively investigating this, so I'm going to un-assign myself.

[floehopper]

This has happened again: https://sentry.io/govuk/app-whitehall/issues/434107507/

[chrisroos]

We've seen some more of these exceptions in the past couple of days. Searching Sentry for ActionController::UrlGenerationError appears to show all of these exceptions.

The following paths are all resulting in these exceptions:

• https://www.gov.uk/government/uploads/system/uploads/attachment%C2%A0data/file/651730/Minister%0Aial%C2%A0Returns%C2%A0-%C2%A0April%C2%A0to%C2%A0June%C2%A02017%C2%A0-%C2%A0Meetings.csv/preview
• https://www.gov.uk/government/uploads/system/uploads/attachment_data%0A/file/613477/project_delivery_capability_framework.pdf
• https://www.gov.uk/government/uploads/system/uploads/attachment_data/%0Afile/287717/11-1116-migration-and-global-environmental-change.pdf
• https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/82727/mcz-annex-a5-121213.pdf%0A%0Ahttp:/www.wildlifetrusts.org/MCZ/otter-estuary

As @floehopper has already suggested, my best guess is that someone has pasted these links into a document and then accidentally inserted newlines (%0A) in the URLs.