🐛 CSP Violations: Nov 1 2024 - Jan 9 2025

[petertgiles]

🐛 Bug

In preparation for activating our CSP headers (#1803) I checked our logs since Nov 1, 2024 for CSP violations. I found 21 reports. It seems like a crawler is having trouble with the robots.txt file?

KQL:

AppServiceConsoleLogs
| where ResultDescription has "CSP Violation"
| project substring(ResultDescription, 56)

🦋 Expected Behaviour

🕵️ Details

[{"age":28190,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-b44666f04258056d2172957900b4a13d'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":56337,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-bf33277c3ee4e14a904ee7a2abf85d3e'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":38923,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-01e9e955951d32165d59f0a747f2e564'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":46891,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-f5dfe1e2a254876cc0b3b5e0374430de'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":0,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-f491da55ef411144a8db38a20eae9a80'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":58445,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-535cc7fb49079d6617b587e7f33b247a'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":38153,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-2f6d03b130bc31fed493c19bca50674e'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":5536,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-03d0e5dd29a1fd560a19ee388ec6a2dc'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":59761,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-448104f42754cbdc7ab162857fd2e121'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":53577,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-fd7fc7b6e2484e0d648ac20c50b8692e'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":35805,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-52d72861a2d862725c0f7d4f3e8c8dd6'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":13538,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-51de59c904e565639a4e28df0e9d2936'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":875,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-038a0156524a0e1549cdf0f58b534b49'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":58277,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-d645783f5886d11222300f9ec5547f5a'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
{"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","body":{"documentURL":"https://talent.canada.ca/robots.txt","disposition":"report","referrer":"","effectiveDirective":"style-src-attr","blockedURL":"inline","originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-646e34ecb55b717e3f83849fbcb9311e'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","statusCode":200,"sample":"","sourceFile":"https://talent.canada.ca/robots.txt","lineNumber":1,"columnNumber":1}}
[{"age":31730,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-c476b0eb286cb8ee8aa2c24860b66d77'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":33716,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-ef8d19a481248372e1186db66cf69729'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":15386,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-f94fd57e033154b2fce4ee53ff1464ee'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":0,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-c19a61002d9f0e18ce063f4310a37e23'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":13682,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-0a909ecfb29ce28e14f7037d3ce6cfc6'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
[{"age":11261,"body":{"blockedURL":"inline","disposition":"report","documentURL":"https://talent.canada.ca/robots.txt","effectiveDirective":"style-src-attr","lineNumber":1,"originalPolicy":"default-src 'self'; font-src fonts.gstatic.com 'self'; style-src-elem fonts.googleapis.com 'self' 'unsafe-inline'; script-src-elem 'self' 'nonce-ff8ff04f8d7b70e6fa945e12de99d83c'; img-src 'self' data:; report-uri /api/csp-report/; report-to csp;","referrer":null,"sample":null,"sourceFile":"https://talent.canada.ca/robots.txt","statusCode":200},"type":"csp-violation","url":"https://talent.canada.ca/robots.txt","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/126.0.0.0 Safari/537.36"}]
**

📋 Steps to Reproduce

1. Open the robots.txt file in the Chrome browser in incognito mode

📸 Screenshot

🙋‍♀️ Proposed Solution

Stope sending CSP headers on the robots.txt files.

✅ Acceptance Criteria

• No CSP violations from opening the robots.txt files

[esizer]

Oooo, so looks liike only robots.txt. Not bad 👍