HTTP spec conformance during CI

[Saturn225]

/claim #3083
fixes #3083

This PR integrates new HTTP conformance tests derived from the research paper "Who's Breaking the Rules? Studying Conformance to the HTTP Specifications and its Security Impact" by Jannis Rautenstrauch and Ben Stock. These tests now acts as a guardrails to ZIO -HTTP implementations adhere to the specifications and help identify potential security issues.

Conclusions

1. The tests taken reference from http-conformance are categorised into 3 levels, Requirement, Recommendations and ABNF. The initial process is to add the conformance suite and I have added the Requirement and Recommendation level conformance tests which are critical to be tested to safeguard.

2. I have ran http-conformance tool with simple zio-http server setup and observed analysis of tool with different categories Dangerous broken, Dangerous not broken, Not dangerous broken and Not Dangerous not broken. I have shifted towards first add tests for Dangerous ones and added them broken/not-broken then added not-dangerous ones.

Changes done:

Status Codes:

This specs verifies behaviour of the different Status Codes in Violations

• 204 No Content which verifies no body is sent.
• 205 Reset Content checks no body is sent.
• 206 Partial Content checks the presence of Content-Range.
• 206 Multipart Content checks Content-Range is excluded in multipart responses.
• 206 Headers checks headers like ETag and Cache-Control are present.
• 401 Unauthorized checks the presence of WWW-Authenticate header.
• 405 Method Not Allowed checks the Allow header is present.
• 407 Proxy Authentication Required verifies the Proxy-Authenticate header is present.
• 304 Not Modified checks no body is returned for 304 Not Modified and verifies consistency with 200 OK and more....

Redirection (Location Header):

This tests added validates the presence of Location header in 300 Multiple Choices, 301 Moved Permanently, 302 Found, 303 See Other, 307 Temporary Redirect and 308 Permanent Redirect responses.

Headers and Metadata:

• Range Header (206) checks Content-Range is present in 206 responses.
• Content-Range (416) validates Content-Range in 416 Range Not Satisfiable.
• Content-Length in CONNECT checks no Content-Length for 2XX CONNECT.
• Transfer-Encoding in CONNECT checks no Transfer-Encoding for 2XX CONNECT.
• CSP Header validates that only one Content-Security-Policy header is sent.

HTTP Methods:

• HEAD Request (no body) verifies no body is sent in HEAD requests.
• GET and HEAD Headers Consistency checks for matching headers in GET and HEAD.
• POST Invalid Response Codes validates POST does not return 206, 304, or 416.
• 501 Unknown HTTP Methods validates unknown methods return 501 Not Implemented.
• 405 Blocked Methods validates non-allowed methods return 405 Method Not Allowed.

HTTP Versions:

• Transfer-Encoding with HTTP/1.0 validates no Transfer-Encoding for HTTP/1.0.
• Transfer-Encoding with HTTP/1.1 validates Transfer-Encoding for HTTP/1.1 or higher.
• Whitespace in Start-Line validates rejection of requests with improper start-line formatting.
• Bare CR in Headers checks compliance with CRLF formatting in headers.

Connection Management and Headers:

• Close in Final Response validates Connection close is included when requested.
• Bad Headers with CR, LF, or NUL validates headers with illegal characters are rejected.
• Upgrade and Expect Headers validates 100 Continue before 101 Switching Protocols when both Upgrade and Expect headers are present.

Some of other added are 426 Upgrade Required and 101 Switching Protocols

Cache-Control Directives:

This spec added checks for unquoted values for max-age and s-maxage and quoted-string form for no-cache and private.

Content-Length and Transfer-Encoding:

This spec validates no Content-Length or Transfer-Encoding in 1xx and 204 responses and also to validate consistency of Content-Length between HEAD and GET as well as between 304 Not Modified and 200 OK.

Cookie Handling:

This spec guardrails no duplicate cookie attributes in Set-Cookie headers and no duplicate cookies with the same name and also the use of IMF-fixdate format for cookie expiration dates.

Miscellaneous:

Some of miscellaneous compliance tests added checks with headers such as Server, Content-Type, Accept-Patch and Date.

Possible Changes to Discuss:

Some tests related to Content-Security-Policy-Report-Only and Strict-Transport-Security (STS) are not yet supported in ZIO-HTTP, but I have raised tickets for their future inclusion #3171 and #3172 . Once these headers are supported, we can add more tests to validate their behavior. HTTP/2 is not yet supported in ZIO-HTTP. Once it is implemented, additional tests for HTTP/2 conformance will be necessary. I have mainly focused on to add the Requirement and Recommended tests which sets initial process to add conformance guardrail for zio-http. The ABNF checks could be added gradually into the suite

Other

I will fix as much as I can of violating checks as per the conformance suite added.
Thanks @JannisBush for the detailed paper and the excellent tool!

[CLAassistant]

All committers have signed the CLA.

[algora-pbc]

💵 To receive payouts, sign up on Algora, link your Github account and connect with Stripe.

[987Nabil]

@Saturn225 HTTP/2 is out of scope. The failing test should be fixed first. I set this PR to draft for now. Once you fixed the issues open again. If you are stuck, you can contact me on discord

[Saturn225]

The failing test is that cors header is adding OPTIONS method automatically in that spec. I will need to take care of this case in toHandler. I'll fix it

[Saturn225]

The validation added in ServerInboundHandler.scala gone too aggressive for the zioHttpTestKit. Tackling it

[Saturn225]

@987Nabil Any suggestions on failing tests on ServerSpec?

[Saturn225]

@987Nabil Any guidance here?

[987Nabil]

Sorry, I am busy these days. I come back to you when I have time.

[Saturn225]

Thanks @987Nabil

[Saturn225]

@987Nabil @jdegoes Seems tricky why the entire workflows failing across the repo with following:

sbt: command not found

Any recent updates to the workflows?