-
Notifications
You must be signed in to change notification settings - Fork 65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore(deps): update dependency aiohttp to v3.9.0 [security] #1278
Conversation
50f24c9
to
7a6db57
Compare
@@ -1,4 +1,4 @@ | |||
aiohttp==3.8.5 | |||
aiohttp==3.9.0 |
Check failure
Code scanning / Trivy
LangChain vulnerable to arbitrary code execution Critical
@@ -1,4 +1,4 @@ | |||
aiohttp==3.8.5 | |||
aiohttp==3.9.0 |
Check failure
Code scanning / Trivy
Langchain Server-Side Request Forgery vulnerability High
@@ -1,4 +1,4 @@ | |||
aiohttp==3.8.5 | |||
aiohttp==3.9.0 |
Check failure
Code scanning / Trivy
LangChain Server Side Request Forgery vulnerability High
1b50f34
to
b5b4dcf
Compare
1aa9a23
to
53b1b84
Compare
53b1b84
to
6f5d909
Compare
This PR contains the following updates:
==3.8.5
->==3.9.0
GitHub Vulnerability Alerts
CVE-2023-47627
Summary
The HTTP parser in AIOHTTP has numerous problems with header parsing, which could lead to request smuggling.
This parser is only used when
AIOHTTP_NO_EXTENSIONS
is enabled (or not using a prebuilt wheel).Details
Bug 1: Bad parsing of
Content-Length
valuesDescription
RFC 9110 says this:
AIOHTTP does not enforce this rule, presumably because of an incorrect usage of the builtin
int
constructor. Because theint
constructor accepts+
and-
prefixes, and digit-separating underscores, usingint
to parse CL values leads AIOHTTP to significant misinterpretation.Examples
Suggested action
Verify that a
Content-Length
value consists only of ASCII digits before parsing, as the standard requires.Bug 2: Improper handling of NUL, CR, and LF in header values
Description
RFC 9110 says this:
AIOHTTP's HTTP parser does not enforce this rule, and will happily process header values containing these three forbidden characters without replacing them with SP.
Examples
Suggested action
Reject all messages with NUL, CR, or LF in a header value. The translation to space thing, while technically allowed, does not seem like a good idea to me.
Bug 3: Improper stripping of whitespace before colon in HTTP headers
Description
RFC 9112 says this:
AIOHTTP does not enforce this rule, and will simply strip any whitespace before the colon in an HTTP header.
Example
Suggested action
Reject all messages with whitespace before a colon in a header field, as the standard requires.
PoC
Example requests are embedded in the previous section. To reproduce these bugs, start an AIOHTTP server without llhttp (i.e.
AIOHTTP_NO_EXTENSIONS=1
) and send the requests given in the previous section. (e.g. byprintf
ing intonc
)Impact
Each of these bugs can be used for request smuggling.
GHSA-pjjw-qhg8-p2p9
Summary
llhttp 8.1.1 is vulnerable to two request smuggling vulnerabilities.
Details have not been disclosed yet, so refer to llhttp for future information.
The issue is resolved by using llhttp 9+ (which is included in aiohttp 3.8.6+).
CVE-2023-49082
Summary
Improper validation makes it possible for an attacker to modify the HTTP request (e.g. insert a new header) or even create a new HTTP request if the attacker controls the HTTP method.
Details
The vulnerability occurs only if the attacker can control the HTTP method (GET, POST etc.) of the request.
Previous releases performed no validation on the provided value. If an attacker controls the HTTP method it will be used as is and can lead to HTTP request smuggling.
PoC
A minimal example can be found here:
https://gist.github.com/jnovikov/7f411ae9fe6a9a7804cf162a3bdbb44b
Impact
If the attacker can control the HTTP version of the request it will be able to modify the request (request smuggling).
Workaround
If unable to upgrade and using user-provided values for the request method, perform manual validation of the user value (e.g. by restricting it to a few known values like GET, POST etc.).
CVE-2023-49081
Summary
Improper validation make it possible for an attacker to modify the HTTP request (e.g. to insert a new header) or even create a new HTTP request if the attacker controls the HTTP version.
Details
The vulnerability only occurs if the attacker can control the HTTP version of the request (including its type).
For example if an unvalidated JSON value is used as a version and the attacker is then able to pass an array as the
version
parameter.Furthermore, the vulnerability only occurs when the
Connection
header is passed to theheaders
parameter.At this point, the library will use the parsed value to create the request. If a list is passed, then it bypasses validation and it is possible to perform CRLF injection.
PoC
The POC below shows an example of providing an unvalidated array as a version:
https://gist.github.com/jnovikov/184afb593d9c2114d77f508e0ccd508e
Impact
CRLF injection leading to Request Smuggling.
Workaround
If these specific conditions are met and you are unable to upgrade, then validate the user input to the
version
parameter to ensure it is astr
.Release Notes
aio-libs/aiohttp (aiohttp)
v3.9.0
Compare Source
==================
Features
Introduced
AppKey
for static typing support ofApplication
storage.See https://docs.aiohttp.org/en/stable/web_advanced.html#application-s-config
#​5864 <https://github.com/aio-libs/aiohttp/issues/5864>
_Added a graceful shutdown period which allows pending tasks to complete before the application's cleanup is called.
The period can be adjusted with the
shutdown_timeout
parameter. -- by :user:Dreamsorcerer
.See https://docs.aiohttp.org/en/latest/web_advanced.html#graceful-shutdown
#​7188 <https://github.com/aio-libs/aiohttp/issues/7188>
_Added
handler_cancellation <https://docs.aiohttp.org/en/stable/web_advanced.html#web-handler-cancellation>
_ parameter to cancel web handler on client disconnection. -- by :user:mosquito
This (optionally) reintroduces a feature removed in a previous release.
Recommended for those looking for an extra level of protection against denial-of-service attacks.
#​7056 <https://github.com/aio-libs/aiohttp/issues/7056>
_Added support for setting response header parameters
max_line_size
andmax_field_size
.#​2304 <https://github.com/aio-libs/aiohttp/issues/2304>
_Added
auto_decompress
parameter toClientSession.request
to overrideClientSession._auto_decompress
. -- by :user:Daste745
#​3751 <https://github.com/aio-libs/aiohttp/issues/3751>
_Changed
raise_for_status
to allow a coroutine.#​3892 <https://github.com/aio-libs/aiohttp/issues/3892>
_Added client brotli compression support (optional with runtime check).
#​5219 <https://github.com/aio-libs/aiohttp/issues/5219>
_Added
client_max_size
toBaseRequest.clone()
to allow overriding the request body size. -- :user:anesabml
.#​5704 <https://github.com/aio-libs/aiohttp/issues/5704>
_Added a middleware type alias
aiohttp.typedefs.Middleware
.#​5898 <https://github.com/aio-libs/aiohttp/issues/5898>
_Exported
HTTPMove
which can be used to catch any redirection requestthat has a location -- :user:
dreamsorcerer
.#​6594 <https://github.com/aio-libs/aiohttp/issues/6594>
_Changed the
path
parameter inweb.run_app()
to accept apathlib.Path
object.#​6839 <https://github.com/aio-libs/aiohttp/issues/6839>
_Performance: Skipped filtering
CookieJar
when the jar is empty or all cookies have expired.#​7819 <https://github.com/aio-libs/aiohttp/issues/7819>
_Performance: Only check origin if insecure scheme and there are origins to treat as secure, in
CookieJar.filter_cookies()
.#​7821 <https://github.com/aio-libs/aiohttp/issues/7821>
_Performance: Used timestamp instead of
datetime
to achieve faster cookie expiration inCookieJar
.#​7824 <https://github.com/aio-libs/aiohttp/issues/7824>
_Added support for passing a custom server name parameter to HTTPS connection.
#​7114 <https://github.com/aio-libs/aiohttp/issues/7114>
_Added support for using Basic Auth credentials from :file:
.netrc
file when making HTTP requests with the:py:class:
~aiohttp.ClientSession
trust_env
argument is set toTrue
. -- by :user:yuvipanda
.#​7131 <https://github.com/aio-libs/aiohttp/issues/7131>
_Turned access log into no-op when the logger is disabled.
#​7240 <https://github.com/aio-libs/aiohttp/issues/7240>
_Added typing information to
RawResponseMessage
. -- by :user:Gobot1234
#​7365 <https://github.com/aio-libs/aiohttp/issues/7365>
_Removed
async-timeout
for Python 3.11+ (replaced withasyncio.timeout()
on newer releases).#​7502 <https://github.com/aio-libs/aiohttp/issues/7502>
_Added support for
brotlicffi
as an alternative tobrotli
(fixing Brotli support on PyPy).#​7611 <https://github.com/aio-libs/aiohttp/issues/7611>
_Added
WebSocketResponse.get_extra_info()
to access a protocol transport's extra info.#​7078 <https://github.com/aio-libs/aiohttp/issues/7078>
_Allow
link
argument to be set to None/empty in HTTP 451 exception.#​7689 <https://github.com/aio-libs/aiohttp/issues/7689>
_Bugfixes
Implemented stripping the trailing dots from fully-qualified domain names in
Host
headers and TLS context when acting as an HTTP client.This allows the client to connect to URLs with FQDN host name like
https://example.com./
.-- by :user:
martin-sucha
.#​3636 <https://github.com/aio-libs/aiohttp/issues/3636>
_Fixed client timeout not working when incoming data is always available without waiting. -- by :user:
Dreamsorcerer
.#​5854 <https://github.com/aio-libs/aiohttp/issues/5854>
_Fixed
readuntil
to work with a delimiter of more than one character.#​6701 <https://github.com/aio-libs/aiohttp/issues/6701>
_Added
__repr__
toEmptyStreamReader
to avoidAttributeError
.#​6916 <https://github.com/aio-libs/aiohttp/issues/6916>
_Fixed bug when using
TCPConnector
withttl_dns_cache=0
.#​7014 <https://github.com/aio-libs/aiohttp/issues/7014>
_Fixed response returned from expect handler being thrown away. -- by :user:
Dreamsorcerer
#​7025 <https://github.com/aio-libs/aiohttp/issues/7025>
_Avoided raising
UnicodeDecodeError
in multipart and in HTTP headers parsing.#​7044 <https://github.com/aio-libs/aiohttp/issues/7044>
_Changed
sock_read
timeout to start after writing has finished, avoiding read timeouts caused by an unfinished write. -- by :user:dtrifiro
#​7149 <https://github.com/aio-libs/aiohttp/issues/7149>
_Fixed missing query in tracing method URLs when using
yarl
1.9+.#​7259 <https://github.com/aio-libs/aiohttp/issues/7259>
_Changed max 32-bit timestamp to an aware datetime object, for consistency with the non-32-bit one, and to avoid a
DeprecationWarning
on Python 3.12.#​7302 <https://github.com/aio-libs/aiohttp/issues/7302>
_Fixed
EmptyStreamReader.iter_chunks()
never ending. -- by :user:mind1m
#​7616 <https://github.com/aio-libs/aiohttp/issues/7616>
_Fixed a rare
RuntimeError: await wasn't used with future
exception. -- by :user:stalkerg
#​7785 <https://github.com/aio-libs/aiohttp/issues/7785>
_Fixed issue with insufficient HTTP method and version validation.
#​7700 <https://github.com/aio-libs/aiohttp/issues/7700>
_Added check to validate that absolute URIs have schemes.
#​7712 <https://github.com/aio-libs/aiohttp/issues/7712>
_Fixed unhandled exception when Python HTTP parser encounters unpaired Unicode surrogates.
#​7715 <https://github.com/aio-libs/aiohttp/issues/7715>
_Updated parser to disallow invalid characters in header field names and stop accepting LF as a request line separator.
#​7719 <https://github.com/aio-libs/aiohttp/issues/7719>
_Fixed Python HTTP parser not treating 204/304/1xx as an empty body.
#​7755 <https://github.com/aio-libs/aiohttp/issues/7755>
_Ensure empty body response for 1xx/204/304 per RFC 9112 sec 6.3.
#​7756 <https://github.com/aio-libs/aiohttp/issues/7756>
_Fixed an issue when a client request is closed before completing a chunked payload. -- by :user:
Dreamsorcerer
#​7764 <https://github.com/aio-libs/aiohttp/issues/7764>
_Edge Case Handling for ResponseParser for missing reason value.
#​7776 <https://github.com/aio-libs/aiohttp/issues/7776>
_Fixed
ClientWebSocketResponse.close_code
being erroneously set toNone
when there are concurrent async tasks receiving data and closing the connection.#​7306 <https://github.com/aio-libs/aiohttp/issues/7306>
_Added HTTP method validation.
#​6533 <https://github.com/aio-libs/aiohttp/issues/6533>
_Fixed arbitrary sequence types being allowed to inject values via version parameter. -- by :user:
Dreamsorcerer
#​7835 <https://github.com/aio-libs/aiohttp/issues/7835>
_Performance: Fixed increase in latency with small messages from websocket compression changes.
#​7797 <https://github.com/aio-libs/aiohttp/issues/7797>
_Improved Documentation
Fixed the
ClientResponse.release
's type in the doc. Changed fromcomethod
tomethod
.#​5836 <https://github.com/aio-libs/aiohttp/issues/5836>
_Added information on behavior of base_url parameter in
ClientSession
.#​6647 <https://github.com/aio-libs/aiohttp/issues/6647>
_Fixed
ClientResponseError
docs.#​6700 <https://github.com/aio-libs/aiohttp/issues/6700>
_Updated Redis code examples to follow the latest API.
#​6907 <https://github.com/aio-libs/aiohttp/issues/6907>
_Added a note about possibly needing to update headers when using
on_response_prepare
. -- by :user:Dreamsorcerer
#​7283 <https://github.com/aio-libs/aiohttp/issues/7283>
_Completed
trust_env
parameter description to honorwss_proxy
,ws_proxy
orno_proxy
env.#​7325 <https://github.com/aio-libs/aiohttp/issues/7325>
_Expanded SSL documentation with more examples (e.g. how to use certifi). -- by :user:
Dreamsorcerer
#​7334 <https://github.com/aio-libs/aiohttp/issues/7334>
_Fix, update, and improve client exceptions documentation.
#​7733 <https://github.com/aio-libs/aiohttp/issues/7733>
_Deprecations and Removals
Added
shutdown_timeout
parameter toBaseRunner
, whiledeprecating
shutdown_timeout
parameter fromBaseSite
. -- by :user:Dreamsorcerer
#​7718 <https://github.com/aio-libs/aiohttp/issues/7718>
_Dropped Python 3.6 support.
#​6378 <https://github.com/aio-libs/aiohttp/issues/6378>
_Dropped Python 3.7 support. -- by :user:
Dreamsorcerer
#​7336 <https://github.com/aio-libs/aiohttp/issues/7336>
_Removed support for abandoned
tokio
event loop. -- by :user:Dreamsorcerer
#​7281 <https://github.com/aio-libs/aiohttp/issues/7281>
_Misc
Made
print
argument inrun_app()
optional.#​3690 <https://github.com/aio-libs/aiohttp/issues/3690>
_Improved performance of
ceil_timeout
in some cases.#​6316 <https://github.com/aio-libs/aiohttp/issues/6316>
_Changed importing Gunicorn to happen on-demand, decreasing import time by ~53%. -- :user:
Dreamsorcerer
#​6591 <https://github.com/aio-libs/aiohttp/issues/6591>
_Improved import time by replacing
http.server
withhttp.HTTPStatus
.#​6903 <https://github.com/aio-libs/aiohttp/issues/6903>
_Fixed annotation of
ssl
parameter to disallowTrue
. -- by :user:Dreamsorcerer
.#​7335 <https://github.com/aio-libs/aiohttp/issues/7335>
_v3.8.6
Compare Source
==================
Security bugfixes
Upgraded the vendored copy of llhttp_ to v9.1.3 -- by :user:
Dreamsorcerer
Thanks to :user:
kenballus
for reporting this, seeGHSA-pjjw-qhg8-p2p9.
.. _llhttp: https://llhttp.org
#​7647 <https://github.com/aio-libs/aiohttp/issues/7647>
_Updated Python parser to comply with RFCs 9110/9112 -- by :user:
Dreamorcerer
Thanks to :user:
kenballus
for reporting this, seeGHSA-gfw2-4jvh-wgfg.
#​7663 <https://github.com/aio-libs/aiohttp/issues/7663>
_Deprecation
Added
fallback_charset_resolver
parameter inClientSession
to allow a user-suppliedcharacter set detection function.
Character set detection will no longer be included in 3.9 as a default. If this feature is needed,
please use
fallback_charset_resolver <https://docs.aiohttp.org/en/stable/client_advanced.html#character-set-detection>
_.#​7561 <https://github.com/aio-libs/aiohttp/issues/7561>
_Features
Enabled lenient response parsing for more flexible parsing in the client
(this should resolve some regressions when dealing with badly formatted HTTP responses). -- by :user:
Dreamsorcerer
#​7490 <https://github.com/aio-libs/aiohttp/issues/7490>
_Bugfixes
Fixed
PermissionError
when.netrc
is unreadable due to permissions.#​7237 <https://github.com/aio-libs/aiohttp/issues/7237>
_Fixed output of parsing errors pointing to a
\n
. -- by :user:Dreamsorcerer
#​7468 <https://github.com/aio-libs/aiohttp/issues/7468>
_Fixed
GunicornWebWorker
max_requests_jitter not working.#​7518 <https://github.com/aio-libs/aiohttp/issues/7518>
_Fixed sorting in
filter_cookies
to use cookie with longest path. -- by :user:marq24
.#​7577 <https://github.com/aio-libs/aiohttp/issues/7577>
_Fixed display of
BadStatusLine
messages from llhttp_. -- by :user:Dreamsorcerer
#​7651 <https://github.com/aio-libs/aiohttp/issues/7651>
_Configuration
📅 Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by Renovate Bot.