Kemono downloader not working

[Rogue9341]

New to github and not really good with any programming, so sorry if this post isn't formatted correctly, but I'm having an issue with downloading from Kemono now that their Patreon importer is back up, and I think maybe the dev changed their API endpoints or implemented something to block mass downloads.
Here's an example of the output log I got when trying to download a Patreon user's page with the command

gallery-dl https://kemono.su/patreon/user/6858411

[downloader.http][warning] HTTPSConnectionPool(host='n3.kemono.su', port=443): Max retries exceeded with url: /data/8a/f9/8af9252e07500027301439e8732fbda4a5d7c68a4be2cf121d8216b5677b35f5.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000002A36AD68520>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')) (1/5)
[downloader.http][warning] HTTPSConnectionPool(host='n3.kemono.su', port=443): Max retries exceeded with url: /data/8a/f9/8af9252e07500027301439e8732fbda4a5d7c68a4be2cf121d8216b5677b35f5.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000002A36AD68E20>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')) (2/5)
[downloader.http][warning] HTTPSConnectionPool(host='n3.kemono.su', port=443): Max retries exceeded with url: /data/8a/f9/8af9252e07500027301439e8732fbda4a5d7c68a4be2cf121d8216b5677b35f5.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000002A36AD78700>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')) (3/5)
[downloader.http][warning] HTTPSConnectionPool(host='n3.kemono.su', port=443): Max retries exceeded with url: /data/8a/f9/8af9252e07500027301439e8732fbda4a5d7c68a4be2cf121d8216b5677b35f5.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000002A36AD78FA0>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')) (4/5)
[downloader.http][warning] HTTPSConnectionPool(host='n3.kemono.su', port=443): Max retries exceeded with url: /data/8a/f9/8af9252e07500027301439e8732fbda4a5d7c68a4be2cf121d8216b5677b35f5.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000002A36AD86880>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')) (5/5)
[download][error] Failed to download 116006521_Jane Doe🧀_01_JD5sS.jpg
[downloader.http][warning] HTTPSConnectionPool(host='n3.kemono.su', port=443): Max retries exceeded with url: /data/d7/5f/d75f5aa2849ee62857ee6a762aa4501455e39481c1e1453497c74453f529ec76.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000002A36AD68D60>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')) (1/5)
[downloader.http][warning] HTTPSConnectionPool(host='n3.kemono.su', port=443): Max retries exceeded with url: /data/d7/5f/d75f5aa2849ee62857ee6a762aa4501455e39481c1e1453497c74453f529ec76.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000002A36AD86550>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')) (2/5)
[downloader.http][warning] HTTPSConnectionPool(host='n3.kemono.su', port=443): Max retries exceeded with url: /data/d7/5f/d75f5aa2849ee62857ee6a762aa4501455e39481c1e1453497c74453f529ec76.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000002A36AD86DF0>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')) (3/5)
[downloader.http][warning] HTTPSConnectionPool(host='n3.kemono.su', port=443): Max retries exceeded with url: /data/d7/5f/d75f5aa2849ee62857ee6a762aa4501455e39481c1e1453497c74453f529ec76.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000002A36AD856D0>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')) (4/5)
[downloader.http][warning] HTTPSConnectionPool(host='n3.kemono.su', port=443): Max retries exceeded with url: /data/d7/5f/d75f5aa2849ee62857ee6a762aa4501455e39481c1e1453497c74453f529ec76.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000002A36AD85FA0>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')) (5/5)
[download][error] Failed to download 113660246_How I'am_٩(ఠ益ఠ)۶_01_d75f5aa2849ee62857ee6a762aa4501455e39481c1e1453497c74453f529ec76.jpg

[mikf]

No connection could be made because the target machine actively refused it

Sounds like kemono's servers are having some issues (although I can personally download from this URL without problems). Wait a bit and try again later, I guess.

maybe the dev changed their API endpoints

#6415

Downloads with default settings shouldn't be affected by this, though.

[God-damnit-all]

@mikf They've been making changes lately. I find myself unable to download today because I think they changed their frontend and the KemonopartyUserExtractor is no longer working. Here's the log with some details replaced with pound signs removed:

[gallery-dl][debug] Version 1.28.0-dev:2024.11.22 - Executable (dev/windows)
[gallery-dl][debug] Python 3.12.7 - Windows-10-10.0.19045-SP0
[gallery-dl][debug] requests 2.32.3 - urllib3 2.2.3
[gallery-dl][debug] Configuration Files ['D:\\#########\\#########']
[gallery-dl][debug] Starting DownloadJob for 'https://kemono.party/patreon/user/#########'
[kemonoparty][debug] Using KemonopartyUserExtractor for 'https://kemono.party/patreon/user/#########'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): kemono.party:443
[urllib3.connectionpool][debug] https://kemono.party:443 "GET /patreon/user/#########HTTP/11" 308 164
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): kemono.su:443
[urllib3.connectionpool][debug] https://kemono.su:443 "GET /patreon/user/######### HTTP/11" 200 1020
[kemonoparty][error] An unexpected error occurred: TypeError - argument of type 'NoneType' is not iterable. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[kemonoparty][debug]
Traceback (most recent call last):
  File "gallery_dl\job.py", line 151, in run
  File "gallery_dl\extractor\kemonoparty.py", line 66, in items
  File "html\__init__.py", line 130, in unescape
TypeError: argument of type 'NoneType' is not iterable

[DrQuantum101]

To add, here's the error in relation to trying to download favourites:

[gallery-dl][debug] Version 1.27.6
[gallery-dl][debug] Python 3.11.4 - Windows-10-10.0.22631-SP0
[gallery-dl][debug] requests 2.31.0 - urllib3 2.1.0
[gallery-dl][debug] Configuration Files ['[REDACTED]\gallery-dl.conf']
[gallery-dl][debug] Starting DownloadJob for 'https://kemono.su/favorites'
[kemonoparty][debug] Using KemonopartyFavoriteExtractor for 'https://kemono.su/favorites'
[cookies][debug] Extracting cookies from C:\Users\XXXXX\AppData\Roaming\Mozilla\Firefox\Profiles\env16i3q.default-release-1713214542371\cookies.sqlite
[cookies][debug] Only loading cookies not belonging to any container
[cookies][info] Extracted 2638 cookies from Firefox
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): kemono.su:443
[urllib3.connectionpool][debug] https://kemono.su:443 "GET /api/v1/account/favorites?type=artist HTTP/1.1" 200 None
[kemonoparty][debug] Using KemonopartyUserExtractor for 'https://kemono.su/patreon/user/XXXXXXXX'
[kemonoparty][debug] Using cached cookies from ('firefox',)
[urllib3.connectionpool][debug] https://kemono.su:443 "GET /patreon/user/73988131 HTTP/1.1" 200 1020
[kemonoparty][error] An unexpected error occurred: TypeError - argument of type 'NoneType' is not iterable. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[kemonoparty][debug]
Traceback (most recent call last):
  File "C:\Users\XXXXX\AppData\Local\Programs\Python\Python311\Lib\site-packages\gallery_dl\job.py", line 151, in run
    for msg in extractor:
  File "C:\Users\XXXXX\AppData\Local\Programs\Python\Python311\Lib\site-packages\gallery_dl\extractor\kemonoparty.py", line 66, in items
    username = text.unescape(text.extract(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\XXXXX\AppData\Local\Programs\Python\Python311\Lib\html\__init__.py", line 130, in unescape
    if '&' not in s:
       ^^^^^^^^^^^^
TypeError: argument of type 'NoneType' is not iterable

I can confirm its not related to config settings as default config results in the same username retrieval issue.

[valdearg]

So this original ticket was due to the server infrastructure changes which were announced ahead of time and nothing to do with gallery-dl.

The issues today are different. The owner pushed out the previously mentioned updates to change the frontend over to react, this makes pretty big changes to the API. It was previously reverted because there were a lot of bugs present. This was initially raised in #6415 which I suggest this conversation is moved back to.

It may be worth checking back tomorrow as I know there's a few issues still, so might be reverted again.

If not it'll need the gallery-dl to support the new API: https://kemono.su/documentation/api

[valdearg]

Aaaaaaaaaaaaaaand it's reverted. Back to the old version.

[God-damnit-all]

Womp womp.

[Hrxn]

label: site-change

[God-damnit-all]

And now it's back on the new version.

[DrQuantum101]

Hoping that gallery-dl will be able to preserve the keyword variable name parity between the site changes. I have a few python scripts to output custom metadata .html files based on select items in the retrieved metadata. It would be a little bit of a hassle to rewrite them with new metadata keyword variable names.

[God-damnit-all]

Hoping that gallery-dl will be able to preserve the keyword variable name parity between the site changes. I have a few python scripts to output custom metadata .html files based on select items in the retrieved metadata. It would be a little bit of a hassle to rewrite them with new metadata keyword variable names.

It depends on if those keywords are accessible via API (and if they are, might not be in the same order). The only alternative would be scraping the page through a headless browser or something, which is outside of the scope of gallery-dl and carries with it a whole slew of complications.