threads.json endpoint being blocked by CloudFlare's Browser Integrity Check

[jiva]

See title. I have a script processing the threads.json endpoint and I recently noticed that it's getting blocked by CloudFlare's browser integrity check. Can the check be disabled for that endpoint, since it's meant to be consumed by scripts?

[desuwa]

This will be fixed soon.

[Regaccio]

It seems like this issue is back. I'm not a developer, just a lowly script user, but I noticed the issue returned when using 4chan X. It seems like the script can't access a.4cdn.org due to 503 errors, and CSS is also broken unless I visit 4cdn.org directly and go through Cloudflare's Browser Integrity Check.
Same issue as ccd0/4chan-x#1746

[desuwa]

This happens when you send too many requests too often.
You'll get unblocked automatically, eventually.

[Regaccio]

I see. I had thread updating set to 1 second so that could be the problem, even though the limit on requests is one per second. So basically I just need to set the thread updater to only update every ten seconds as the API rules dictate and that should reduce the number of requests, thus preventing this from happening anymore?

Also, what exactly do you mean by too many requests too often? In too short of a time frame (e.g. a bunch of tabs open all making new requests quickly), or over a long period (e.g. going afk for hours and leaving browser open with a tab making regular quick requests)? Just want to know so I can avoid this in the future.

If there's any way to get unblocked manually that would be nice as well.

[desuwa]

The doc says no more than one request per second. In practice that limit is much higher and unless you have tons of threads constantly autorefreshing every second you should be fine.

[chairus]

@desuwa Given a scenario where I have N threads serving users and each thread would most likely run concurrently and doing 1 request per second, what would be the minimum number of N so that I wouldn't get rate limited?

Thanks

[desuwa]

Anything above 1rps can be subject to rate limiting.

[chairus]

Does that mean I am limited to only having 1 thread to service my users?

[desuwa]

It's not directly related to the number of threads you use. As long as your application doesn't make more than 1 request per second on average, you'll be fine.
Again, that limit is higher than that in practice and is enforced in a pretty lenient manner, but there's no guarantee it'll always stay this way.