Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

False positive detection of Yandex mobile browser as a bot due to HTTP_SEC_CH_UA header #541

Closed
IlyaZholobov opened this issue Sep 27, 2024 · 3 comments

Comments

@IlyaZholobov
Copy link

Hi, since the latest patch (version 1.2.120) of crawler-detect, I've encountered an issue where the Yandex mobile browser is being falsely detected as a bot. This seems to be due to the addition of the HTTP_SEC_CH_UA header in the detection process.

In the Yandex mobile browser, the HTTP_SEC_CH_UA header includes the following string: "Not/A)Brand";v="8", "Chromium";v="126", "Yandex";v="24". The string "Yandex" triggers the bot detection, but this is a legitimate browser, not a bot.

Could this detection rule be adjusted in the next patch, or is there a way to configure it to avoid this false positive?

@module17
Copy link
Contributor

module17 commented Sep 30, 2024

Hi, since the latest patch (version 1.2.120) of crawler-detect, I've encountered an issue where the Yandex mobile browser is being falsely detected as a bot. This seems to be due to the addition of the HTTP_SEC_CH_UA header in the detection process.

In the Yandex mobile browser, the HTTP_SEC_CH_UA header includes the following string: "Not/A)Brand";v="8", "Chromium";v="126", "Yandex";v="24". The string "Yandex" triggers the bot detection, but this is a legitimate browser, not a bot.

Could this detection rule be adjusted in the next patch, or is there a way to configure it to avoid this false positive?

For these requests could you provide the other HTTP_* headers? Importantly the HTTP_USER_AGENT value.

It looks like this could be addressed already with #542

@IlyaZholobov
Copy link
Author

UserAgent Example:
Mozilla/5.0 (Linux; arm_64; Android 14; 22071212AG) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.6478.61 YaBrowser/24.7.9.61.00 SA/3 Mobile Safari/537.36 "Not/A)Brand";v="8", "Chromium";v="126", "Yandex";v="24"

HTTP_SEC_CH_UA :
"Not/A)Brand";v="8", "Chromium";v="126", "Yandex";v="24"

@IlyaZholobov
Copy link
Author

IlyaZholobov commented Nov 14, 2024

@module17

#542 - yes, this solves the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants