Tweaks

raw/Crawlers.txt

@@ -40,6 +40,7 @@ Abonti
 Aboundex
 aboutthedomain
 Accoona-AI-Agent
+acebookexternalhit\/
 acoon
 acrylicapps\.com\/pulp
 Acunetix
@@ -114,6 +115,7 @@ Backlink-Ceck
 backlink-check
 BacklinkHttpStatus
 BackStreet
+BackupLand
 BackWeb
 Bad-Neighborhood
 Badass
@@ -179,6 +181,7 @@ catexplorador
 CC Metadata Scaper
 Cegbfeieh
 censys
+centuryb.o.t9[at]gmail.com
 Cerberian Drtrs
 CERT\.at-Statistics-Survey
 cg-eye
@@ -227,6 +230,7 @@ Crowsnest
 Criteo
 CSHttp
 CSSCheck
+Cula
 curb
 Curious George
 curl
@@ -250,6 +254,7 @@ Demon
 DeuSu
 developers\.google\.com\/\+\/web\/snippet\/
 Devil
+DHSH
 Digg
 Digincore
 DigitalPebble
@@ -337,6 +342,7 @@ feeder
 Feedly
 FeedshowOnline
 Feedspot
+FeedViewer\/
 Feedwind\/
 FeedZcollector
 feeltiptop
@@ -506,6 +512,7 @@ IDG Twitter Links Resolver
 IDwhois\/
 Iframely
 igdeSpyder
+iGooglePortal
 IlTrovatore
 Image Fetch
 Image Sucker
@@ -777,6 +784,7 @@ Outlook-Express
 Outlook-iOS
 ow\.ly
 Owler
+Owlin
 ownCloud News
 OxfordCloudService
 Page Valet
@@ -954,6 +962,7 @@ shortURL lengthener
 ShortLinkTranslate
 shrinktheweb
 Sideqik
+Siege
 SimplePie
 SimplyFast
 Siphon
@@ -1199,11 +1208,13 @@ WeSEE
 wf84
 Wfuzz\/
 wget
+WhatCMS
 WhatsApp
 WhatsMyIP
 WhatWeb
 WhereGoes\?
 Whibse
+WhoAPI\/
 WhoRunsCoinHive
 Whynder Magnet
 WinHttp-Autoproxy-Service

tests/crawlers.txt

@@ -3863,4 +3863,5 @@ Goldfire Server
 OhDear.app (+https://ohdear.app/docs/checks/uptime)
 Mozilla/5.0 (compatible;Cloudflare-Healthchecks/1.0;+https://www.cloudflare.com/; healthcheck-id: 755842180c7ae904)
 WinHttp-Autoproxy-Service/5.1
-DynatraceSynthetic/
+DynatraceSynthetic/
+acebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php)

tests/devices.txt

@@ -148933,7 +148933,6 @@ Mozilla/5.0 (iPhone; CPU iPhone OS 11_1 like Mac OS X) AppleWebKit/604.3.5 (KHTM
 Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.2; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; Tablet PC 2.0; InfoPath.3; MASEJS; ms-office)
 Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.1.6) Gecko/20091215 Ubuntu/9.10 (karmic) Firefox/3.5.6
 Mozilla/5.0 (iPhone; CPU iPhone OS 9_0_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13A452 [FBAN/FBIOS;FBAV/142.0.0.79.88;FBBV/72578945;FBDV/iPhone8,1;FBMD/iPhone;FBSN/iPhone OS;FBSV/9.0.2;FBSS/2;FBCR/O2;FBID/phone;FBLC/en_GB;FBOP/5;FBRV/73520697]
-acebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php)
 Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_2 like Mac OS X) AppleWebKit/603.2.4 (KHTML, like Gecko) Mobile/14F89 (4299223920)
 Mozilla/5.0 (Linux; Android 6.0.1; Aquaris E5 Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/53.0.2785.124 Mobile Safari/537.36
 Mozilla/5.0 (Linux; Android 6.0.1; ZUK Z2132 Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/63.0.3239.111 Mobile Safari/537.36