Skip to content

v1.2.93: Add 4 new crawlers. Add scraper as a catch all word (#359)
Compare
Choose a tag to compare
@MaxGiting MaxGiting released this 16 Feb 12:35
· 276 commits to master since this release
v1.2.93
608ec84
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode. 
* Add 4 new user agents

* Add the word scraper as a generic user agent flag

* build crawler json and txt export files
Assets 2
Loading