DNS crawler 1.4.2

March 24, 2020

Version 1.4.2 of DNS crawler is out!

New features:

  • added an option to save binary content if the webserver returns it for root (or redirects to it)
  • added a max_ips_per_domain…

DNS crawler 1.4.1

March 23, 2020

Version 1.4.1 of DNS crawler was published. The changes include better handling of CNAME records and workarounds for broken web pages.