You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
With warcmiddleware.py, doing 'scrapy fetch http://www.pcaobus.org' I get a redirect loop.
I've isolated the problem to the line "headers.setdefault('Host', netloc)" in _warcrec_from_scrapy_request
When removing it everything seems to be working fine. Also - I think the middleware doesn't intend to change the headers, but it does.
The text was updated successfully, but these errors were encountered:
With warcmiddleware.py, doing 'scrapy fetch http://www.pcaobus.org' I get a redirect loop.
I've isolated the problem to the line "headers.setdefault('Host', netloc)" in _warcrec_from_scrapy_request
When removing it everything seems to be working fine. Also - I think the middleware doesn't intend to change the headers, but it does.
The text was updated successfully, but these errors were encountered: