Only first 20 linked pages get downloaded

[tal-amir]

Hi there,

Is there a hard-coded limit to the number of downloaded linked pages? (i.e. breadth, not depth)

I'm observing it when trying to capture a webpage, including all depth-1 links.
It seems that WebScrapBook only downloads the first 20 links that match the inclusion criteria. The remaining links keep pointing to their original web address.
I verified this by changing the setting "Included URLs for capturing linked pages". This has led to a different subset of links beging downloaded, but still of size 20.

Is it possible to change that limit to an arbitrary number or disable it altogether?

Thanks!

[danny0838]

There should be no such limit. Please provide the webpage URL and the capture config for further investigation (copy it from Capture As > Advanced).

[tal-amir]

Can I send them to you by email? I prefer not to share it publically.

[danny0838]

You can, but it would be much more problematic to track the issue. Mask sensitive information instead whenever possible.

[tal-amir]

I don't think this is feasible, but the problem does seem to be specific to that website.
I'll try to find another solution. Thanks