Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

关于Spider#setScheduler(Scheduler)存在bug #1172

Open
KamiNoYuki opened this issue Jul 6, 2024 · 0 comments
Open

关于Spider#setScheduler(Scheduler)存在bug #1172

KamiNoYuki opened this issue Jul 6, 2024 · 0 comments

Comments

@KamiNoYuki
Copy link

KamiNoYuki commented Jul 6, 2024

方法代码:

public Spider setScheduler(Scheduler updateScheduler) {
        checkIfRunning();
        //此处存在问题,这样会导致直接将内部的scheduler设置为updateScheduler,然后又尝试自己poll自己进行数据迁徙
        SpiderScheduler oldScheduler = this.scheduler;
        //应该修改为下面这样
        //Scheduler oldScheduler = scheduler.getScheduler();
        scheduler.setScheduler(updateScheduler);
        Request request;
        while ((request = oldScheduler.poll(this)) != null) {
            System.out.println("move oldScheduler task to updateScheduler");
            this.scheduler.push(request, this);
        }
        return this;
    }
blanexie pushed a commit to blanexie/webmagic that referenced this issue Aug 6, 2024
sutra pushed a commit that referenced this issue Aug 6, 2024
Co-authored-by: xiezhicheng <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant