We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
多线程情况下,tryFinish()会误判CrawlerThread的运行状态,导致提前stop,以下是运行XxlCrawlerTest,开启3个thread,并打印日志:
CrawlerThread
概率比较小,大概试10次能出现一次,原因可能如下: thread-3调用tryFinish()并提前获取了3个CrawlerThread的isRunning状态均为false,刚好此时thread-1调用了crawler.getRunData().getUrl()并将running设为true(但thread-3已经无法知晓),最后thread-3判断runData.getUrlNum()==0为true,由此isEnd为true,导致了误判:
tryFinish()
isRunning
crawler.getRunData().getUrl()
runData.getUrlNum()==0
isEnd
public void tryFinish(){ boolean isEnd = runData.getUrlNum()==0; boolean isRunning = false; for (CrawlerThread crawlerThread: crawlerThreads) { if (crawlerThread.isRunning()) { isRunning = true; break; } } isEnd = isEnd && !isRunning; if (isEnd) { logger.info(">>>>>>>>>>> xxl crawler is finished."); stop(); } }
private volatile boolean running;
The text was updated successfully, but these errors were encountered:
后续测试发现即使做了上述修改,极低的概率还是会出问题,猜测是CrawlerThread在调用完crawler.getRunData().getUrl();后时间片到期,没有来得及running = true;,导致其他线程继续误判状态 这样的话只能改进整个tryFinish()的逻辑流程了
crawler.getRunData().getUrl();
running = true;
Sorry, something went wrong.
No branches or pull requests
多线程情况下,tryFinish()会误判
CrawlerThread
的运行状态,导致提前stop,以下是运行XxlCrawlerTest,开启3个thread,并打印日志:概率比较小,大概试10次能出现一次,原因可能如下:
thread-3调用
tryFinish()
并提前获取了3个CrawlerThread的isRunning
状态均为false,刚好此时thread-1调用了crawler.getRunData().getUrl()
并将running设为true(但thread-3已经无法知晓),最后thread-3判断runData.getUrlNum()==0
为true,由此isEnd
为true,导致了误判:tryFinish()
,先判断runData.getUrlNum()==0
,再逐一获取CrawlerThread的状态,防止调用crawler.getRunData().getUrl()
无法获取running的最新状态:The text was updated successfully, but these errors were encountered: