Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

使用gevent时,全局DB_session造成的异常“RuntimeError: reentrant call inside <_io.BufferedReader name=1880>” #151

Open
bjtudujunlin opened this issue Aug 22, 2018 · 0 comments

Comments

@bjtudujunlin
Copy link

在startProxyCrawl进程中,启动时会检测数据库中代理是否可用,代码位置在:
spawns.append(gevent.spawn(detect_from_db, self.myip, proxy, self.proxies))
当代理不可用时会调用更新操作:
sqlhelper.update({'ip': proxy[0], 'port': proxy[1]}, {'score': score})
因为程序使用gevent,所以在不可用代理较多时会存在并发调用update操作的情况,但是所有update操作共用一个SqlHelper的session实例,该session不是线程安全的,会导致系统报如下异常:
RuntimeError: reentrant call inside <_io.BufferedReader name=2296>

需要为该session的更新、删除、添加操作增加同步锁

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant