python 中关于多线程的两个问题

0 0 python
闭麦听歌1v9
闭麦听歌1v9

声望值:118 0人

2019-01-07 13:48:17 提问

关注 0关注

收藏 0收藏, 154浏览

第一个问题:

import queue
import threading
import time

for i in range(20):
    a = threading.Thread(target = worker)
    a.setDaemon(True)
    a.start()
    

我想知道,在这个for循环里,这里设置的setDaemon里所绑定的主线程都是一样的吗?还是说因为for循环把每个子线程的守护线程都绑定为了上一个子线程?

第二个问题:

import queue
import threading
import time

aqueue = queue.Queue()
for i in range(10):
    aqueue.put(i)

def worker():
    while True:
        time.sleep(5)
        item = aqueue.get()
        print(item)

for i in range(20):
    a = threading.Thread(target = worker)
    a.setDaemon(True)
    a.start()
    
aqueue.join()    

在添加queue模块后,让我疑惑的是,程序在跑完aqueue里的所有项后却没有退出,我查了一下官网queue.join()的定义是Blocks until all items in the queue have been gotten and processed.这里子线程和主线程都执行完了,为什么没有退出呢?

请先 登录 后评论

3个回答

  • ERP↘SAP135声望 2018-09-25 09:30

    import Queue import threading import time aqueue = Queue.Queue() for i in range(10): aqueue.put(i) def worker(): while True: time.sleep(1) item = aqueue.get() print(item) aqueue.task_done() for i in range(1): a = threading.Thread(target = worker) a.setDaemon(True) a.start() aqueue.join() print 'the main thread is already over' 用 aqueue.task_done() 去通知阻塞的 join 是一种方法
    请先 登录 后评论
  • -II、执念213声望 2018-09-25 09:30

    第一个问题 一样的第二个问题1、队列只有十个值,你开了20个线程,aqueue.get()在取不到值的时候会使线程暂停,后面的线程取不到值程序当然不会退出。2、while True:也会使线程不停止。3、aqueue.join()会优先于线程执行,并阻塞主进程继续 如果你是想主进程等待线程结束再停止可以这么写 import queue import threading import time aqueue = queue.Queue() for i in range(10): aqueue.put(i) def worker(): time.sleep(5) item = aqueue.get() print(item) for i in range(20): a = threading.Thread(target=worker) a.setDaemon(True) a.start() a.join() # aqueue.join()
    请先 登录 后评论
  • 攻城狮44声望 2018-09-25 09:30

    我也刚看,试了好久,初步知道怎么回事了.先回答你的两个问题:1、setDaemon所绑定的都是主线程,都是一样的,即运行py文件第一次创建的那个线程(也是主进程),有且只有一个2、queue.join()如果单独使用会被无限挂起,字面翻译为等待队列为空,再执行别的操作.但是他等待的队列不是我们创建的aqueue,而是一个与aqueue长度相等的一个"需要完成的子线程"队列,他判断的很可能是这个列表,当这个列表为空时queue.join()才成立,所以如楼上的那位所说queue.join()要结合aqueue.task_done()函数来使用,aqueue.task_done()的意思是每task_done一次 就从"需要完成的子线程"队列里删掉一个元素,这样在最后join的时候根据队列长度是否为零来判断队列是否结束,aqueue.task_done()用在子线程函数中,这样queue.join()才知道完成了多少个子线程,才不会无限挂起,也就是为什么你没退出的原因 但是你的程序也有问题:主要的问题是,守护线程机制:主线程运行完毕去结束子线程时,由于有大量的子线程还在time.sleep(5),结束这些子线程会触发异常: Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads Thread 0x000019cc (most recent call first): File "test.py", line 13 in worker File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap 这是因为:下面的一些说法是错误的,可以看我后面的补充,这里就不改了,是一种猜测,但是应该确实不是这样喂不饱的子线程:你aqueue.put(i)了是10个元素,但是却运行了20个子线程来"吃"这10个元素,又因为这20个线程都是守护进程,当10个吃饱了,这个10个卡在了item = aqueue.get()等待接受元素,另外的那些很有可能卡在了time.sleep(5)上,而主线程运行完毕,准备结束所有的守护进程,我判断,结束卡在item = aqueue.get()的守护进程不会报错,而结束其他程序,如time.sleep会触发异常,这也是为啥楼上那位吧你的for i in range(20):改成了for i in range(1):的原因. 还有两种退出的方式:1、是在子线程里判断队列是否为空: if aqueue.empty() == True: break 为空就退出2、主线程通过queue向子线程发送退出命令其实这里一般用pipe管道的,Queue队列不好针对特定的线程进行退出操作,大概的伪码为: 主线程: aqueue.put('quit') 子线程"" item = aqueue.get() if item == 'quit': return else: # "do something" 所以多进程,多线程还是要控制好程序逻辑,控制好调度. 我没用过守护进程,我一般用的是普通的用户进程,这样主线程跑完会等待所有子线程运行完毕再结束 其实还没完=_=:后面的都是我的猜测,算是附加的,选修:这程序的细节也值得我们关注:这程序如果真去实践的话很容易炸,以下都是我的猜测如上所述,我们修改程序,注意注释 import queue import threading import time aqueue = queue.Queue() # 发送了0到9的十个数据 for i in range(10): aqueue.put(i) def worker(): while True: # 输出当前线程,和主线程(可以看到主线程都一样) print('thread %s is running...\nmain_thread %s is running...\n\n' % (threading.current_thread().name,threading.main_thread())) time.sleep(1) item = aqueue.get() print(item) aqueue.task_done() print('thread end') n = 1 """注意这!!!!!!!!!!!!!!!!!!""" # 运行了10个线程来处理这十个数据 for i in range(10): print("run thread Counts: ",n) n += 1 a = threading.Thread(target = worker) a.setDaemon(True) a.start() aqueue.join() print('after to do') print('end') 10个数据10个线程,time.cleep(5)秒结果会怎么样?运行正常?很不幸运的炸了23333333有一定的几率报错,和上面一样: Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads Thread 0x0000063c (most recent call first): File "test.py", line 13 in worker File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap 我电脑上大部分运行正常,只有几次会这样,为什么会这样?换成10个数据9个线程就不会出错了我猜测的情况是,初步认为:直接看完整的输出,注意看有注释的地方: run thread Counts: 1 thread Thread-1 is running... main_thread <_MainThread(MainThread, started 6680)> is running... run thread Counts: 2 thread Thread-2 is running... main_thread <_MainThread(MainThread, started 6680)> is running... run thread Counts: 3 thread Thread-3 is running... main_thread <_MainThread(MainThread, started 6680)> is running... run thread Counts: 4 thread Thread-4 is running... main_thread <_MainThread(MainThread, started 6680)> is running... run thread Counts: 5 thread Thread-5 is running... main_thread <_MainThread(MainThread, started 6680)> is running... run thread Counts: 6 thread Thread-6 is running... main_thread <_MainThread(MainThread, started 6680)> is running... run thread Counts: 7 thread Thread-7 is running... main_thread <_MainThread(MainThread, started 6680)> is running... run thread Counts: 8 thread Thread-8 is running... main_thread <_MainThread(MainThread, started 6680)> is running... run thread Counts: 9 thread Thread-9 is running... main_thread <_MainThread(MainThread, started 6680)> is running... run thread Counts: 10 thread Thread-10 is running... main_thread <_MainThread(MainThread, started 6680)> is running... 0 1 3 4 thread end thread end 5 2 thread Thread-2 is running... main_thread <_MainThread(MainThread, started 6680)> is running... thread end thread end thread end thread Thread-3 is running... main_thread <_MainThread(MainThread, started 6680)> is running... thread end thread Thread-4 is running... main_thread <_MainThread(MainThread, started 6680)> is running... thread Thread-1 is running... main_thread <_MainThread(MainThread, started 6680)> is running... thread Thread-5 is running... main_thread <_MainThread(MainThread, started 6680)> is running... 6 8 thread Thread-6 is running... main_thread <_MainThread(MainThread, started 6680)> is running... thread end thread end thread Thread-7 is running... main_thread <_MainThread(MainThread, started 6680)> is running... 7 9 thread Thread-9 is running... main_thread <_MainThread(MainThread, started 6680)> is running... """到这里主线程已经完毕""" thread end after to do """ 这里其实系统在调用守护线程的结束程序来结束所有子线程 """ """但是在结束守护进程程序后,又运行了一个不该运行的子线程,这就是为什么会报错的原因""" thread Thread-10 is running... main_thread <_MainThread(MainThread, started 6680)> is running... end thread end thread Thread-8 is running... main_thread <_MainThread(MainThread, started 6680)> is running... Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads Thread 0x00000424 (most recent call first): File "test.py", line 14 in worker File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap Thread 0x00000f04 (most recent call first): File "test.py", line 14 in worker File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap Thread 0x0000165c (most recent call first): File "test.py", line 13 in worker File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap Thread 0x000018b0 (most recent call first): File "test.py", line 14 in worker File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap Thread 0x00000db4 (most recent call first): File "test.py", line 14 in worker File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap Thread 0x00001a88 (most recent call first): File "test.py", line 14 in worker File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap Thread 0x0000111c (most recent call first): File "test.py", line 14 in worker File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap Thread 0x0000177c (most recent call first): File "test.py", line 14 in worker File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap Thread 0x000008e4 (most recent call first): File "test.py", line 14 in worker File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap Thread 0x00001788 (most recent call first): File "test.py", line 14 in worker File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap Current thread 0x00001a18 (most recent call first): 看主要的几句 """到这里主线程已经完毕""" thread end after to do """ 这里其实系统在调用守护线程的结束程序来结束所有子线程 """ """但是在结束守护进程程序后,又运行了一个不该运行的子线程,这就是为什么会报错的原因""" thread Thread-10 is running... main_thread <_MainThread(MainThread, started 6680)> is running... end thread end 先这样吧,可以自己运行下程序看看 看了另一个答主的,使用了a.join()也是一种选择 改成发送10个数据,用9个线程跑,效率是最高的(理论) import queue import threading import time aqueue = queue.Queue() # 发送了0到9的十个数据 for i in range(10): aqueue.put(i) def worker(): while True: # 输出当前线程,和主线程(可以看到主线程都一样) print('thread %s is running...\nmain_thread %s is running...\n\n' % (threading.current_thread().name,threading.main_thread())) time.sleep(5) item = aqueue.get() print(item) aqueue.task_done() print('thread end') n = 1 """注意这!!!!!!!!!!!!!!!!!!""" for i in range(9): print("run thread Counts: ",n) n += 1 a = threading.Thread(target = worker) a.setDaemon(True) a.start() aqueue.join() print('after to do') print('end')
    请先 登录 后评论

注册新账号

悬赏追问
10
  • 10
  • 20
  • 50
  • 100
  • 200
  • 输入数值
发布追问