inter thread communication with message queues
I was reading about message queues with respect to inter thread communication, and I have a specific use case that I want to describe. It's rather complex, so I want to be careful that I design it so that there are no race conditions. Pool A is a buffer (of type 1) handle manager, that feeds buffer handles into thread set 1, and receives old handles from thread set 2. There is only one thread running pool A. Thread set 1 is a set of N threads that fill up buffers of type 1 with regular data. It pushes full buffers to thread set 2. Thread set 2 is a set of M threads that compress buffers of type 1 into data buffers of type 2. Once the buffer of type 1 has be used up, it is cleared and sent back to Pool A. It pulls fresh data buffers of type 2 from Pool B, and sends freshly filled compressed buffers to Pool C. Pool C does something with the buffer, clears it and hands it to Pool B. Can this be done safely using an implementation like what I see here: http://www.justsoftwaresolutions.co.uk/threading/implementing-a-thread-safe-... ? Note 1: -= denotes a 1-to-N connect and =- a N-to-1 connect RR = Request reply relationship +--------------------------------------------------------------------+ | | v | Pool A RR Thread Set 1 Router X | PULL-PUSH --==> PULL-read-PUSH ==--> PULL-PUSH --==> PULL Thread Set 2 PUSH \ / +-compress-+ Pool B RR / \ PULL-PUSH --==> PULL PUSH ^ || | || +----------- PUSH-write-PULL <--==++ Pool C Questions: how do I have a separate thread poll on multiple shared message queues? Like, I have a thread pool that both receives requests for buffer handles and that receives old buffer handles to recycle.
On 01/24/2014 04:05 PM, Kenneth Adam Miller wrote:
+--------------------------------------------------------------------+ | | v | Pool A RR Thread Set 1 Router X | PULL-PUSH --==> PULL-read-PUSH ==--> PULL-PUSH --==> PULL Thread Set 2 PUSH \ / +-compress-+ Pool B RR / \ PULL-PUSH --==> PULL PUSH ^ || | || +----------- PUSH-write-PULL <--==++ Pool C
Questions: how do I have a separate thread poll on multiple shared message queues? Like, I have a thread pool that both receives requests for buffer handles and that receives old buffer handles to recycle.
Your architecture looks a like a case of premature optimization. I suggest that you use the Leader/Followers design pattern [1] instead. You have just one thread pool and let each thread perform operation A, B, and C before it returns to recycle the buffer and wait for the next task. This way you can avoid several queue and thread context-switches. [1] http://www.cs.wustl.edu/~schmidt/PDF/lf.pdf
participants (2)
-
Bjorn Reese
-
Kenneth Adam Miller