Sign In/My Account | View Cart  
advertisement

Article:
 Amazon's Simple Queue Service
Subject: RACEfullness
Date: 2005-03-10 17:16:54
From: occupant@heavybit.com

I am designing a RESTful message queue service along these lines. I happened on this article while researching approaches to this. One of my criteria is that the service should offer high performance when there are multiple consumers of one queue. Your implementation seems not to address concurrency at all. If two consumers each request messages at approximately the same time, they will both receive links to the same message, and both will attempt to dequeue the same message. The loser of the race will presumably get an error when it tries to fetch the just-dequeued message. It has just made two HTTP requests and accomplished nothing. Under high load with many consumers, this would equal a meltdown.


To avoid the race condition, the dequeueing of the message at the head of the queue needs to be a one-step operation. But this means it would not be RESTful to use GET to get a message off the queue, because getting a message means making it immediately unavailable to other clients. This bugs me because it feels right to use GET for getting, but here getting is not idempotent.


Previous Message Previous Message   Next Message Next Message


Titles Only Titles Only Newest First
  • RACEfullness
    2005-03-15 10:01:52 jgregorio

    "If two consumers each request messages at approximately the same time, they will both receive links to the same message..."


    That is a function of how 'Read' is implemented on the server. Two different Read operations that come in, no matter how close together, should return two different "Queue Subsets", each one representing a different subset of the queue.





Sponsored By: