ActiveMQ Network of Brokers Again (Multi-DC)
Network of Brokers with ActiveMQ
In the past, I've written about the ActiveMQ network of brokers and some of the issues that can come up (see for example: https://www.randomparallels.com/2012/07/network-of-brokers-revisited.html).
That was some years ago, so what is it like now? Overall, it's very good and reliably works between different data centers. It also provides for an important use case: the ability to write to one location and read anywhere. This is write-read is done via the 'local' broker to the apps so that if both the producer and consumer are in the same location, it's done 'locally' without having to commit to all brokers (or a quorum) everywhere first. That fact can have important performance benefits. However, there does seem to be a limit to the number of queues that can be bridged across a network of brokers - I don't know the number or what affects it, but assume a few hundred at most. The good news is that you can have a number of network connectors between two brokers and that means you can have much more than 100s of queues bridged between the same pair of brokers. The key is to create new networkConnectors as needed to handle groups of related (or similarly named) queues or topics. Here's an example of different groups of queues per networkConnector within the networkConnectors part of the ActiveMQ configuration xml:
Here's an example of the network connector config:
Splitting out groups of queues like this allows the brokers to bridge more queues (or topics, the same applies for them) between the brokers without any issues. For another example, look at the Duplex Connector example on the ActiveMQ docs or in this article.
In the past, I've written about the ActiveMQ network of brokers and some of the issues that can come up (see for example: https://www.randomparallels.com/2012/07/network-of-brokers-revisited.html).
That was some years ago, so what is it like now? Overall, it's very good and reliably works between different data centers. It also provides for an important use case: the ability to write to one location and read anywhere. This is write-read is done via the 'local' broker to the apps so that if both the producer and consumer are in the same location, it's done 'locally' without having to commit to all brokers (or a quorum) everywhere first. That fact can have important performance benefits. However, there does seem to be a limit to the number of queues that can be bridged across a network of brokers - I don't know the number or what affects it, but assume a few hundred at most. The good news is that you can have a number of network connectors between two brokers and that means you can have much more than 100s of queues bridged between the same pair of brokers. The key is to create new networkConnectors as needed to handle groups of related (or similarly named) queues or topics. Here's an example of different groups of queues per networkConnector within the networkConnectors part of the ActiveMQ configuration xml:
Here's an example of the network connector config:
<networkConnectors>
<networkConnector name="QUEUE_SET_1" duplex="true" uri="static:(tcp://192.x.x.x:61616)">
<dynamicallyIncludedDestinations>
<queue physicalName="stock.nasdaq.>" />
</dynamicallyIncludedDestinations>
</networkConnector>
<networkConnector name="QUEUE_SET_2" duplex="true" uri="static:(tcp://192.x.x.x:61616)">
<dynamicallyIncludedDestinations>
<queue physicalName="stock.nyse.>"/>
</dynamicallyIncludedDestinations>
</networkConnector>
</networkConnectors>Splitting out groups of queues like this allows the brokers to bridge more queues (or topics, the same applies for them) between the brokers without any issues. For another example, look at the Duplex Connector example on the ActiveMQ docs or in this article.
Comments
Post a Comment