Skip to content

Commit

Permalink
Merge pull request DIRACGrid#5996 from chaen/8.0_FEAT_stomp8
Browse files Browse the repository at this point in the history
[8.0] Upgrade Stomp version and implement simpler interface
  • Loading branch information
fstagni authored May 12, 2022
2 parents 1acade6 + 153f135 commit 9fbc009
Show file tree
Hide file tree
Showing 5 changed files with 474 additions and 1 deletion.
1 change: 1 addition & 0 deletions docs/source/AdministratorGuide/Resources/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ contributing with their computing and storage capacity, available as conventiona
catalog
computingelements
messagequeues
simpleMQ
storage
agents2CS
proxyprovider
Expand Down
66 changes: 66 additions & 0 deletions docs/source/AdministratorGuide/Resources/simpleMQ.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
=====================
Simple Message Queues
=====================

.. warning ::
Technology preview, will likely change interface
The very abstracted implementation of MQ in DIRAC offers flexibility, but at the cost of complexity and limitations (multiple host behind a broker alias for example). A simpler implementation is proposed, but is still under development. At the moment, this simpler implementation only supports ``Stomp``, but that is the only one used so far.

See :py:mod:`DIRAC.Resources.MessageQueue.Simple.StompInterface` for a more detailed documentation.

How to migrate
==============

The configuration of the MQ can remain the same in the CS.

The difference is in the code.

.. code-block:: python
# Before
from DIRAC.Resources.MessageQueue.MQCommunication import createProducer, createConsumer
# New
from DIRAC.Resources.MessageQueue.Simple.StompInterface import createProducer, createConsumer
The ``mqURI`` should also be changed to just the service name. For example

.. code-block:: python
# Before
result = createProducer("Monitoring::Queues::dirac.monitoring")
# New
result = createProducer("Monitoring", destination="dirac.monitoring")
# or, if there is only one destination defined in the CS
# but this should be avoided
result = createProducer("Monitoring")
The simpler interface can also listen to multiple destinations.

.. code-block:: python
# Before
result = createConsumer("Monitoring::Queues::dirac.monitoring")
# New
result = createConsumer("Monitoring", destinations=["dirac.monitoring"])
# or, if no destination is specified, will listen to ALL the destinations
# in the CS
result = createProducer("Monitoring")
There is however a compatibility layer, such that full ``mqURI`` are still accepted.

Consumer are now driven by ``listener`` class instead of callback functions. Please see :py:mod:`DIRAC.Resources.MessageQueue.Simple.StompInterface` for example on how to use it



.. warning ::
The generic implementation was always doing a ``json.dumps`` before sending, and always doing a ``json.loads`` when upon receiving. The simple interface does not do it. You have to manage the serialization yourself
2 changes: 1 addition & 1 deletion environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ dependencies:
- requests >=2.9.1
- six >=1.10
- sqlalchemy
- stomp.py =4.1.23
- stomp.py
- suds-jurko >=0.6
- xmltodict
- pycurl
Expand Down
Loading

0 comments on commit 9fbc009

Please sign in to comment.