Message transformation and publication

For any source service added in the configuration, the framework subscribes to or receives updates from AMQP sources (queues or topics) specified in the sourceName configuration parameter. The type of messages retrieved from such sources can be any AMQP-compliant message type, such as TextMessage, BytesMessage, or MapMessage. The adapter contains a handful of payload converters that can convert such JMS-compliant messages into Diffusion topic types. These converters can be specified in the configuration as shown below, so that the framework uses them to convert the received AMQP messages into Diffusion-specific types and publish them to Diffusion topics.

    {
      "serviceName": "simpleQueueListener",
      "serviceType": "QUEUE_LISTENER",
      "description": "Listens to queue updates",
      "config": {
          "framework": {
            "payloadConverters" : [
              {
                "name": "$BytesMessageToJSONConverter"
              }
            ]
          },
          "application": {
            "amqpServerConnection": {
              "connectionUrl": "amqp://localhost:5672"
            },
            "sourceName": "testQueue",
            "diffusionTopicTemplate": "target/testQueue"
          }
        }
    }

With the above configuration, a message of type BytesMessage received from the configured testQueue queue on the AMQP server will be converted to a Diffusion JSON type and published to the Diffusion topic path target/testQueue. If the received message is of any type other than BytesMessage, a payload converter exception will be thrown.

By default, $TextMessageToStringConverter will be used for any services if the payloadConverters parameter is not specified.

If the payloadConverters parameter is not specified in the configuration but a topicType parameter is explicitly specified, the framework will use a default payload converter to create the specified topic type, which does not support AMQP messages. Therefore, for the AMQP adapter, it is essential to explicitly specify a payload converter that supports AMQP messages if the topicType parameter is specified, unless the default $TextMessageToStringConverter is to be used.

The table below lists all payload converters available in the adapter that can be used for a specific type of AMQP message:

List of available PayloadConverters
AMQP Message type (Input type)	Diffusion topic type (Output type)	Payload converter Name	Detail	Configuration Parameter
TextMessage	String	$TextMessageToStringConverter	This is the default converter used with source services when no payload converter is specified in the configuration. It extracts only the text content of the message, ignoring any headers or properties.	Not required
TextMessage	JSON	$TextMessageToJSONConverter	Only the text content of the message is extracted, while any headers or properties are ignored.	Not required
TextMessage	JSON	$TextMessageToJSONWithPropertiesConverter	The final JSON output will include the text content along with all the headers and properties of the message.	Not required
TextMessage	Binary	$TextMessageToBinaryConverter	Only the text content of the message is extracted and converted to the Binary format, while any headers or properties are ignored.	`charset` - The charset used to encode the text in the message. By default, "UTF-8" is used. If an invalid charset value is provided, the default "UTF-8" will be applied to encode the text content of the message.
BytesMessage	String	$BytesMessageToStringConverter	This converter expects the byte array in the BytesMessage to contain string-encoded bytes, which are then converted to a String. Only the text content of the message is extracted, while any headers or properties are ignored.	`charset` - The charset used to encode the textual content of the message. By default, "UTF-8" is used. If an invalid charset value is provided, the default "UTF-8" will be applied to encode the text content.
BytesMessage	JSON	$BytesMessageToJSONConverter	This converter expects the byte array in the BytesMessage to contain string-encoded bytes, which are then converted to JSON format. Only the text content of the message is extracted, while any headers or properties are ignored.	`charset` - The charset used to encode the textual content of the message. By default, "UTF-8" is used. If an invalid charset value is provided, the default "UTF-8" will be applied to encode the text content.
BytesMessage	JSON	$BytesMessageToJSONWithPropertiesConverter	This converter expects the byte array in the BytesMessage to contain string-encoded bytes, which are then converted to a String. The final JSON output will include the text content along with all the headers and properties of the message.	`charset` - The charset used to encode the textual content of the message. By default, "UTF-8" is used. If an invalid charset value is provided, the default "UTF-8" will be used to encode the text content.
BytesMessage	Binary	$BytesMessageToBinaryConverter	The byte array of the message body will be converted to binary format. Any headers or properties of the message are ignored.	Not required
MapMessage	JSON	$MapMessageToJSONConverter	The output generated by this converter will contain a JSON node with the map contents of the message. Any headers or properties of the message are ignored.	Not required
MapMessage	JSON	$MapMessageToJSONWithPropertiesConverter	The final JSON output will include the text content, along with all the headers and properties of the message.	Not required

As mentioned above, some payload converters, such as $TextMessageToBinaryConverter or $BytesMessageToJSONConverter, can be configured to specify the charset to be used. They can be specified as follows in the framework configuration section:

    {
        "payloadConverters" : [
          {
            "name": "$BytesMessageToJSONConverter",
            "parameters": {
              "charset": "ASCII"
            }
          }
        ]
    }

Converters like $BytesMessageToJSONWithPropertiesConverter and $TextMessageToJSONWithPropertiesConverter will include both the message headers and properties along with the message content in the final JSON output. This output will contain a JSON node with a field named payload, which holds the content of the message, and another field, properties, which contains the headers and properties of the message.

Example 1. Example output from converters that include message headers and properties

    {
        "payload" : "This is the message",
        "properties": {
          "property1": "value1",
          "property2": "value2",
          "property3": "value3"
        }
    }

If this list of available payload converters does not include one that supports your business needs for handling a message, a custom payload converter can be created and used with the adapter. See the details here.

For a deeper understanding of how payload converters are used, please refer here. See here for the list of all issued payload converters by the framework.

Message Publication from a Single Source to Multiple Diffusion Topics

In the diffusionTopicTemplate configuration parameter, it is possible to specify a placeholder for the value of a property or header of an AMQP message using a property or header name in the format ${propertyKey}. For example, if the configuration’s value is 'target/${JMSDestination}/${JMSPriority}/${location}', and a message from a JMS destination 'test_topic' is received with priority '4' and the user-supplied property key 'location' has the value 'london', the resulting Diffusion topic to which the message will be published will be 'target/test_topic/4/london'.

Thus, it is possible for messages from a single AMQP source, if they contain a specific property with different values, to be redirected to multiple Diffusion topics.

However, this will incur a slight overhead in fetching the property value and creating a Diffusion topic for each message.

If fetching the property or header value fails, or is null or empty, the generation of the Diffusion topic will fail. This can indicate that either the Diffusion topic template is not configured correctly, or that the required information to create the Diffusion topic is unavailable. Hence, the service will be paused in such cases.