| Interface | Description |
|---|---|
| InboundPayloadConvertor<V,D> |
The interface that an inbound payload convertor implements.
|
| OutboundPayloadConvertor<V,D> |
The interface that an outbound payload convertor implements.
|
| PayloadConvertor<V,D> |
The base interface that a payload convertor implements.
|
Payload convertors may be specified for services within the configuration.
A payload convertor provides conversion of external data values to or from Diffusion data types. A standard convertor could be used by many different services. Alternatively a specialised convertor could be used for some services which as well as converting the value, could also perform any data transformation that may be required.
A payload convertor may be 'inbound', 'outbound' or both.
An inbound convertor converts values from an external data type to a
Diffusion data type and is used to convert values passed to a
Publisher within a
SourceHandler or
HybridHandler.
An outbound convertor converts values from a Diffusion data type to an
external data type and is used before passing values to a
SinkHandler via its
update
method, or to a HybridHandler via its
update
method.
Payload convertor implementations are declared with configuration files in
the resource directory META-INF/services. The file name should
be
META-INF/services/com.diffusiondata.gateway.framework.convertors.InboundPayloadConvertor
for inbound convertors and
META-INF/services/com.diffusiondata.gateway.framework.convertors.OutboundPayloadConvertor
for outbound convertors. Each line within the file declares the full name of
a payload convertor class available on the application classpath. If a
payload convertor is both inbound and outbound then it must be declared in
both files.
Topic Type |
Inbound Conversion | Outbound Conversion |
|---|---|---|
BINARY |
If the value is an instance of
Binary, it is
returned as is.If the value is a byte[], it will be converted to a Binary object.
The caller must not write further data to the array.All other types will throw a PayloadConversionException. |
A byte[] representation of the topic's
Binary value is
returned. This is a byte[] view of the actual value and as such must not be
modified. |
DOUBLE |
If the value is an instance of Double, it is returned as is.If the value is an instance of Number, the product of its
doubleValue method is returned.If the value is null, null will be returned (leading to a null topic value}. For any other type the Double#parseDouble method is used on the
toString value of the supplied value and if the
string value cannot be parsed as a double, a
PayloadConversionException will be thrown. |
A Double value is returned. |
INT64 |
If the value is an instance of Long, it is returned as is.If the value is an instance of Number, the product
of its longValue method is returned.If the value is null, null will be returned (leading to a null topic value}. For any other type the parseLong method is
used on the toString value of the supplied
value and if the string value cannot be parsed as a long, a
PayloadConversionException will be thrown. |
A Long value is returned. |
JSON |
If the value is an instance of
JSON, it is returned
as is.Otherwise the input object type is assumed to contain a JSON string (either it is a String or its toString method returns a JSON String, and the String value is converted to
a JSON object.If the value's toString does not contain valid JSON, or the value is null, a PayloadConversionException is thrown. |
A String is returned which contains the string
representation of the JSON object. |
STRING |
If the value is null, null is returned (leading to a null topic
value). If the value is a String it is returned as is.For any other type the product of its toString method is returned. |
A String is returned. |
A payload convertor must implement either
InboundPayloadConvertor or
OutboundPayloadConvertor depending upon the direction of conversion. A
single class may implement both interfaces to provide a two-way convertor.
A payload convertor must implement the
getName method to define the name by which it is to be referred to in
service configuration. The name should not start with a '$' character as this
is reserved for convertors that are provided by DiffusionData. A convertor
must also implement the
getDiffusionType method which defines the Diffusion topic type that the
service will map to as follows:-
| Convertor type | Topic type |
|---|---|
JSON.class |
JSON |
String.class |
STRING |
Long.class |
INT64 |
Double.class |
DOUBLE |
Binary.class |
BINARY |
An inbound convertor must implement the
toDiffusionType method to convert an external object to a Diffusion typed
object.
An outbound convertor must implement the
fromDiffusionType method to convert a Diffusion typed object to an external
object format.
The Diffusion com.pushtechnology.diffusion.datatype package
(described within the Diffusion Client API) should be used for reading and
writing Diffusion values.
A payload convertor may be called from many threads at the same time and therefore should ideally be stateless and must otherwise be threadsafe.
Payload convertors are called in line when publishing or updating and so must operate synchronously. However, there is no reason why a convertor should not call out to some external service to perform a conversion as long as it blocks awaiting the response.
Simple CSV (Comma Separated Values) payload conversion capability is provided with the product.
For inbound conversion of CSV input that has headers an inbound CSV to JSON
convertor is provided. To use it the class
com.diffusiondata.gateway.convertors.csv.CsvToJSONConvertor
should be specified in the
META-INF/services/com.diffusiondata.gateway.framework.convertors.InboundPayloadConvertor
file and the name
CSV_TO_JSON specified to the source handler's
source service properties. The convertor can take a File or a String as input and convert to a
Diffusion JSON value containing a json array where each element is a json
object corresponding to a line within the input. The input is expected to
have a header row which defines the object field names.
To allow for input that does not have a header row, or to provide some
validation of the values, extend
com.diffusiondata.gateway.convertors.csv.AbstractCsvToJSONConvertor
and implement the getSchema() method to return a
Jackson
CSV schema (com.fasterxml.jackson.dataformat.csv.CsvSchema).
For outbound conversion of JSON input to CSV string an abstract outbound JSON
to CSV convertor
com.diffusiondata.gateway.convertors.csv.AbstractJSONToCsvStringConvertor
is provided. It's subclass should implement the getSchema()
method to return a
Jackson
CSV schema (com.fasterxml.jackson.dataformat.csv.CsvSchema).
This will be used by the convertor to get headers for CSV string.
Simple Apache Avro payload conversion capability is provided with the product.
For inbound conversion of Avro input, an inbound Avro to JSON convertor is
provided. To use it the class
com.diffusiondata.gateway.convertors.avro.AvroToJSONConvertor
should be specified in the
META-INF/services/com.diffusiondata.gateway.framework.convertors.InboundPayloadConvertor
file and the name
AVRO_TO_JSON specified in the source handler's
source service properties. It can take an object of type
org.apache.avro.generic.GenericContainer and convert it to
Diffusion JSON format.
For outbound conversion of JSON input, an outbound JSON to Avro convertor is
provided. To use it the class
com.diffusiondata.gateway.convertors.avro.JSONToAvroConvertor
should be instantiated by sink handler by passing a file path for schema or
schema object and the name
JSON_TO_AVRO specified in the sink handler's
sink service properties. It can take an object of Diffusion JSON format and
convert it to org.apache.avro.generic.GenericContainer.
Copyright © 2023 DiffusionData Limited. All rights reserved.