Diffusion 6.10 Release Notes

6.10.10 (13 April 2026)

Fixes in 6.10.10

Apple Client

DIF-5016: EXC_BAD_ACCESS on PTDiffusionSessionState in iOS 26+

Session.state has been made atomic in order to mitigate inconsistent state due to multi-threading.

6.10.9 (14 January 2026)

Fixes in 6.10.9

.NET Client

DIF-4847: Failure to Reconnect : System.ArgumentOutOfRangeException

The .NET client could fail to reconnect to the Diffusion server because the recovery buffer had become corrupted. This has now been resolved.

DIF-5002: Fatal Error From Server Ignored on Reconnection

The .NET client would attempt to reconnect to the Diffusion server even after the server has disconnected the client due to a fatal error, such as message loss on reconnection. This has now been resolved.

Configuration

DIF-3758: NullPointerException in SSLUtils.loadSSLContext if Keystore Password not Supplied

Not supplying a keystore password in the XML configuration would cause a NullPointerException. This has now been resolved.

Federation

DIF-5126: Deadlock involving RemoteServerPrimaryLinkImpl$PrimaryConnection

In rare circumstances, a deadlock could occur in the server involving RemoteServerPrimaryLinkImpl$PrimaryConnection. This has now been resolved.

JavaScript Client

DIF-2280: Multiple Calls to TopicStream.set With the Same Mutated Object Have No Effect After the First Call

Topic update streams would cache the value to calculate diffs, for the updates. For pure JSON objects, these values were wrongly stored by reference. When TopicStream.set was called multiple times with the same mutated object, no updates would be sent to the server. Now, deep copies are stored instead, leading to the expected behaviour.

DIF-2938: FetchRequest.fetch() - Non-errors thrown when given bad topic selectors

In the case that FetchRequest.fetch failed due to a server error, the promise was rejected with a non-specified type. This has been corrected and it will now be rejected with a RuntimeError.

DIF-4517: Passing SessionId Object into Functions that take SessionId or String Fails in the Modular Client

Functions such as ClientControl.close may take a SessionId or a string as parameter. When using the modular JS client, these functions sent the wrong data to the server when a SessionId was passed. This has been fixed and SessionId instances can be passed successfully.

DIF-4808: Calling setSessionProperties With a Map Does Not Correctly Set Session Properties

Calling ClientControl.setSessionProperties with a Map object instead of a plain JSON object incorrectly set the session properties. The key and value pairs were reversed. This has been fixed.

DIF-4815: Memory Overflow when Many Topic Selectors Used

The topic selector cache has now been size-limited to avoid memory overflow.

Persistence

DIF-2613: Persistence Compaction on Restore is Inefficient

When a Diffusion Server is restored from persistence, a compacted file is rewritten. Previously, the compaction mechanism was performing individual writes to the file, which was inefficient and led to slower restores. Compaction now batches writes so that the performance of restore is significantly improved.

Python Client

DIF-5134: CBOR2 5.8.0 or Later Breaks Client

CBOR2>=5.8.0 breaks the client by consuming the entire buffer passed into it: https://github.com/agronholm/cbor2/issues/272 .

There are various partial workarounds but the author has confirmed this is a bug. Until this is rectified, or we migrate from using cbor2 for decoding we will continue to restrict cbor2 to <5.8.0.

Server

DIF-4487: Zero Length Value Dumps are Produced, and Dumps Directory is Never Tidied

When an invalid CBOR value is passed as an update to a topic, the value is dumped to a file in the persistence/dumps directory. However, this occurred even if the value was empty, which is pointless. Additionally, the number of these dump files can grow indefinitely, potentially leading to file resource failures. At this release, if an empty value is passed, a warning is logged, but no dump file is produced. In addition, there is now a tidying mechanism that ensures that when there are more than 50 of these dump files, any that are more than a week old will be removed.

DIF-4971: Server Runs Out of Native Memory when Topic Persistence is Enabled

In previous releases, if topic persistence was enabled, the server could reserve excessive native memory when persisting topic updates with large payloads. This was due to sub-optimal caching of native byte buffers and could lead to out of memory errors. The problem has been fixed in this release.

DIF-4976: Possible Infinite Loop Resulting from a Fetch Request

In extremely rare circumstances, a fetch request could trigger an infinite loop in a server thread. Diagnostics would show the server consuming excessive CPU in the EnumConverter.longToSet method. This issue has now been resolved.

6.10.8 (9 July 2025)

Fixes in 6.10.8

.NET Client

DIF-4368: Corrupt Data when Using UpdateStreams

The Dotnet SDK client did not store messages sent to the client during the reconnection phase. Therefore on successful reconnection these messages were lost. This bug has been fixed in this release.

C Client

DIF-3850: Memory leaks in C SDk

Several internal memory leaks in the C SDK have been detected and fixed.

DIF-4301: Transport Loop Not Breaking on Invalid Host

When providing an invalid hostname to the C SDK the transport layer could hang indefinitely when closing. This issue has now been resolved.

Federation

DIF-4098: FanOut Secondary Server Fails to Recover when Closed Due to Data Loss

In exceptionally rare circumstances, a secondary fanout server that lost its connection due to data loss (for example, as caused by the problem reported in DIF-2866) would fail to reconnect. This problem has now been resolved.

Java Client

DIF-4346: Java Session Does Not Wait for the Server to Acknowledge Session Following Failover and Reconnection

The Session.close() operation waits for a short grace period (default 1 second) for the server acknowledgement before shutting down the connection, improving the likelihood that the closure notification has been delivered to the server and the server has correctly recorded the reason for the session closure. Due to a bug in previous releases, the grace period was not applied if the session was closed following fail over and reconnection. This has been fixed in this release.

JavaScript Client

DIF-3702: ErrorReportList is stringified incorrectly

The toString() function of ErrorReportList was not implemented. This resulted in a messages containing '[Object]'. The function has now been implemented to return a meaningful message and errors based on ErrorReportList contain more meaningful information.

DIF-4082: Javascript docs are not built properly

The documentation for the JS client API was not generated properly. This has been fixed.

DIF-4379: Incorrect ClientCloseReason enum Values

The numeric ClientCloseReason enum values for CLOSED_BY_CONTROLLER and FAILED_OVER were incorrect and did not match those returned by the server. They have been updated to the correct values.

Python Client

DIF-2127: Incorrect set deserialisation

Set data structures were being deserialised incorrectly by fetching more elements than were declared for a set. This only manifested when there was more data in a protocol message after the given set, otherwise, deserialisation ends when there is no more data. The only known, currently implemented Python Client feature this could have adversely affected is list_session_metric_collector - due to group_by_properties being a set of String objects.

DIF-3217: Non Messaging APIs are not Concurrency-safe

This fix migrates internal serialisation to use concurrency-safe methods. .

DIF-3258: Messaging API is not Concurrency-safe

The Messaging APIs were previously not entirely safe for concurrent calls. This is resolved in this release.

DIF-3604: Structlog Configuration is Overriden by Client, and Exceptions are not Logged Correctly

This fix prevents Diffusion from overriding structlog settings. It also logs exceptions correctly according to the structlog API.

DIF-3844: ValueStreamHandler being Triggered when Selector Should Not Match

ValueStreamHandlers were being triggered regardless of their associated topic selector when no topic-specific selector was available - effectively, all datatype-compatible ValueStreamHandlers were being treated as fallback handlers. This has now been resolved.

Please check that any ValueStreamHandlers are associated with the intended topic selectors. This brings the behaviour into line with the other clients.

Security

DIF-3755: Javascript SDK security update

Vulnerabilities were detected in the JS SDK via its dependencies and their versions have been updated.

DIF-4032: TopicNotifications Feature Notifies Topics that the Caller does not have READ_TOPIC Permission for

In previous releases, the Topic Notifications feature notified the listener of all topics that satisfy the specified selector, even though the documentation states that the listener will only receive notifications for topics for which it has READ_TOPIC permissions. This has now been resolved, and the listener will only receive notifications for topics for which it has READ_TOPIC permissions.

Server

DIF-3630: SubscriptionControl methods do not validate SELECT_TOPIC permissions correctly

The SubscriptionControl feature documents that the caller must have SELECT_TOPIC permissions to the path prefix of the selector being used to subscribe other sessions. However, in previous releases, this permission check was only being applied when subscribing using a filter and not when subscribing a single session. This has been resolved in this release.
In addition, the same SELECT_TOPIC permission check should have been applied when unsubscribing sessions. From release 6.12, the same checks will be applied when unsubscribing sessions.

Topic Views

DIF-3499: Unable to Set the Whole Value of a Topic Using the Process Clause

In previous releases, it was not possible to specify an empty JSON pointer (as permitted by the JSON Pointer Specification RFC6901) within the Topic Views 'process' clause (would fail to parse the specification). This meant that it was not possible to set the whole output value to a specified value. This has now been resolved so that leaving the pointer parameter as empty indicates the whole JSON document - for example set(,"a string").

DIF-3534: Deadlock in Diffusion Server when using Delayed Topic Views

In previous releases, there was a possibility of a thread deadlock between UniversalTopicImpl and DerivedReferenceTopicImpl when using delayed topic views. This has been resolved in this release.

6.10.7 (21 January 2025)

Fixes in 6.10.7

JavaScript Client

DIF-3135: The roles parameter in SystemAuthenticationScriptBuilder.allowAnonymousConnections should be optional

While the roles parameter in SystemAuthenticationScriptBuilder.allowAnonymousConnections(roles) is specified as optional, calling the function without parameters resulted in an exception. This has been fixed, and the roles parameter can now be left undefined, defaulting to the empty array, as specified in the API documentation.

Product Release

DIF-3473: The self-signed sample certificate has expired

The sample self-signed certificate packaged in previous releases expired on 16th January 2025. In this release, the certificate has been refreshed with a new certificate that expires on 18th January 2035. The installer places the certificate in etc/sample.keystore. The certificate is not secure and should be replaced in production installations with a customer provided certificate.

Python Client

DIF-3380: Pydantic 1.10.20 breaks ForwardRefs on Python 3.12.4 (ForwardRef._evaluate() got an unexpected keyword argument 'type_params')

For Pydantic V1 users, there was a regression in Pydantic 1.10.20 as described here: https://github.com/pydantic/pydantic/issues/11230.

This change blacklists this version with the expectation that the regression will be fixed in an imminent Pydantic 1.x release.

Replication

DIF-3235: Replicated time series topic memory leak

In previous releases, expired events were not properly removed from replicated time series topics, causing a server memory leak. The problem has been fixed in this release.

DIF-3350: Assertion failure when recovering a replicated time series topic from file

Due to a bug in previous releases, a server could with an assertion error when recovering a replicated time series topic from file. The bug has been fixed in this release.

Topics

DIF-3359: Automatic Topic Removal with an "or" operator in the expression does not respect one of the clauses if another is "subscriptions less than" AND there are subscriptions to the topic

In previous releases, if an automatic topic removal clause specified 'subscriptions less than' with an 'or' operator where the or'd expression is satisfied AND the topic in question had subscribers the topic would not be removed. This was because the 'subscriptions less than' expression was wrongly taking priority over the satisfied 'or' expression. This problem has now been resolved.

6.10.6 (26 November 2024)

Fixes in 6.10.6

Federation

DIF-998: Problems with Propagation of Missing Topic Notifications over Remote Server Connections

Missing topic notifications were being propagated from a secondary server to a primary server even if the subscription selector used did match topics on the secondary. A notification should only be sent to the primary server if the selector matches no topics on the secondary server. In addition, when a notification was received on the primary server it would only be propagated onwards if the selector matched no topics on the primary server, where it should be propagated regardless of any matches on the primary server. These problems have been resolved in this release.

Replication

DIF-3016: Memory leak due to never-ending retry of replicated topic removal

In previous releases, if a cluster failure caused removal of replicated topics to be aborted, the operation would be repeatedly rescheduled by the coordinating server. If the cluster failure remained unresolved, the Java heap memory of the coordinating server would eventually be exhausted. In this release, the rescheduling code has been re-written to have a constant memory footprint and to eventually abort the removal operation if the cluster does not recover.

DIF-3034: Warnings Logged when Hazelcast is Shut Down

Unnecessary HazelcastInstanceNotActiveException messages can be logged when a server is stopped if they are not individually handled by Diffusion code. In this release, further instances have been suppressed.

Server

DIF-2832: Spurious PUSH-000773 multiplexer slow progress warnings

Spurious PUSH-000773 Multiplexer slow progress warnings could be logged with mostly idle servers, from this release this is fixed.

DIF-2907: Poor topic recovery performance in a cluster

In previous releases, replication of a large number of low priority topics could take an unnecessarily long time and cause multiplexer slow progress warnings to be logged. The problem has been fixed in this release.

DIF-3056: Deadlock in DiffusionMetricRegistryImpl during Server Shutdown

In very rare circumstances a thread deadlock could occur between the classes MeasuredEntityClassMetricsCollator and DiffusionMetricRegistryImpl. This problem has now been resolved.

6.10.5 (7 November 2024)

Fixes in 6.10.5

C Client

DIF-2538: API_ERROR_DIFFUSION_VALUE_INCORRECT_DATATYPE when reading double value

When using read_diffusion_double_value to read a DIFFUSION_VALUE_T containing a double value, it would return API_ERROR_DIFFUSION_VALUE_INCORRECT_DATATYPE. This issue has now been fixed.

DIF-58: Missing on_subscription callback for stream causes SEGV

When on_subscription or on_value handlers are omitted from a value stream, a segmentation fault would occur. This issue has now been resolved. .

Federation

DIF-2831: Possible thread deadlock during server startup or closedown when remote servers are in use

In rare circumstances, a thread deadlock could occur when starting or closing a server when remote servers are in use. A thread dump will show a deadlock between the class DefaultPicoContainer and one or more other remote server classes. This has now been resolved.

Java Client

DIF-1975: NullPointerException warning logged by HTTP long polling session

In previous releases, a Java session using HTTP long polling could log a NullPointerException when failing to establish a connection. The problem has been fixed in this release.

DIF-2740: Deadlock when using RecoverableUpdateStream and set fails

In previous versions a deadlock could occur when using recoverable update streams. This has been resolved at this release.

JavaScript Client

DIF-2528: diffusion.connect never resolves on Safari browser

Calling diffusion.connect with invalid credentials would never resolve on Safari. This has been fixed and the Promise will now reject as expected.

Logging

DIF-2532: Errors logged on server closedown

Error messages could be logged in the server log from the class FileRecordWriterImpl when the server is closing down. This has now been resolved.

Python Client

DIF-1062: Diffusion Core shows console noise on console at startup

Coverage instrumentation code had accidentally been left in the release version of diffusion-core, occasionally causing warnings to be emitted on stdout/stderr. This new version removes this instrumentation code.

DIF-2054: Setuptools no longer accepts 'tools.setuptools.dynamic.readme.content_type' key, change to 'content-type'

Setuptools no longer accepts 'tools.setuptools.dynamic.readme.content_type' key, which was being used in earlier version of the Diffusion Python Client. This has been changed to 'content-type' as per Setuptools' requirements. This should only affect build processes rather than end user experience, as pip installs the generated wheel rather than relying on the pyproject.toml, nor does it need the README.MD, but this bug report is included for completeness.

DIF-2550: 'no running event loop' due to API breakage in aiohttp

Fixes the 'no running event loop' issue due to breaking behavioural changes backported into aiohttp 3.10 (see https://github.com/aio-libs/aiohttp/issues/8555).

Replication

DIF-2575: Shutting down a server can discard writes to the file recovery log

In previous releases, the most recent persistent file store writes could be lost when a server is shut down, resulting in lost topic data. This has been fixed in this release.

DIF-2887: Possible topic loss if cluster topology changes while a server is recovering file persisted topics

Due to a bug in previous releases, topic data could be lost if the cluster topology changed while a server was recovering file persisted topics. The bug has been fixed in this release.

Security

DIF-1123: Permissions not correctly reevaluated on removal

There was a problem where in certain circumstances the removal of path permissions from a role was not correctly re-evaluated. This has now been resolved.

DIF-2287: Role permissions not correctly updated for inherited roles

There was a problem with inherited roles whereby if the permissions of a role were changed whilst the server was running, the permissions applied to other roles that included that role would not be re-evaluated until the server was restarted. This has now been resolved.

Server

DIF-1962: Session location properties not provided when connecting through proxies

In previous releases if a client connected through a proxy or load balancer the CLIENT_IP session property could be set to a comma-delimited string of IP addresses. This in turn led to the location properties (COUNTRY, LANGUAGE etc) not being set. The CLIENT_IP session property will now be set to the originating client as documented, thus resolving this issue.

DIF-2294: Possible thread deadlock in RemoteServerSubscriptionQueueImpl

In previous releases, a thread deadlock could occur between RemoteServerTopicManagerImpl/RemoteServerSubscriptionQueueImpl when closing down. This has now been resolved.

DIF-2517: DiffusionQuorumStartupCheck does not always log when quorum is met

The log message (PUSH-000753) which is logged when a quorum has been met would sometimes not be logged if the quorum was met immediately. This has now been resolved.

DIF-2767: Subscriptions are unnecessarily re-evaluated on reconnection

Previously, on reconnection, a session's subscriptions were always being re-evaluated even if the session's properties had not changed. This led to an unnecessary CPU load on the server. This has now been resolved - subscriptions are only re-evaluated if the session's properties change during reconnection.

DIF-2781: Topic set selectors are incorrectly combined with existing selectors

In previous releases, set selectors were incorrectly merged with existing selectors, potentially incorrectly subscribing sessions to topics. This could also have a serious performance impact when using set selectors. The bug has been fixed in this release.

DIF-2866: NullPointerException in multiplexer thread

Due to a bug introduced in 6.10, adding topics that are replicated across a cluster was incorrectly coordinated with adding or modifying topic views. This could cause corruption of the servers internal state, and lead to NullPointerExceptions. The bug has been fixed in this release.

DIF-2895: Deadlock in Server when using Throttled Topics

A thread deadlock between classes ThrottlerImpl and DerivedReferenceTopicImpl could occur when using throttled topics. This has now been resolved.

Topic Views

DIF-1854: Deadlock between UniversalTopicImpl and DerivedReferenceTopicImpl when using topic views that create reference topics as children of source topics

In certain circumstances, a thread deadlock could occur between UniversalTopicImpl and DerivedReferenceTopicImpl. The most likely situation to provoke this would be a topic view that creates a reference topic that is a child of the source topic being processed by the view. This has been resolved in this release.

DIF-2396: Deadlock between UniversalTopicImpl and DerivedReferenceTopicImpl when using Throttle in topic views

There was the possibility of a thread deadlock occurring between UniversalTopicImpl and DerivedReferenceTopicImpl when using a topic view with throttling applied if the throttle being released coincided exactly with an update to the source topic. This has now been resolved.

Topics

DIF-2994: ClassCastException when performing a Time Series Range Query on a non existent Topic

If an attempt was made to perform a time series range query on a topic that does not exist but where there were topics below it in the tree would cause a ClassCastException to be reported in the server log and no response returned to the client. For example, requesting a range query on a non-existent topic called 'Test' where a topic called 'Test/Topic' exists would result in this failure. This has been resolved at this release.

6.10.4 (7 May 2024)

Fixes in 6.10.4

Java Client

DIF-1826: Invalid data errors with a shared value stream cause IllegalStateExceptions

Invalid data received for a topic is reported to the application using the ValueStream.onError() callback. Invalid data errors occur if the update API sends incompatible binary data to a topic that has its VALIDATE_VALUES topic property set to false, or due to a bug in Diffusion (there is no known such bug at this time). .

Streams can be associated with multiple topic subscriptions. If onError() is called for a topic, no further events will be delivered to the stream, even for other topics. Due to a bug in previous releases, if an invalid data error occurred for a topic, subsequent events for the other topics would cause an IllegalStateException to be logged as an uncaught exception. The bug has been fixed in this release.

Server

DIF-1546: Topic removal service can return incorrect number of topics removed

In previous releases there were circumstances in which the server could report that the user had removed more topics than had actually been removed. This has been resolved at this release.

DIF-1759: Deadlock when session closed prevents multiplexer processing

Due to a bug in previous releases, session close processing could deadlock the server. The bug has been fixed in this release.

DIF-1853: Failure to acquire SessionLock

Due to a bug in previous releases, a server belonging to a cluster could have incorrect session lock state, preventing sessions from acquiring session locks that should be available. The bug has been fixed in this release.

DIF-1856: NullPointerException in Server when empty string supplied as separator to Topic View

In previous releases, it was not possible to use an empty string for a topic view separator clause. Doing so would provoke a NullPointerException in the server logs. This has been fixed at this release.

DIF-1874: Performance issues when sessions are repeatedly closed with a large topic tree

In previous releases, repeatedly opening and closing sessions against a server with a large number of topics caused a unnecessary processing to be carried out by the server multiplexer threads, leading to poor system performance. The processing has been optimised in this release.

Topic Views

DIF-1850: Inefficient topic view indexing

In previous releases, Topic View indexing interned data which was inefficient. This has been resolved in this release.

DIF-1877: Topic view inefficiencies when using the insert clause

The processing of topic views with 'insert' clauses after 'expand' clauses or many 'insert' clauses could lead to excessive CPU and memory utilisation. This problem has now been resolved.

6.10.3 (8 April 2024)

Fixes in 6.10.3

Apple Client

DIF-1560: Apple SDK unable to decode 16 Bit Float

The Apple SDK raised "Parsing of type 25 is unhandled." when attempting to parse a 16-bit floating point. This issue has now been resolved.

DIF-1642: iOS SDK Stability of PTDiffusionRequestStreamRegistry and PTDiffusionTopicRoutingStreamRegistry

Several improvements were made to PTDiffusionRequestStreamRegistry and PTDiffusionTopicRoutingStreamRegistry.

DIF-1710: iOS crash in PTDiffusionConversation openWithConversationId:

NSInternalInconsistencyException was being raised in PTDiffusionConversation when the session disconnected while the App was in the background. This issue has been resolved.

Gateway

DIF-1595: Connected Gateway Adapters only showing in a Console connected to one Cluster Member

Connection of Gateway Adapters was only showing in a console connected to one member of a cluster and not the others. Adapters would only be visible on a console that was connected to the cluster member that was the partition 0 owner. This problem has now been resolved and adapters will be visible on a console connected to any member of a cluster.

Persistence

DIF-1536: Edit events of persisted replicated time series topics are not recovered from file persistence

Due to a bug in previous releases, edit events of replicated time series topics would not be correctly recovered from file persistence. If a cluster was restarted, the edit events and their corresponding original events would be silently discarded. The bug has been fixed in this release.

Server

DIF-1574: Memory leak caused by session data retention

Over some time it could be observed that memory usage was increasing and class histograms showed client session-related objects (e.g. ClientInfo and SessionData) numbering far more than the number of connected sessions. This problem has now been resolved.

Topic Views

DIF-1537: Time series topic edit events are not correctly mapped through topic views

Due to a bug in previous releases, time series edit events could fail to be reflected through some types of topic view. The bug has been fixed in this release.

6.10.2 (14 February 2024)

Fixes in 6.10.2

.NET Client

DIF-1177: Client ignores maximum message size set by user

The Dotnet client ignored the maximum message size set by the user. This has been fixed.

DIF-1196: GetSessionProperties does not throw NoSuchSessionException

Calling GetSessionProperties for a session that has been closed did not throw NoSuchSessionException. This has been fixed.

DIF-238: Authentication Control example throws exception before completing

If SetAuthenticationHandlerAsync is called with await then creating a session throws SessionEstablishmentTransientException before the authenticator is called. This has been fixed.

DIF-63: SessionClosedException thrown when not in closed state

If timeout occurs during reconnection the client can hang. This has been fixed.
A connection attempt does not timeout if there is no response from the server. This has been fixed.
Connection and reconnection failure will transition the session state to CLOSED_FAILED.

Apple Client

DIF-779: Missing Session State Listener

The Session State Listener was missing from PTDiffusionSessionConfiguration. This issue has been fixed.

DIF-797: Invalid URL string when opening a session in the Apple SDK raises uncaught NSInvalidArgumentException

When an invalid URL string is passed when opening a session a NSInvalidArgumentException is raised.
In Objective-C we can trap it, but in Swift it’s not possible resulting in an uncaught exception that crashes the application. This has been fixed.

DIF-847: Reconnection Timeout closes successfully recovered connection

When a successful reconnection occurred, the reconnection timeout was not properly cleared and resulted in a disconnection. This issue has been resolved.

Console

DIF-178: Confusing displays for licences without connection limits

Licences without hard connection limits could previously display an incorrect message on the network overview tab of the console.

DIF-180: Topic views errors do not show the correct column position

The console topic view editor provided incorrect column information when presenting syntax error feedback for topic views.

DIF-792: Metrics table can fail to pad collector metrics correctly

When a metrics table is displaying metrics from multiple collectors with different numbers of dimensions, rows for those collectors with fewer dimensions are incorrectly shifted left by one or more columns.

Logging

DIF-834: Existing PUSH message numbers changed at 6.10

Some of the PUSH message numbers, as documented in the user manual changed their meaning between 6.9 and 6.10. For example, PUSH-000882 had a different meaning at 6.10 than at 6.9. All numbers from PUSH-000880 were incorrectly assigned.
This has now been resolved.

Python Client

DIF-1162: ListEncoder.read uses Byte for length instead of Int32

This fix resolves an issue with list serialisation in the Python Client, due to the representation of length being given by a byte rather than a 32 bit integer, preventing lists larger than 255 items being serialised or deserialised correctly. The features formerly affected include: Topic/Session Metric Collector lists, Conjunction Constraints lists, Branch Mapping lists, Range Query event lists, Error Report lists and Session Tree branch lists at the time of discovering this bug. .

DIF-30: No support for Python 3.10-12

The Python client now supports Python versions 3.8-3.12 on MacOS (arm64 and x86_64), Windows and any x86_64 Linux distribution supported by the Manylinux project (https://github.com/pypa/manylinux).

DIF-33: Python SDK does not support SSL context for WSS

Previously the Python SDK did not support setting the SSL context for a WebSocket secure connection. This has now been resolved.

Server

DIF-1023: Corruption of client queue index can cause unnecessary CPU use

Due to a bug in previous releases, long lived sessions that subscribed to many different topics could perform unnecessary processing. The bug has been fixed in this release.

DIF-849: The UNSUBSCRIBE conflation policy does not handle branch mapping subscriptions correctly

Due to a bug in previous releases, the UNSUBSCRIBE conflation policy did not correctly handle branch mapping subscriptions. This has been fixed in this release.

DIF-917: Server can fail to accept updates at a high rate

Due to a bug in previous releases, a server could fail to accept updates at a high rate (more than tens of thousands of updates a second). The bug has been fixed in this release.

Session Trees

DIF-608: Range queries fail on timeseries topics exposed through session trees

An attempt to perform a range query on a topic exposed by session trees would fail with "Range query failed because topic 'xxx' does not exist.". This has now been resolved so that range queries can be performed on session tree time series topics.

6.10.1 (12 October 2023)

Fixes in 6.10.1

.NET Client

DIF-36: Unable to use 'and' method with two value constraints

When using the update constraints API there was an issue where it was not possible to use 'and' composition with two value comparison constraints. For example, it was not possible to say 'value GT 5' and ' value LT 10'. This has now been resolved. The relaxation of validation does mean that it is now technically possible to define a constraint that could never evaluate to true - for example 'value EQ 5' and 'value NE 5' so it is up to the user to ensure constraints are possible.

Apple Client

DIF-36: Unable to use 'and' method with two value constraints

When using the update constraints API there was an issue where it was not possible to use 'and' composition with two value comparison constraints. For example, it was not possible to say 'value GT 5' and ' value LT 10'. This has now been resolved. The relaxation of validation does mean that it is now technically possible to define a constraint that could never evaluate to true - for example 'value EQ 5' and 'value NE 5' so it is up to the user to ensure constraints are possible.

DIF-5016: EXC_BAD_ACCESS on PTDiffusionSessionState in iOS 26+

Session.state has been made atomic in order to mitigate inconsistent state due to multi-threading.

C Client

DIF-36: Unable to use 'and' method with two value constraints

When using the update constraints API there was an issue where it was not possible to use 'and' composition with two value comparison constraints. For example, it was not possible to say 'value GT 5' and ' value LT 10'. This has now been resolved. The relaxation of validation does mean that it is now technically possible to define a constraint that could never evaluate to true - for example 'value EQ 5' and 'value NE 5' so it is up to the user to ensure constraints are possible.

Configuration

DIF-146: Difficulty Setting Maximum Message Size for Docker Containers

It is often a requirement to set the maximum message size to something other than the default, especially when using Docker containers. It is now possible to override the Server.xml maximum message size by setting a system property or environment variable called diffusion.max.message.size to the required maximum message size specified in bytes.

Java Client

DIF-36: Unable to use 'and' method with two value constraints

When using the update constraints API there was an issue where it was not possible to use 'and' composition with two value comparison constraints. For example, it was not possible to say 'value GT 5' and ' value LT 10'. This has now been resolved. The relaxation of validation does mean that it is now technically possible to define a constraint that could never evaluate to true - for example 'value EQ 5' and 'value NE 5' so it is up to the user to ensure constraints are possible.

JavaScript Client

DIF-20: Javascript API has no mechanism for setting trusted certificates

Additional options have been added to diffusion.connect, allowing modification of the underlying TLS configuration. The new options are passed directly to tls.createSecureContext().

DIF-36: Unable to use 'and' method with two value constraints

When using the update constraints API there was an issue where it was not possible to use 'and' composition with two value comparison constraints. For example, it was not possible to say 'value GT 5' and ' value LT 10'. This has now been resolved. The relaxation of validation does mean that it is now technically possible to define a constraint that could never evaluate to true - for example 'value EQ 5' and 'value NE 5' so it is up to the user to ensure constraints are possible.

DIF-479: Shared session has initialising state immediately after connecting

Shared sessions would return the connected session before updating the state to connected. This could lead to a new session to appear to still be in the initialising state. The order of internal state updates has been corrected. A new session will now reliably be in connected state.

Replication

DIF-149: Error while handling a multiplexer event - IllegalStateException: Has already connected

In previous release, when new servers joined a cluster, a server could log a stack trace with the error message "IllegalStateException: Has already connected". This is harmless, but overly concerning. From this release, the error message will no longer be logged.

DIF-404: Thread gets blocked when another cluster member is killed

Fixed a rare issue with non-blocking replication that could cause a server to block new topic updates indefinitely.

Server

DIF-513: Update CAS failure after killing cluster member

When using a cluster with a quorum a Diffusion instance could reject further updates if another instance was restarted. This issue has now been resolved.

DIF-556: Multiplexers can encounter errors when topics are updated during partition recovery

In a rare situation during cluster reconfiguration an update to a replicated topic could fail to be sent to clients. This has been fixed.

6.10.0 (30 August 2023)

Improvements in 6.10.0

.NET Client

FB-29884: Remote Server support for reverse connections

The remote server feature now supports the creation of remote servers for reverse connections (introduced in release 6.9), that is connections initiated from a primary server rather than from a secondary server.

FB-30210: SessionIdFromString method

The .NET client can now create a session ID from a string with the SessionIdFromString method.

Apple Client

FB-29881: Remote Server support for reverse connections

The remote server feature now supports the creation of remote servers for reverse connections (introduced in release 6.9), that is connections initiated from a primary server rather than from a secondary server.

C Client

FB-29766: Removal of unused message types

Several unused and deprecated message types have been removed from the C Client.

FB-29882: Remote Server support for reverse connections

The remote server feature now supports the creation of remote servers for reverse connections (introduced in release 6.9), that is connections initiated from a primary server rather than from a secondary server.

Client

FB-28527: Ability to retrieve metrics using the API

At this release, there is a new API for retrieving metrics. This is enabled in the Metrics feature. This is available in the Java, .NET, and JavaScript SDKs.

FB-28682: Update Stream Recovery

The update stream builder now allows a RecoverableUpdateStream to be created that can be used to recover automatically from transient clustering exceptions.

FB-28999: TopicAddFailReason INCOMPATIBLE_PARENT and IncompatibleParentTopicException Removed

The deprecated TopicAddFailReason.INCOMPATIBLE_PARENT and associated IncompatibleParentTopicException have been removed at this release from the Java SDK. Equivalents have also been removed from all other SDKs.

FB-29000: TopicPermission and associated methods removed

TopicPermission was deprecated at release 6.5 in favor of PathPermission. It has been removed at this release along with all associated methods.

FB-30017: Conditional update constraints

At this release the UpdateConstraint API has been extended to allow for conditional constraints which can check the value of a topic or JSON field using operators like GT (greater than), LT (less than) etc. This functionality is available in all client SDKs other than Python (for a future release).

FB-30328: OR operator for Update Constraints

It is now possible to form compound update constraints using an 'or' operator as well as 'and'. This functionality is available in all SDKs except Python at this release.

Console

FB-30139: Improvements to permissions handling in the console

The Diffusion management console now behaves more predictably and provides more user feedback when attempting to browse a topic tree with restricted permissions at the root.

Gateway

FB-28152: REST API to Manage Gateway Framework

In this release, we have introduced the Gateway Control REST API, a powerful feature that allows for the management and monitoring of Gateway applications through REST API calls. Prior to this release, Gateway applications could only be managed through the Diffusion management console. With the new REST API, all operations previously available exclusively in the console can now be performed using REST calls. For more information, please refer to the updated manual.

Java & Android Client

FB-30025: Java Client and Diffusion server now use HTTP/1.0 standard headers

The Java client and the Diffusion server now follow HTTP/1.0 standards, including an ASCII space following the colon in HTTP headers. Other Diffusion clients already follow this practice and are unchanged.

Java Client

FB-29772: The SLF4j logging API library has been updated to SLF4j 2.0

The SLF4j logging API library has been updated to SLF4j 2.0.

Applications that use the 'with-dependencies' version of the Java client may need to update their logging library to a version compatible with SLF4J 2.0. See https://www.slf4j.org/faq.html#changesInVersion200.

Python Client

FB-26147: Topic update using constraints

The Python SDK is now capable of updating topics using update constraints.

FB-29743: Make serialiser resolution order-invariant, test without caches


Replication

FB-30726: Hazelcast upgraded to 5.2.4

Due to security vulnerabilities in earlier versions, Hazelcast has been upgraded to release 5.2.4.

System Monitoring/Statistics

FB-28456: Metrics Improvements for 6.10


Topic Views

FB-29798: Floating point support in topic views

Topic views now support the use of floating point numbers in all clauses, particularly in process calculations, etc. This includes the ability to specify the scale (number of places after the decimal point) and the rounding policy to use for calculation results.

For full details see the Topic Views page in the user manual.

Topics

FB-29802: Routing Topics Removed

ROUTING topics, which were deprecated in release 6.7, have been removed at this release. There is no longer support for routing topics in the server. Users who were using routing topics should now use the more powerful 'Session trees' feature.

Fixes in 6.10.0

.NET Client

FB-29978: NoReconnection does not correctly disable reconnection

Addresses an issue where setting NoReconnection on a .NET session did not disable reconnection. This has now been resolved.

FB-30284: .NET client incorrectly accepts empty string as valid JSON

The dotnet client incorrectly accepted empty strings as valid JSON. This has now been fixed.

FB-30740: Unable to connect successfully through a load balancer

Addresses an issue where the .NET client was unable to connect through a GCP (Google Cloud Platform) load balancer.

Apple Client

FB-30396: Frequent crash in Diffusion iOS client library

We have addressed a bug that crashed the Diffusion Apple Client in PTDiffusionRequestStreamRegistry.
This issue is now resolved.

C Client

FB-29801: Session properties listener seg faults when an update event is received

Addresses an issue where a C client application could crash (segfault) if a session close event was received in a session properties listener.

Federation

FB-29642: Remote server connections can block indefinitely

Networking issues between clustered Diffusion instances could cause the server to stall.

This has been fixed in this release.

Java & Android Client

FB-30219: Some API calls return ExecutionException rather than CompletionException

All API calls that return a CompletableFuture should fail exceptionally with a CompletionException and not ExecutionException. However, at the previous release, there were some cases where this was not true including:

1) When UpdateStream methods were returning InvalidUpdateStreamException.

2) Calls to Metrics.removeSessionMetricCollector, Metrics.removeTopicMetricCollector, and Metrics.putTopicMetricCollector.

JavaScript Client

FB-29448: Transport level error does not notify session establishment promise

In Safari 16.0, there is an experimental feature called "NSURLSession WebSocket". With this enabled, any JS API session failure will terminate the transport level websocket connection, resulting in errors. However, such errors were not being propagated back to the client's session establishment promise, leading to session establishment never completing. This problem may also exist in other browsers.
The problem has been resolved in this release.

FB-29729: MutableRecordModel.add not working as expected

MutableRecordModel.add was not working as expected and records were not added. This has been fixed.

FB-30327: Incorrect close reason type on SessionPropertiesListener.onSessionClose

The TypeScript definition of SessionPropertiesListener.onSessionClose() did not specify the correct type for the close reason. A new ClientCloseReason type has been introduced to the signature of the callback.

Logging

FB-30400: Compaction corruption messages logged at ERROR level

Log messages that detect a persistence file corruption were previously being logged at ERROR level, even though the problems are automatically corrected. These messages are now logged at WARN level and have PUSH code messages.

Persistence

FB-30715: Topic persistence operations could fail silently

Fixed an issue that could cause commit of topic operations to fail silently.

Replication

FB-29959: Cluster repartitioning during a request to add a topic can cause a the request to hang

If cluster repartitioning occurred while a client was adding a topic then instead of the client being notified with an exception the operation would never complete. This has been fixed.

FB-30611: If cluster topology changes during topic removal, topics can be left in a state where they can never be removed

If the cluster topology changes, that is, a server joins or leaves the cluster, in-flight operations can fail with a transient CLUSTER_REPARTITION error. Due to a bug introduced in Diffusion 6.5, topic removal operations that fail if the cluster topology changes can leave the topics in an invalid state such that they can never be removed.

The bug has been fixed in this release.

FB-30697: Failure to start connector because readiness conditions not triggered

Fixed an issue that could result in Diffusion instances joining a cluster failing to reach readiness.

FB-30701: Problems when restarting a cluster member with an out of date persistence file

Fixed an issue that could cause unexpected restarts when an instance with an out-of-date persistence file joined a cluster.

Security

FB-29728: Users with no permissions can view the server logs in the console

Users without adequate permissions were able to view server logs in the Diffusion Management Console. This has now been resolved.

FB-30250: A path permission assignment for a role would sometimes not override permissions inherited from a parent path

Due to a bug in previous releases, if a security role had path permission assignments for paths X and Y, where X is a parent path of Y, the path permission assignments for Y were sometimes not applied correctly.

The bug has been fixed in this release.

Server

FB-29670: Multiplexer stuck processing selection of a missing topic notification handler

Due to a concurrency bug in previous releases, the server could stall while trying to select a control session to handle a request.

The bug has been fixed in this release.

FB-30364: StringIndexOutOfBoundsException in PathSelector.confirmSelects

In certain circumstances, the Diffusion server could report a StringIndexOutOfBoundsException in PathSelector.confirmSelects. This has now been resolved.

Topic Views

FB-29995: Topic view INSERT can incorrectly insert the value of a topic with DONT_RETAIN_VALUE=true

Due to a bug in previous releases, a topic view INSERT transformation could insert a stale value of a stateless topic (that is, a topic with its DONT_RETAIN_VALUE property set to true). This would happen if the topic view was added after the stateless topic, and there were other topic views with topic selectors that referenced the stateless topic.

The bug has been fixed in this release. An INSERT transformation will never insert the value of a stateless topic.

FB-29999: Reference topics can incorrectly use the stale value of a topic with DONT_RETAIN_VALUE=true

Due to a bug in previous releases, topic view could create reference topics using a stale value of a stateless topic (that is, a topic with its DONT_RETAIN_VALUE property set to true). This would happen if the topic view was added after the stateless topic, and there were other topic views with topic selectors that referenced the stateless topic.

The bug has been fixed in this release.

FB-30686: Mapping single value topic to time series topic does not respect specified retained range

Using topic views to map a single value topic to a time series topic would always assume the default retained range (10) regardless of what was specified in the with properties clause. This was because the retained range for time series reference topics is always limited to the source retained range so that cross cluster inconsistencies do not occur. This has now been changed so that when mapping a single value topic to a time series topic any specified retained range is respected rather than assuming the default.
However, there is a separate known issue relating to this type mapping in that using a single value topic mapping to a time series target can lead to inconsistent views of the target topic events across a cluster and therefore such a mapping is not suitable for use in a clustered environment. Even in a single server environment it is worth noting that the retained event range of the target time series topic will not be restored in server restart.

Topics

FB-29971: Subscriptions fail to be re-evaluated if topic selections match topics in the source branch of a removed branch mapping

Due to a bug in previous releases, if a subscription to a topic was redirected by a branch mapping, and the branch mapping was removed, the session may not be resubscribed to a topic at the original topic path. The bug has been fixed in this release.

Known Issues

Gateway

DIF-4227: Details of the Passive Gateway Clients are not Available After Session Re-creation

After a server restart, the details of passive Gateway clients are lost. Once a Gateway client re-validates, the session information for the passive instance is stored; however, other required details are not set correctly, causing subsequent passive client validations to fail when they should not.
This issue is resolved in version 6.12 of the Diffusion server. To address this issue for adapters, they must be updated to use the latest version of the Gateway Framework, which includes version 6.12 of the Java client SDK, and should be used in conjunction with version 6.12 of the Diffusion server.

DIF-4242: Passive Gateway Client Validation Fails when JSON Configuration is Logically Equivalent but Structurally Different from the Active Client

When a passive Gateway client submits a validation request with a hash of its identifier that includes a JSON configuration containing logically similar but structurally different content compared to that of the active client, the validation would fail.

This issue is resolved in version 6.12 of the Diffusion server. To address this issue for adapters, they must be updated to use the latest version of the Gateway Framework, which includes the corresponding fix for this issue, and should be used in conjunction with version 6.12 of the Diffusion server.

Python Client

DIF-1928: Internal Type Conversion Code Allows Conversion to/from BinaryDataType, no IBytes Representation Available

This known issue is fixed in Diffusion 6.11 and later, which is a breaking change.

In clients prior to 6.11, the concept of an IBytes interface (representing data convertible to bytes and from any Diffusion data type, regardless of source/destination) has been conflated with the Binary data type, which is incorrect.

In practice, the places where this may affect the end user, requiring an application code change are as follows:

1. In a range query, resulting in no data being returned, as the server would reject a request for BinaryDataType data, as is the default type for a Range Query in diffusion.features.timeseries.TimeSeries.range_query for versions before 6.11 (https://docs.diffusiondata.com/docs/6.10.2/python/api/timeseries.html#diffusion.features.timeseries.TimeSeries.range_query). .

2. When subscribing to Binary data types, data may be converted to Binary currently, but will not in future as only Time Series of BinaryDataType will be converted to BinaryDataType in 6.11 and later (existing BinaryDataType streams were and will continue to be provided to subscribers when requested).

Topic Views

DIF-166: Reference Topics Retained by 'preserve topics' are not Persisted Across Server Instances or Clusters

A new 'preserve topics' clause was introduced to topic views in release 6.6. .

This clause means that reference topics created by a view (that have a path dependent upon the source topic value) are retained until the source topic is removed or the topic view is removed. .

Though this is true in the context of a single server instance, it is not the case if the server is restarted as all such topics created during the previous server instance will be lost. It is also not the case if a new server enters a cluster as the new server will only have reference topics generated from the point in time where it joined the cluster and will not reflect reference topics previously created within other cluster peers.

This issue occurs because reference topics are not persisted, either to file or across the cluster.

DIF-167: Restrictions on Mapping Single Value Topics to Time Series Reference Topics

The ability to specify a target type in a topic view was introduced in release 6.7.0.

When using this feature to map a single value topic to a time series topic there are the following restrictions.

1) The retained events in the target topic are not replicated across the cluster (as reference topics are not replicated). This means that a new server joining the cluster will not have the same number of retained events as other cluster members. For this reason mapping single value to time series topics should not be used in a clustered environment.

2) Retained events in the target topic are not persisted therefore when a server is restarted the target time series will initially start with a single event and will only grow as the source topic is updated.

The ability to map single-value topics to time series topics is deprecated at release 6.11 and an error message will be logged if used. If used in a single server a warning message will be logged.

DIF-2482: The Memory Overhead of Remote Topics is not Accounted for in Metrics and Fetched Topic Sizes

Remote Topic Views maintain local caches of the values of the topics from the primary servers, however, this memory overhead is not accounted for in the metrics relating to topic sizes and memory. This means that when using remote topic views, there is an 'invisible' memory overhead which amounts to the size of each topic selected by the remote topic views from the primary server. .

Furthermore, from release 6.11 it is also possible to 'fetch' the size of topics. For reference topics that are mapped from remote topics, a similar problem to the above occurs. A reference topic that is directly mapped from a remote topic will report its size as zero because under normal circumstances the size of such a reference topic is accounted for in the source topic. Similarly, a reference topic that is indirectly mapped from a remote topic will report its value size but will not consider the additional size of the source topic. The reference topic cannot simply add the size of the source topic as many reference topics can be mapped from the same remote topic and that would result in over accounting for the sizes.

Both of the above issues will be resolved in a future release.