Event streams often have an ordering which is important for a set of events for the same entity, but not for events of different entities. For example, consider events for a user management system, you might have user created and user deleted events. Here's an example stream:
eventID: 1, eventType: com.example.user.created, data: alice
eventID: 2, eventType: com.example.user.created, data: bob
eventID: 3, eventType: com.example.user.deleted, data: bob
Now in this case, it matters which order events 2 and 3 are handled in, since they are both for the user called bob, user bob cannot be deleted before he was created, logically that doesn't make sense. However, event 1 and 2 can be handled in any order, since they are for two users, alice and bob, it doesn't matter which order they are handled.
For performance and scalability reasons, many message transports, such as Kafka and Kinesis, implement partitioning of event streams. This partitioning however can cause events to be processed out of order. This could happen if two events that need to be processed in order are published to two different partitions. To ensure that message ordering is maintained, these transports typically implement a concept of message keys, where each message has a key associated with it that correlates to the entity that the event that the message carries is for, and all messages with the same key go to the same partition.
This key could be expressed in the CloudEvents spec using an optional eventKey field of type Object.
A use case where this might be helpful is if there was some middleware doing a generic transformation of events from one source to another, as it would allow the transformation to be done while maintaining the event ordering, without any knowledge of the domain data or underlying transports. The middleware would only need to ensure that the eventKey from incoming events was propagated to outgoing events.
Assigned to: @jroper
Event streams often have an ordering which is important for a set of events for the same entity, but not for events of different entities. For example, consider events for a user management system, you might have user created and user deleted events. Here's an example stream:
Now in this case, it matters which order events 2 and 3 are handled in, since they are both for the user called
bob, userbobcannot be deleted before he was created, logically that doesn't make sense. However, event 1 and 2 can be handled in any order, since they are for two users,aliceandbob, it doesn't matter which order they are handled.For performance and scalability reasons, many message transports, such as Kafka and Kinesis, implement partitioning of event streams. This partitioning however can cause events to be processed out of order. This could happen if two events that need to be processed in order are published to two different partitions. To ensure that message ordering is maintained, these transports typically implement a concept of message keys, where each message has a key associated with it that correlates to the entity that the event that the message carries is for, and all messages with the same key go to the same partition.
This key could be expressed in the CloudEvents spec using an optional
eventKeyfield of typeObject.A use case where this might be helpful is if there was some middleware doing a generic transformation of events from one source to another, as it would allow the transformation to be done while maintaining the event ordering, without any knowledge of the domain data or underlying transports. The middleware would only need to ensure that the
eventKeyfrom incoming events was propagated to outgoing events.Assigned to: @jroper